Ghosting image detection method and device and ghosting image detection system

Abstract

The invention discloses a ghosting detection method, a ghosting detection device and a ghosting detection system, and belongs to the field of optics. The ghost image detection method comprises the following steps: calculating at least one actual ghost image parameter based on an obtained actual main image and an actual ghost image; performing simulation imaging on the initial image based on module parameters of the target lens module to obtain a simulation main image, at least one simulation ghost image, at least one simulation ghost image parameter corresponding to the simulation ghost image and an imaging reason corresponding to each simulation ghost image; and matching the at least one actual ghost image parameter, the at least one simulation ghost image parameter and the imaging reason corresponding to each simulation ghost image, and determining the imaging reason corresponding to each actual ghost image. According to the ghost image detection method provided by the invention, the specific reasons generated by each ghost image can be quickly and accurately determined, the calculation efficiency and precision are relatively high, the operation is simple, and the use threshold is relatively low.

Description

Technical Field

[0001] The present application relates to the field of optics, and in particular to a ghost detection method, device and system. Background Art

[0002] The Pancake optical solution utilizes a folded optical path design. After the image source enters the semi-reflective, semi-transmissive beamsplitter, the light is repeatedly reflected and retraced between the lens, phase retarder, and reflective polarizer film before exiting the reflective polarizer film and entering the human eye. This multiple reflection of light can lead to significant ghost images. However, the lack of effective quantitative ghost image detection methods in related technologies makes ghost image detection difficult. Summary of the Invention

[0003] This application aims to solve at least one of the technical problems existing in the prior art. To this end, this application proposes a ghost detection method, device, and system that can quickly and accurately determine the specific cause of each ghost image, with high computational efficiency and accuracy, simple operation, and a low user threshold.

[0004] In a first aspect, the present application provides a ghost detection method, the method comprising:

[0005] Based on the acquired actual main image and actual ghost image, at least one actual ghost image parameter is calculated; the actual main image and the actual ghost image are images obtained by imaging the initial image captured by the image sensor through the target lens module; the actual ghost image parameter includes at least one of an actual ghost image intensity and an actual ghost image field angle;

[0006] Performing simulated imaging on the initial image based on the module parameters of the target lens module to obtain a simulated main image, at least one simulated ghost image, at least one simulated ghost image parameter corresponding to the simulated ghost image, and an imaging cause corresponding to each of the simulated ghost images; the simulated ghost image parameter including at least one of a simulated ghost image intensity and a simulated ghost image field angle;

[0007] The at least one actual ghost image parameter, the at least one simulated ghost image parameter, and the imaging cause corresponding to each simulated ghost image are matched to determine the imaging cause corresponding to each actual ghost image.

[0008] According to the ghost detection method of the present application, by matching the real ghost image parameters with the simulated ghost image parameters to obtain the cause corresponding to each actual ghost image based on the simulation results, the specific cause of each ghost image can be determined quickly and accurately. The calculation efficiency and accuracy are high, the operation is simple, and it has a low usage threshold.

[0009] According to one embodiment of the present application, matching the at least one actual ghost image parameter, the at least one simulated ghost image parameter, and the imaging cause corresponding to each simulated ghost image to determine the imaging cause corresponding to each actual ghost image includes:

[0010] Extracting a target simulated ghost image field angle from the at least one simulated ghost image field angle, the difference between which and the target actual ghost image field angle in the at least one actual ghost image field angle does not exceed a first target threshold, and determining a simulated ghost image corresponding to the target simulated ghost image field angle as a target simulated ghost image image;

[0011] The imaging cause corresponding to the target simulated ghost image is determined as the imaging cause corresponding to the target actual ghost image.

[0012] According to one embodiment of the present application, the simulated imaging of the initial image based on the module parameters of the target lens module to obtain a simulated main image, at least one simulated ghost image, at least one simulated ghost image parameter corresponding to the simulated ghost image, and an imaging reason corresponding to each of the simulated ghost images includes:

[0013] The module parameters corresponding to the target subassembly in the target lens module are input into the optical simulation module to obtain the simulated ghost image after the initial image output by the optical simulation module is imaged by the target subassembly, at least one simulated ghost image parameter corresponding to the simulated ghost image, and the imaging reason corresponding to each simulated ghost image.

[0014] According to one embodiment of the present application, the step of calculating at least one actual ghost image parameter based on the acquired actual main image and the actual ghost image includes:

[0015] Calibrate the field of view angle of the image sensor based on a target calibration plate to obtain a calibration relationship, where the calibration relationship is used to represent a correlation between the field of view angle and the pixel value;

[0016] The field angle of view of the ghost image features in the actual ghost image is calculated based on the calibration relationship to obtain the actual ghost image field angle.

[0017] According to one embodiment of the present application, the step of calculating at least one actual ghost image parameter based on the acquired actual main image and the actual ghost image includes:

[0018] Based on the real brightness corresponding to the center point of the target image in the actual main image and the actual ghost image collected by the spectrometer corresponding to the image sensor and the grayscale value corresponding to each pixel in the target image, the first brightness corresponding to each pixel in the target image is corrected; based on the first brightness corresponding to the ghost image feature in the actual ghost image and the first brightness corresponding to the main image feature in the actual main image, the actual ghost image intensity is determined.

[0019] According to one embodiment of the present application, after matching the at least one actual ghost image parameter, the at least one simulated ghost image parameter, and the imaging cause corresponding to each simulated ghost image to determine the imaging cause corresponding to each actual ghost image, the method further includes:

[0020] Based on the imaging reason, a process improvement strategy corresponding to the imaging reason is obtained; the improvement strategy is used to optimize the target lens module.

[0021] According to one embodiment of the present application, before calculating at least one actual ghost image parameter based on the acquired actual main image and actual ghost image, the method further includes:

[0022] The image sensor is controlled to collect images at multiple different focus distances, and collects the initial image after imaging by the target lens module.

[0023] In a second aspect, the present application provides a ghost detection device, the device comprising:

[0024] a first processing module, configured to calculate at least one actual ghost image parameter based on an acquired actual main image and an actual ghost image; the actual main image and the actual ghost image are images obtained by imaging an initial image captured by an image sensor through a target lens module; the actual ghost image parameter includes at least one of an actual ghost image intensity and an actual ghost image field angle;

[0025] a second processing module, configured to perform simulated imaging on the initial image based on the module parameters of the target lens module, to obtain a simulated main image, at least one simulated ghost image, at least one simulated ghost image parameter corresponding to the simulated ghost image, and an imaging cause corresponding to each of the simulated ghost images; the simulated ghost image parameter including at least one of a simulated ghost image intensity and a simulated ghost image field angle;

[0026] The third processing module is configured to match the at least one actual ghost image parameter, the at least one simulated ghost image parameter, and the imaging cause corresponding to each of the simulated ghost images to determine the imaging cause corresponding to each of the actual ghost images.

[0027] According to the ghost detection device of the present application, by matching the real ghost image parameters with the simulated ghost image parameters to obtain the cause corresponding to each actual ghost image based on the simulation results, the specific cause of each ghost image can be determined quickly and accurately. The calculation efficiency and accuracy are high, the operation is simple, and the usage threshold is low.

[0028] In a third aspect, the present application provides a ghost detection system, comprising:

[0029] An image sensor, the image sensor being arranged behind the target lens module and being used to capture an initial image formed by the target lens module;

[0030] As described in the ghost detection device of the second aspect, the ghost detection device is electrically connected to the image sensor.

[0031] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the ghost detection method as described in the first aspect above.

[0032] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the ghost detection method as described in the first aspect above.

[0033] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0034] By matching real ghost image parameters with simulated ghost image parameters to obtain the causes of each actual ghost image based on the simulation results, the specific cause of each ghost image can be quickly and accurately determined. The calculation efficiency and accuracy are high, the operation is simple, and the usage threshold is low.

[0035] Furthermore, after determining the cause of the ghost image, the process improvement strategy for improving the ghost image is automatically matched, so as to optimize the treatment of the ghost image phenomenon. There is no need for manual inspection, the detection efficiency and accuracy are high, and it has a high degree of automation. It does not require users to have professional knowledge and rich experience, effectively lowering the threshold for use.

[0036] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0038] Figure 1This is one of the flow charts of the ghost detection method provided in the embodiment of the present application;

[0039] Figure 2 This is one of the principle diagrams of the ghost detection method provided in the embodiment of the present application;

[0040] Figure 3 This is the second schematic diagram of the principle of the ghost detection method provided in the embodiment of the present application;

[0041] Figure 4 This is the third schematic diagram of the principle of the ghost detection method provided in the embodiment of the present application;

[0042] Figure 5 This is the second flow chart of the ghost detection method provided in the embodiment of the present application;

[0043] Figure 6 This is the third flow chart of the ghost detection method provided in the embodiment of the present application;

[0044] Figure 7 This is the fourth schematic diagram of the principle of the ghost detection method provided in the embodiment of the present application;

[0045] Figure 8 This is the fifth principle diagram of the ghost detection method provided in the embodiment of the present application;

[0046] Figure 9 This is the sixth schematic diagram of the principle of the ghost detection method provided in the embodiment of the present application;

[0047] Figure 10 This is the seventh schematic diagram of the principle of the ghost detection method provided in the embodiment of the present application;

[0048] Figure 11 is a schematic structural diagram of a ghost detection device provided in an embodiment of the present application;

[0049] Figure 12 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0051] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0052] The ghost detection method, ghost detection device, electronic device and readable storage medium provided by the embodiments of the present application are described in detail below with reference to specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0053] The ghost detection method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0054] The terminal includes but is not limited to portable communication devices such as mobile phones or tablet computers. It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer.

[0055] The ghost detection method provided in the embodiment of the present application can be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the ghost detection method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras and wearable devices, etc. The ghost detection method provided in the embodiment of the present application is explained below using an electronic device as an example of the execution subject.

[0056] like Figure 1 As shown, the ghost detection method includes: step 110, step 120 and step 130.

[0057] Step 110: Calculate at least one actual ghost image parameter based on the acquired actual main image and actual ghost image; the actual main image and actual ghost image are images obtained by imaging an initial image captured by the image sensor through a target lens module; the actual ghost image parameter includes at least one of an actual ghost image intensity and an actual ghost image field angle;

[0058] In this step, the main image is the corresponding image formed on the image plane or the designated plane by the main light emitted by the object following the laws of geometric optics.

[0059] Ghost is a "virtual image" formed at a certain position in the picture by stray light caused by multiple reflections on the lens interface, scattering from lens defects, and scattering from physical structures.

[0060] Ghost images may be superimposed on the original object or formed near the object, causing the edges of the object to be blurred or doubled.

[0061] The initial image can be Figure 2 The image shown is a black background circular image; of course, in other embodiments, the initial image may also be an image including other features, such as an image including features such as a rectangle or an ellipse, and this application does not limit this.

[0062] The image sensor is used to collect the virtual image generated after the target lens module performs an optical path action on the initial image.

[0063] Image sensors can include: a high-resolution camera with a wide-viewing conoscope lens, a spectrometer, and an autofocus system.

[0064] The high-resolution camera with a wide-viewing angle conoscope lens may be a camera with a human eye lens to simulate the images seen by the human eye.

[0065] The target lens module can be a Pancake optical lens, such as a Pancake head-mounted display device.

[0066] The target lens module includes multiple lenses, film layers and other optical devices.

[0067] like Figure 3 As shown, the target lens module may include: a display screen, a linear polarizer (LP), a quarter-wave plate (QWP), a beam splitter (50:50 BS), and a polarization beam splitter (PBS) arranged in sequence.

[0068] Among them, the polarization beam splitter is also a reflective polarizer.

[0069] The actual main image is an image including a relatively clear main image.

[0070] The actual ghost image is an image including relatively clear ghost images, and the number of the actual ghost images may be one or more.

[0071] Continue to refer Figure 3 In the actual implementation process, taking the Pancake head-mounted display device as an example, the following can be displayed on the display screen of the Pancake head-mounted display device: Figure 2The initial image is shown, and the image sensor is set on the side facing the initial image. The light path on the display screen is imaged through the various optical film layers in the Pancake head-mounted display device, and the virtual image generated is collected by the image sensor to obtain the actual main image and the actual ghost image.

[0072] The actual ghost image parameter includes at least one of an actual ghost image intensity and an actual ghost image field angle.

[0073] Actual ghost intensity is used to describe the brightness of ghost images.

[0074] In actual implementation, the field of view angle corresponding to the ghost image can be calculated based on the height and focal length of the ghost image.

[0075] In some embodiments, there is a matching relationship between the actual ghost image field angle and the actual pixel value corresponding to the pixel point in the actual ghost image.

[0076] In this embodiment, based on the matching relationship, the actual ghost image field angle can be determined according to the actual pixel values corresponding to the ghost images in the actual ghost image.

[0077] It should be noted that in the present application, during the application process, the image sensor does not capture the initial image itself, but rather the virtual image after imaging by the target lens module.

[0078] By pre-establishing the matching relationship between the field of view angle and the pixel, compared with the existing technology, in addition to obtaining the field of view angle corresponding to the real image, the field of view angle corresponding to the virtual image can also be further obtained with higher accuracy.

[0079] In some embodiments, step 110 may further include:

[0080] Calibrate the field of view angle of the image sensor based on the target calibration plate to obtain a calibration relationship, which is used to characterize the correlation between the field of view angle and the pixel value;

[0081] The field of view angle of the ghost image features in the actual ghost image is calculated based on the calibration relationship to obtain the actual ghost image field of view angle.

[0082] In this embodiment, the target calibration plate may be a calibration plate including a plurality of regular grids.

[0083] Each grid has a fixed grid width. The specific value of the grid width can be customized by the user. For example, the target calibration plate can be set to a 20×20 checkerboard with a length of 150 cm.

[0084] The calibration relationship is used to characterize the correlation between the field of view angle and the pixel value, and can be expressed as a curve graph showing the field of view angle changing with the pixel value.

[0085] In actual implementation, the image sensor may be calibrated in advance before capturing the initial image.

[0086] For example, an image sensor is used to photograph a target calibration plate to obtain a calibration image; the pixel coordinates of each grid in the calibration image are obtained; based on the pixel coordinates and the spatial physical coordinates corresponding to each grid, the distance from the image sensor to the target calibration plate is calculated; based on the ratio between the physical width of the grid and the distance from the image sensor to the target calibration plate, the field of view angle corresponding to each grid can be obtained; then the field of view angle corresponding to each grid is associated with the pixel value of the grid to obtain the calibration relationship.

[0087] After obtaining the calibration relationship, in the subsequent calculation of the actual ghost image field angle, the field angle corresponding to the actual pixel value corresponding to the ghost image feature in the actual ghost image collected can be obtained through the calibration relationship, thereby obtaining the actual ghost image field angle.

[0088] In some embodiments, the calibration relationship may also be written into the calculation code to perform angle calculations on the rings of the main image and the ghost image.

[0089] According to the ghost detection method provided in the embodiment of the present application, the image sensor is calibrated by a target calibration plate to obtain a calibration relationship, and the actual ghost image field angle is calculated based on the calibration relationship. The field angle of the image obtained by shooting the virtual image after the target lens module is acted on can be effectively calculated, the calculation efficiency is high, and the accuracy and precision of the calculation results are high.

[0090] In some embodiments, step 110 may further include:

[0091] Based on the actual main image and the actual ghost image captured by the spectrometer corresponding to the image sensor and the true brightness corresponding to the center point of the target image and the grayscale value corresponding to each pixel in the target image, a first brightness corresponding to each pixel in the target image is corrected;

[0092] The actual ghost image intensity is determined based on the first brightness corresponding to the ghost image feature in the actual ghost image and the first brightness corresponding to the main image feature in the actual main image.

[0093] In this embodiment, a spectrometer is used to collect real brightness.

[0094] The target image may be an actual main image, or may be any actual ghost image.

[0095] In the actual implementation process, an image sensor and a spectrometer can be used simultaneously, with the image sensor collecting grayscale values and the spectrometer collecting real brightness.

[0096] Taking the actual main image as an example, the true brightness of the center point of the actual main image is measured by a spectrometer. Based on the true brightness of the center point and the grayscale value of the center point, the brightness of each pixel in the entire actual main image is corrected to calculate the brightness value corresponding to each pixel (i.e., the first brightness).

[0097] After obtaining the first brightness corresponding to the actual main image and the first brightness corresponding to each actual ghost image, the first brightness of the ghost image and the main image are extracted to obtain the ghost image brightness and the main image brightness, and the ratio of the ghost image brightness to the main image brightness is determined as the actual ghost image intensity corresponding to the ghost image.

[0098] like Figure 5 As shown, in some embodiments, before step 110, the method may further include:

[0099] The image sensor is controlled to collect an initial image after imaging by a target lens module at multiple different focus distances.

[0100] In this embodiment, when a picture such as Figure 2 In the case of the black background donut diagram shown in FIG, the main image and the ghost image are focused clearly by autofocus, so that the main image and the ghost image in the head display are captured by the image sensor at different focus distances, as shown in FIG. Figure 4 As shown, Figure 4 (a) is the actual main image captured by focusing on the main image, Figure 4 (b) to Figure 4 (d) are the actual ghost images captured by focusing on different ghost images (ghost image 1, ghost image 2 and ghost image 3).

[0101] Step 120: Performing simulated imaging on the initial image based on the module parameters of the target lens module to obtain a simulated main image, at least one simulated ghost image, at least one simulated ghost image parameter corresponding to the simulated ghost image, and an imaging cause corresponding to each simulated ghost image; the simulated ghost image parameter includes at least one of a simulated ghost image intensity and a simulated ghost image field angle;

[0102] In this step, the module parameters may include lens parameters and film layer parameters.

[0103] The simulated ghost image parameters may include at least one of a simulated ghost image intensity and a simulated ghost image field angle.

[0104] After obtaining the simulated ghost image and the simulated main image, the ratio of the ghost image brightness in the simulated ghost image to the main image brightness in the simulated main image can be determined as the simulated ghost image intensity corresponding to the ghost image.

[0105] The specific calculation method is similar to the calculation method of the actual ghost image intensity, and is not described in detail in this application.

[0106] Continue to refer Figure 5 In some embodiments, step 120 may include:

[0107] The module parameters corresponding to the target subassembly in the target lens module are input into the optical simulation module to obtain a simulated ghost image after the initial image output by the optical simulation module is imaged by the target subassembly, at least one simulated ghost image parameter corresponding to the simulated ghost image, and the imaging reason corresponding to each simulated ghost image.

[0108] In this embodiment, the target subassembly may be any one or more lenses or film layers included in the lens module.

[0109] The optical simulation module can be any optical simulation software.

[0110] Taking Zemax optical simulation software as an example, during the simulation process, the parameters of each lens and each film layer of the pancake lens can be substituted into the Zemax optical simulation software;

[0111] Then, the light path of each film layer is traced to calculate the field angle of the main image (imaging by the main light path) (i.e., the simulated main image field angle) and the field angle of the ghost image (imaging by other film layers) (i.e., the simulated ghost image field angle).

[0112] It can be understood that, based on tracing the light paths of each film layer, the relevant information of the film layer corresponding to each ghost image can be effectively tracked, thereby determining the imaging cause corresponding to each simulated ghost image.

[0113] According to the ghost detection method provided in the embodiment of the present application, by performing optical simulation on the same initial image based on the module parameters of the target lens module, the simulated main image parameters, simulated ghost image parameters and the corresponding causes of each simulated ghost image can be obtained quickly and accurately, which is helpful for subsequent matching with the measured data.

[0114] Step 130: Match at least one actual ghost image parameter, at least one simulated ghost image parameter, and the imaging cause corresponding to each simulated ghost image to determine the imaging cause corresponding to each actual ghost image.

[0115] In this step, different ghost images may have different causes.

[0116] Continue to refer Figure 3 , Figure 3 Three ghost images and their corresponding optical paths are illustrated. The dotted lines represent the optical path schematic diagram of the process of each ghost image being generated. It can be seen that for different ghost images, the lenses or film layers involved in generating them may be different.

[0117] In this application, by matching the actual ghost image parameters determined by actual measurement with the simulated ghost image parameters obtained through optical path simulation, the lens or film layer involved in generating each actual ghost image can be determined, thereby determining the imaging cause corresponding to the actual ghost image.

[0118] During the research and development process, the inventors discovered that in the relevant technology, there is a method of processing the acquired imaging image including the main light spot and stray light spots, and obtaining the spatial angular distribution of the stray light spots and the light intensity at the corresponding angle based on the optical path characteristics. However, this method cannot accurately detect the specific causes of each light spot or stray light spot, and the algorithm involved is relatively cumbersome.

[0119] In the present application, the actual image obtained by image acquisition of the initial image after module processing of the target lens module is calculated to obtain the actual main image parameters and the actual ghost image parameters, and then the same initial image is optically simulated based on the module parameters of the target lens module to obtain the simulated main image parameters, simulated ghost image parameters and the causes corresponding to each simulated ghost image, and the simulated ghost image and the actual ghost image are matched based on the parameters to determine the causes corresponding to the actual ghost image based on the matching results. The operation is simple and convenient, and the accuracy of the final result is high.

[0120] According to the ghost detection method provided in the embodiment of the present application, by matching the real ghost image parameters with the simulated ghost image parameters to obtain the cause corresponding to each actual ghost image based on the simulation results, the specific cause of each ghost image can be determined quickly and accurately. The calculation efficiency and accuracy are high, the operation is simple, and it has a low usage threshold.

[0121] The implementation of step 130 is described in detail below.

[0122] like Figure 6 As shown, in some embodiments, step 130 may include:

[0123] Extracting a target simulated ghost image field angle from the at least one simulated ghost image field angle, the difference between which and the target actual ghost image field angle in the at least one actual ghost image field angle does not exceed a first target threshold, and determining a simulated ghost image image corresponding to the target simulated ghost image field angle as a target simulated ghost image image;

[0124] The imaging cause corresponding to the target simulated ghost image is determined as the imaging cause corresponding to the target actual ghost image.

[0125] In this embodiment, the target actual ghost image field angle may be any field angle among the at least one actual ghost image field angle.

[0126] The first target threshold is a smaller range, and the specific value can be customized by the user.

[0127] The difference can be expressed as a difference or a ratio, etc., which is not limited in this application.

[0128] When the difference does not exceed the first target threshold, it can be approximately considered that the target actual ghost image field angle is consistent with the target simulated ghost image field angle, and thus the actual ghost image corresponding to the target actual ghost image field angle and the ghost image corresponding to the target simulated ghost image field angle are considered to be the same. Then the causes of the two are approximately considered to be the same, and the simulated ghost image image corresponding to the target simulated ghost image field angle can be determined as the target simulated ghost image image to match the target simulated ghost image image with the target actual ghost image image.

[0129] Continue to refer Figure 6 During the simulation process, the lens or module involved in each simulated ghost image (i.e., the imaging cause) can be obtained. After matching the simulated ghost image with the actual ghost image one by one, for any actual ghost image, the lens or module involved in generating the actual ghost image can be determined based on its corresponding simulated ghost image, thereby obtaining the corresponding cause of the actual ghost image.

[0130] The following uses Pancake VR lens as an example to illustrate the specific implementation method.

[0131] 1. Display in Pancake VR headset Figure 2 The donut diagram shown above shows the display image captured by the human eye lens camera in the head-mounted display. The test results at different focus distances are as follows: Figure 4 As shown;

[0132] Through the algorithm Figure 4 The angle and intensity of the main image and ghost image in the actual main image and actual ghost image are extracted respectively, and the following results are obtained:

[0133] Main image: angle D0, intensity I0;

[0134] Ghost image 1: angle Dgh1, intensity Igh1;

[0135] Ghost image 2: angle Dgh2, intensity Igh2;

[0136] Ghost image 3: angle Dgh3, intensity Igh3;

[0137] The positional relationship between each ghost image and the main image in the test results is shown in Table 1:

[0138] Table 1

[0139]

[0140] Table 1 illustrates actual ghost image parameters and actual main image parameters.

[0141] Then the Pancake VR lens is simulated, and the simulation results are as follows:

[0142] 1) For the main image

[0143] The light path diagram during the main image simulation process is as follows Figure 7 shown.

[0144] The main image is the image with the highest brightness, and its test results (including the actual main image field of view) are basically consistent with the simulation results (including the simulated main image field of view).

[0145] Test results: The actual field of view of the main image is 76.86°. Figure 4 (a)

[0146] Simulation results: The angle of the simulated main image is 76.9°( Figure 7 The angle shown is half of the field of view), and the simulated main image is at 1.6m.

[0147] 2) For ghost image 1

[0148] The optical path diagram during the ghost image 1 simulation process is as follows Figure 8 shown.

[0149] The field of view (FOV) of ghost image 1 (Derect ghost) is slightly smaller than that of the main image, and its size is also slightly smaller than that of the main image. The test results are basically consistent with the simulation results.

[0150] Test results: The actual field of view of ghost image 1 is 169.94°. Figure 4 (b)

[0151] Simulation results: The field of view of the simulated ghost image 1 is 68°, and the simulated ghost image 1 is at 41.8mm.

[0152] 3) For Ghost Image 2

[0153] The light path diagram during the ghost image 2 simulation process is as follows Figure 9 shown.

[0154] Ghost image 2 (the reflection of the BS film between the Display and Plastic_S1 surfaces) has a FOV of 17.2°, which is approximately 58% of the main image. The test results are basically consistent with the simulation results.

[0155] Test results: The actual field of view of ghost image 2 is 43°. Figure 4 (c)

[0156] Simulation results: The field of view of simulated ghost image 2 is 42.2°, at 68.5mm.

[0157] 4) For ghost image 3

[0158] The light path diagram during the ghost image 3 simulation process is as follows Figure 10 shown.

[0159] Ghost image 3 (reflection from the RP film between the display and Glass_S2) has a FOV of 17.04°, which is comparable to Ghost 3. The test results are basically consistent with the simulation results.

[0160] Test results: The actual field of view of ghost image 3 is 44.06°. Figure 4 (d)

[0161] Simulation results: The field of view of simulated ghost image 3 is 43° at 163mm.

[0162] The positional relationship between each ghost image and the main image in the simulation results is shown in Table 2:

[0163] Table 2

[0164]

[0165] Through matching, it can be found that ghost image 1 is directly formed by the display screen; ghost image 2 is caused by the reflection between the display and the BS film on the Plastic_S1 surface; ghost image 3 is caused by the reflection between the display and the RP film on the Glass_S2 surface.

[0166] The optical simulation virtual image distance is basically close to the actual measured distance of the image sensor, and the optical simulation angle corresponds to the actual measured result.

[0167] Continue to refer Figure 5 In some embodiments, after step 130, the method may further include:

[0168] Based on the imaging reasons, a process improvement strategy corresponding to the imaging reasons is obtained; the improvement strategy is used to optimize the target lens module.

[0169] In this embodiment, the improvement strategy is used to optimize the target lens module.

[0170] For different imaging reasons, the lenses or film layers involved may be different, and the corresponding process improvement strategies may also be different.

[0171] During the actual implementation process, the corresponding relationship between each imaging cause and the process improvement strategy can be established in advance, so that in the subsequent application process, the corresponding process improvement strategy can be matched directly based on the imaging cause.

[0172] Let’s continue with the Pancake VR lens as an example.

[0173] After obtaining the results shown in Tables 1 and 2 above, we further matched the corresponding process improvement strategies based on the causes of each ghost image.

[0174] For example, if the test system output includes the following data:

[0175] Ghost image 1 (Direct ghost) has a high intensity;

[0176] Direct ghost causes:

[0177] 1. The relative angle between QWP2, which is close to the screen, and the screen is large;

[0178] 2. The relative angle between QWP2 close to the screen and QWP1 on the lens surface is large;

[0179] Based on the cause and intensity of ghost image 1, the process improvement strategy can be matched as follows:

[0180] The process improvement strategy for ghost image 1 (Direct ghost) is:

[0181] 1. The QWP2 optical axis is 45±0.5° to the screen Y axis;

[0182] 2. The optical axis of QWP2 is aligned with the optical axis of QWP1, and the specification requirement is <0.5°.

[0183] According to the ghost detection method provided in the embodiment of the present application, after determining the corresponding cause of the ghost image, the process improvement strategy for improving the ghost image is automatically matched, so as to optimize the processing of the ghost image phenomenon. No manual detection is required, the detection efficiency and accuracy are high, and it has a high degree of automation. It does not require users to have professional knowledge and rich experience, effectively lowering the threshold for use.

[0184] The ghost detection method provided in the embodiment of the present application can be executed by a ghost detection device. In the embodiment of the present application, the ghost detection device provided in the embodiment of the present application is described by taking the ghost detection method performed by the ghost detection device as an example.

[0185] An embodiment of the present application also provides a ghost detection device.

[0186] like Figure 11 As shown, the ghost detection device includes: a first processing module 1110 , a second processing module 1120 and a third processing module 1130 .

[0187] A first processing module 1110 is configured to calculate at least one actual ghost image parameter based on the acquired actual main image and actual ghost image image; the actual main image and actual ghost image image are images obtained by imaging an initial image captured by the image sensor through a target lens module; the actual ghost image parameter includes at least one of an actual ghost image intensity and an actual ghost image field of view angle;

[0188] The second processing module 1120 is configured to perform simulated imaging on the initial image based on the module parameters of the target lens module to obtain a simulated main image, at least one simulated ghost image, at least one simulated ghost image parameter corresponding to the simulated ghost image, and an imaging cause corresponding to each simulated ghost image; the simulated ghost image parameter includes at least one of a simulated ghost image intensity and a simulated ghost image field angle;

[0189] The third processing module 1130 is configured to match at least one actual ghost image parameter, at least one simulated ghost image parameter, and the imaging cause corresponding to each simulated ghost image to determine the imaging cause corresponding to each actual ghost image.

[0190] According to the ghost detection device provided in the embodiment of the present application, by matching real ghost image parameters with simulated ghost image parameters to obtain the cause corresponding to each actual ghost image based on the simulation results, the specific cause of each ghost image can be quickly and accurately determined. The calculation efficiency and accuracy are high, the operation is simple, and it has a low usage threshold.

[0191] In some embodiments, the third processing module 1130 may also be used to:

[0192] Extracting a target simulated ghost image field angle from the at least one simulated ghost image field angle, the difference between which and the target actual ghost image field angle in the at least one actual ghost image field angle does not exceed a first target threshold, and determining a simulated ghost image image corresponding to the target simulated ghost image field angle as a target simulated ghost image image;

[0193] The imaging cause corresponding to the target simulated ghost image is determined as the imaging cause corresponding to the target actual ghost image.

[0194] In some embodiments, the second processing module 1120 may also be used to:

[0195] The module parameters corresponding to the target subassembly in the target lens module are input into the optical simulation module to obtain a simulated ghost image after the initial image output by the optical simulation module is imaged by the target subassembly, at least one simulated ghost image parameter corresponding to the simulated ghost image, and the imaging reason corresponding to each simulated ghost image.

[0196] In some embodiments, the first processing module 1110 may also be used to:

[0197] Calibrate the field of view angle of the image sensor based on the target calibration plate to obtain a calibration relationship, which is used to characterize the correlation between the field of view angle and the pixel value;

[0198] The field of view angle of the ghost image features in the actual ghost image is calculated based on the calibration relationship to obtain the actual ghost image field of view angle.

[0199] In some embodiments, the first processing module 1110 may also be used to:

[0200] Based on the actual main image and the actual ghost image captured by the spectrometer corresponding to the image sensor and the true brightness corresponding to the center point of the target image and the grayscale value corresponding to each pixel in the target image, a first brightness corresponding to each pixel in the target image is corrected;

[0201] Determine the actual ghost intensity based on the first brightness corresponding to the ghost image feature in the actual ghost image and the first brightness corresponding to the main image feature in the actual main image

[0202] In some embodiments, the apparatus may further include a fourth processing module configured to:

[0203] After matching at least one actual ghost image parameter, at least one simulated ghost image parameter and the imaging cause corresponding to each simulated ghost image, and determining the imaging cause corresponding to each actual ghost image, a process improvement strategy corresponding to the imaging cause is obtained based on the imaging cause; the improvement strategy is used to optimize the target lens module.

[0204] In some embodiments, the device may further include a fifth processing module for controlling the image sensor to capture the initial image after imaging by the target lens module at multiple different focusing distances before calculating at least one actual ghost image parameter based on the acquired actual main image and actual ghost image.

[0205] The ghost detection device in the embodiment of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a car-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application does not make specific limitations.

[0206] The ghost detection device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0207] The ghost detection device provided in the embodiment of the present application can achieve Figures 1 to 10 To avoid repetition, the various processes implemented in the method embodiment are not described here.

[0208] An embodiment of the present application also provides a ghost detection system, including: an image sensor and a ghost detection device.

[0209] In this embodiment, the image sensor is used to be arranged behind the target lens module, such as Figure 3 As shown, it is used to capture the initial image after imaging by the target lens module.

[0210] Image sensors can include: a high-resolution camera with a wide-viewing conoscope lens, a spectrometer, and an autofocus system.

[0211] The high-resolution camera with a wide-viewing angle conoscope lens may be a camera with a human eye lens to simulate the images seen by the human eye.

[0212] In some embodiments, the resolution of the high-resolution camera of the wide-view conoscope lens can be greater than or equal to 6024×4024, the lens FOV is greater than or equal to 120°×80°, the angular resolution is greater than or equal to 50PPD, the 4mm entrance pupil front design simulates the human eye pupil, has a refractive power of 0 to 4D electric focus, and integrates a high-sensitivity, high dynamic range spectroradiometer, which couples the overall image brightness and color of the lens test.

[0213] The image sensor can automatically focus on the main image and ghost image at different focus levels to capture the ghost image and main image respectively; the spectroradiometer performs brightness and color correction on each pixel of the test image (i.e., the initial image) to derive the brightness and color of the captured image.

[0214] The ghost detection device is electrically connected to the image sensor.

[0215] The ghost detection device is used to execute the ghost detection method described in any of the above embodiments.

[0216] According to the ghost detection system provided in the embodiment of the present application, by matching real ghost image parameters with simulated ghost image parameters to obtain the cause corresponding to each actual ghost image based on the simulation results, the specific cause of each ghost image can be determined quickly and accurately. The calculation efficiency and accuracy are high, the operation is simple, and the usage threshold is low.

[0217] In some embodiments, the ghost detection device may further include: a control code module and an output module.

[0218] In this embodiment, the control code module encapsulates optical simulation software and image calculation code.

[0219] The control code module is electrically connected to the image sensor, and is used to receive the actual main image and the actual ghost image captured by the image sensor, and calculate the actual ghost image parameters and the actual main image parameters based on the actual main image and the actual ghost image.

[0220] The optical simulation software is used to simulate imaging of the initial image based on the module parameters of the target lens module to obtain a simulated main image, at least one simulated ghost image, and at least one simulated ghost image parameter corresponding to the simulated ghost image.

[0221] The output module is electrically connected to the control code module, and is used to match at least one actual ghost image parameter, at least one simulated ghost image parameter, and the imaging cause corresponding to each simulated ghost image, determine the imaging cause corresponding to each actual ghost image, and output the imaging cause;

[0222] In some embodiments, the output module may be further configured to further match and obtain a corresponding process improvement strategy based on the imaging reason, and output the process improvement strategy.

[0223] In some embodiments, the output module can output the results on the PC.

[0224] According to the ghost detection system provided in the embodiment of the present application, after determining the cause of the ghost image, it automatically matches the process improvement strategy for improving the ghost image, so as to optimize the processing of the ghost image phenomenon. It does not require manual inspection, and the detection efficiency and accuracy are high. It has a high degree of automation and does not require users to have professional knowledge and rich experience, effectively lowering the threshold for use.

[0225] In some embodiments, as Figure 12 As shown, an embodiment of the present application also provides an electronic device 1200, including a processor 1201, a memory 1202, and a computer program stored in the memory 1202 and executable on the processor 1201. When the program is executed by the processor 1201, each process of the above-mentioned ghost detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0226] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0227] An embodiment of the present application also provides a non-transitory computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various processes of the above-mentioned ghost detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0228] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0229] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned ghost detection method when executed by a processor.

[0230] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0231] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned ghost detection method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0232] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0233] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0234] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0235] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

[0236] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0237] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A ghost detection method, characterized in that: include: Based on the acquired actual main image and the actual ghost image, at least one actual ghost image parameter is calculated; The actual main image and the actual ghost image are images obtained by imaging the initial image captured by the image sensor through the target lens module; The actual ghost image parameter includes at least one of an actual ghost image intensity and an actual ghost image field angle; Performing simulated imaging on the initial image based on the module parameters of the target lens module to obtain a simulated main image, at least one simulated ghost image, at least one simulated ghost image parameter corresponding to the simulated ghost image, and an imaging reason corresponding to each of the simulated ghost images; The simulated ghost image parameters include at least one of simulated ghost image intensity and simulated ghost image field angle; The at least one actual ghost image parameter, the at least one simulated ghost image parameter, and the imaging cause corresponding to each simulated ghost image are matched to determine the imaging cause corresponding to each actual ghost image.

2. The ghost detection method according to claim 1, wherein: The matching of the at least one actual ghost image parameter, the at least one simulated ghost image parameter, and the imaging cause corresponding to each simulated ghost image to determine the imaging cause corresponding to each actual ghost image includes: Extracting a target simulated ghost image field angle from the at least one simulated ghost image field angle, the difference between which and the target actual ghost image field angle in the at least one actual ghost image field angle does not exceed a first target threshold, and determining a simulated ghost image corresponding to the target simulated ghost image field angle as a target simulated ghost image image; The imaging cause corresponding to the target simulated ghost image is determined as the imaging cause corresponding to the target actual ghost image.

3. The ghost detection method according to claim 1, wherein: The method of performing simulated imaging on the initial image based on the module parameters of the target lens module to obtain a simulated main image, at least one simulated ghost image, at least one simulated ghost image parameter corresponding to the simulated ghost image, and an imaging reason corresponding to each simulated ghost image includes: The module parameters corresponding to the target subassembly in the target lens module are input into the optical simulation module to obtain the simulated ghost image after the initial image output by the optical simulation module is imaged by the target subassembly, at least one simulated ghost image parameter corresponding to the simulated ghost image, and the imaging reason corresponding to each simulated ghost image.

4. The ghost detection method according to any one of claims 1 to 3, characterized in that: The step of calculating at least one actual ghost image parameter based on the acquired actual main image and the actual ghost image includes: Calibrate the field of view angle of the image sensor based on a target calibration plate to obtain a calibration relationship, where the calibration relationship is used to represent a correlation between the field of view angle and the pixel value; The field angle of view of the ghost image features in the actual ghost image is calculated based on the calibration relationship to obtain the actual ghost image field angle.

5. The ghost detection method according to any one of claims 1 to 3, characterized in that: The step of calculating at least one actual ghost image parameter based on the acquired actual main image and the actual ghost image includes: Based on the actual brightness corresponding to the center point of the target image in the actual main image and the actual ghost image collected by the spectrometer corresponding to the image sensor and the grayscale value corresponding to each pixel in the target image, correct and obtain a first brightness corresponding to each pixel in the target image; The actual ghost image intensity is determined based on a first brightness corresponding to a ghost image feature in the actual ghost image and a first brightness corresponding to a main image feature in the actual main image.

6. The ghost detection method according to any one of claims 1 to 3, characterized in that: After matching the at least one actual ghost image parameter, the at least one simulated ghost image parameter, and the imaging cause corresponding to each simulated ghost image to determine the imaging cause corresponding to each actual ghost image, the method further includes: Based on the imaging reason, a process improvement strategy corresponding to the imaging reason is obtained; the improvement strategy is used to optimize the target lens module.

7. The ghost detection method according to any one of claims 1 to 3, characterized in that: Before calculating at least one actual ghost image parameter based on the acquired actual main image and actual ghost image, the method further includes: The image sensor is controlled to collect images at multiple different focus distances, and collects the initial image after imaging by the target lens module.

8. A ghost detection device, characterized in that: include: A first processing module is configured to calculate at least one actual ghost image parameter based on the acquired actual main image and the actual ghost image; The actual main image and the actual ghost image are images obtained by imaging the initial image captured by the image sensor through the target lens module; The actual ghost image parameter includes at least one of an actual ghost image intensity and an actual ghost image field angle; a second processing module, configured to perform simulated imaging on the initial image based on the module parameters of the target lens module, to obtain a simulated main image, at least one simulated ghost image, at least one simulated ghost image parameter corresponding to the simulated ghost image, and an imaging reason corresponding to each of the simulated ghost images; The simulated ghost image parameters include at least one of simulated ghost image intensity and simulated ghost image field angle; The third processing module is configured to match the at least one actual ghost image parameter, the at least one simulated ghost image parameter, and the imaging cause corresponding to each of the simulated ghost images to determine the imaging cause corresponding to each of the actual ghost images.

9. A ghost detection system, characterized in that: include: An image sensor, the image sensor being arranged behind the target lens module and being used to capture an initial image formed by the target lens module; The ghost detection device according to claim 8, wherein the ghost detection device is electrically connected to the image sensor.

10. The ghost detection system according to claim 9, wherein: The ghost detection device comprises: A control code module encapsulating optical simulation software and image calculation code; the control code module is electrically connected to the image sensor; An output module is electrically connected to the control code module.

Images