Binocular eye box space area position detection method, device and equipment, storage medium and program product

Abstract

The embodiment of the invention provides a binocular eye box space area position detection method and device, equipment, a storage medium and a program product. The method comprises the following steps: respectively acquiring a preset image projected by each display module in binocular display modules through a sampling device, and obtaining a target detection image corresponding to each display module; the sampling device is located on the boundary of an eye box space area corresponding to each display module; performing boundary detection processing on each target detection image to obtain eye box boundary information corresponding to each display module; and determining a consistency detection result between the two eye box space areas based on a boundary comparison result between the two pieces of eye box boundary information. According to the embodiment of the invention, the consistency detection of the positions of the spatial regions of the binocular eye box can be realized, so that the problem that the positions of the spatial regions of the binocular eye box are inconsistent can be found and solved in the early stage of production, the product quality can be improved, and the visual experience of a user is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a binocular eye box space area position detection method, device, equipment, storage medium and program product. Background Art

[0002] Due to the actual processing deviation of the diffraction optical waveguide and the coupling deviation during the assembly process of the projector and the diffraction optical waveguide, after the binocular display module is assembled, the spatial area position of the left eye box and the spatial area position of the right eye box in the Cartesian coordinate system are inconsistent, resulting in the inability of both eyes to receive the ideal target detection image at the eye point position, causing a bad experience.

[0003] Since it is difficult to determine the position of the eye box space area, the actual production process does not detect and determine the position of the binocular eye box space area. After the whole machine is assembled, the actual human eye experience process finds inconsistencies, which makes it difficult to return for repair and even causes the possibility of material loss. Therefore, before actual application, a method for consistency detection of the binocular eye box space area position is urgently needed. Summary of the invention

[0004] The present invention provides a binocular eye box space area position detection method, device, equipment, storage medium and program product, which can realize the consistency detection of the binocular eye box space area position, so that the problem of inconsistent binocular eye box space area position can be discovered and solved in the early stage of production, thereby improving product quality and enhancing user visual experience.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for detecting the position of a binocular eye box space area, comprising: using a sampling device to respectively collect preset images projected by each display module in a binocular display module, and obtain a target detection image corresponding to each of the display modules; the sampling device is located on the boundary of the eye box space area corresponding to each of the display modules; performing boundary detection processing on each of the target detection images, and obtaining eye box boundary information corresponding to each of the display modules; and determining a consistency detection result between the two eye box space areas based on a boundary comparison result between the two eye box boundary information.

[0006] In the second aspect, the embodiments of the present disclosure also provide a binocular eye box space area position detection device, including: an acquisition module, used to respectively acquire preset images projected by each display module in the binocular display module through a sampling device, and obtain a target detection image corresponding to each of the display modules; the sampling device is located on the boundary of the eye box space area corresponding to each of the display modules; a processing module, used to perform boundary detection processing on each of the target detection images, and obtain eye box boundary information corresponding to each of the display modules; a consistency detection module, used to determine the consistency detection result between the two eye box space areas based on the boundary comparison result between the two eye box boundary information.

[0007] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the binocular eye box spatial area position detection method as described in the embodiment of the present disclosure.

[0008] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the binocular eye box spatial area position detection method as described in the embodiments of the present disclosure.

[0009] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product, including a computer program, which, when executed by a processor, implements the binocular eye box spatial area position detection method as described in the embodiments of the present disclosure.

[0010] The technical solution of the disclosed embodiment is to respectively collect preset images projected by each display module in the binocular display module through a sampling device to obtain a target detection image corresponding to each of the display modules; the sampling device is located on the boundary of the eye box space area corresponding to each of the display modules; a boundary detection process is performed on each of the target detection images to obtain the eye box boundary information corresponding to each of the display modules; and a consistency detection result between the two eye box space areas is determined based on the boundary comparison result between the two eye box boundary information. The disclosed embodiment is to respectively collect preset images in the binocular display module through a sampling device, perform boundary detection processing on each of the target detection images, and determine the consistency detection result between the two eye box space areas based on the boundary comparison result of the two eye box boundary information, so as to realize the consistency detection of the binocular eye box space area position, so as to discover and solve the problem of inconsistent binocular eye box space area position in the early stage of production, thereby improving product quality and enhancing user visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0012] Figure 1 A schematic flow chart of a binocular eye box spatial area position detection method provided by an embodiment of the present invention;

[0013] Figure 2 A schematic diagram of a binocular display module and a sampling device provided in an embodiment of the present invention;

[0014] Figure 3 A schematic diagram of the consistent positions of binocular eye box space areas provided by an embodiment of the present invention;

[0015] Figure 4 A schematic diagram of the inconsistency of the spatial area positions of the binocular eye boxes provided by an embodiment of the present invention;

[0016] Figure 5 A schematic diagram of a rectangular grating provided in an embodiment of the present invention;

[0017] Figure 6 A schematic diagram of a blazed grating provided in an embodiment of the present invention;

[0018] Figure 7 A schematic diagram of a tilted grating provided in an embodiment of the present invention;

[0019] Figure 8 A schematic diagram of a diffractive optical waveguide processing process provided by an embodiment of the present invention;

[0020] Fig. 9 A schematic diagram of coupling a projector and a diffractive optical waveguide provided in an embodiment of the present invention;

[0021] Fig.10 A schematic diagram of coupling between another projector and a diffractive optical waveguide provided in an embodiment of the present invention;

[0022] Fig.11 A schematic diagram of the structure of a binocular eye box space area position detection device provided by an embodiment of the present disclosure;

[0023] Fig.12 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0025] It should be understood that the various steps recorded in the method implementation of the present disclosure can be performed in different orders and / or in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect. The term "including" and its variations used herein are open inclusions, that is, "including but not limited to". It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more". It is understandable that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of the corresponding laws, regulations and relevant provisions.

[0026] Figure 1 The present invention provides a flow chart of a binocular eye box spatial area position detection method. The present invention is applicable to the situation where the consistency of the binocular eye box spatial area position is detected, and can be applied to related products such as augmented reality (AR) glasses based on diffraction schemes. The method can be executed by a binocular eye box spatial area position detection device, which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, which can be a mobile terminal, a PC, or a server. Figure 1 As shown, the method includes:

[0027] S110, respectively collecting the preset images projected by each display module in the binocular display module through a sampling device to obtain a target detection image corresponding to each of the display modules.

[0028] Wherein, the sampling device is located on the boundary of the eye box space area corresponding to each display module.

[0029] Wherein, the binocular display module includes two display modules, which can be respectively recorded as a left-eye display module and a right-eye display module; the sampling device can collect a preset image projected by the left-eye display module to obtain a target detection image corresponding to the left-eye display module. The sampling device can also collect a preset image projected by the right-eye display module to obtain a target detection image corresponding to the right-eye display module.

[0030] Optionally, the sampling device is used to collect the preset images projected by each display module in the binocular display module to obtain the target detection image corresponding to each display module, including: projecting the preset image to the corresponding diffraction light waveguide coupling-in area through each projector, and transmitting the preset image to the diffraction light waveguide coupling-in area through the diffraction light waveguide coupling-in area; and collecting the preset image projected by each diffraction light waveguide coupling area to the corresponding eye box space area from the humanoid eye lens through the image sensor to obtain the target detection image corresponding to each display module.

[0031] The sampling device includes a humanoid eye lens and an image sensor; each display module includes a projector, a diffraction light waveguide coupling-in region and a diffraction light waveguide coupling-out region.

[0032] like Figure 2 As shown, Figure 2 Schematic diagram of a binocular display module and a sampling device provided in an embodiment of the present invention. Figure 2 In the embodiment, the right-eye display module includes a right-eye projector 1, a right-eye diffraction light waveguide coupling-in region 21, and a right-eye diffraction light waveguide coupling-out region 22. The right-eye diffraction light waveguide 2 may include a right-eye diffraction light waveguide coupling-in region 21 and a right-eye diffraction light waveguide coupling-out region 22. Figure 2 The right eye box space area 23 is also shown. Both the left eye display module and the right eye display module can have a sampling device, and the sampling device includes a human eye lens 3 and an image sensor 4. The sampling device corresponding to the right eye display module is located on the boundary of the right eye box space area 23 corresponding to the right eye display module. After the sampling device collects the target detection image corresponding to the right eye display module, the sampling device can also be moved to the boundary of the left eye box space area 63 corresponding to the left eye display module, so that the target detection image corresponding to the left eye display module can also be collected. The left eye display module includes a left eye projector 5, a left eye diffraction light waveguide coupling-in area 61, and a left eye diffraction light waveguide coupling-out area 62, and the left eye diffraction light waveguide 6 can include a left eye diffraction light waveguide coupling-in area 61 and a left eye diffraction light waveguide coupling-out area 62. Figure 2 The left eye box space region 63 is also shown.

[0033] Since the process of the sampling device collecting the preset image projected by each display module is the same, in this embodiment, the sampling device collecting the preset image of the right display module is taken as an example. Figure 2As shown, the right-eye projector 1 projects a preset image onto the right-eye diffraction light waveguide coupling-in area 21, and the preset image is transmitted to the right-eye diffraction light waveguide coupling-out area 22 through the right-eye diffraction light waveguide coupling-in area 21; through the image sensor 4, the preset image projected from the right-eye diffraction light waveguide coupling-out area 22 to the eye box space area 23 is collected from the humanoid eye lens 3 to obtain the target detection image corresponding to the right-eye display module.

[0034] The diffractive optical waveguide can be understood as a transparent or semi-transparent optical substrate with a diffractive optical element disposed on its surface or inside.

[0035] The eye box space area can be understood as a three-dimensional area between the diffraction optical waveguide and the human eye where a target detection image that meets the preset conditions can be observed. The target detection image that meets the preset conditions can be understood as a target detection image whose clarity is greater than a set clarity threshold.

[0036] Among them, the distance between the anthropomorphic eye lens and the diffractive light waveguide outcoupling area is the eye-friendly distance; the eye-friendly distance is the distance between the pupil and the nearest diffractive light waveguide outcoupling area in the head-mounted display device.

[0037] In this embodiment, the eye distance is intended to meet the visual comfort condition. This embodiment does not impose specific restrictions on the visual comfort condition, and it is sufficient to ensure that the collected target detection image is natural, clear, and does not cause visual fatigue or discomfort. Taking the field of view angle of the image as an example, the visual comfort condition can be that the field of view angle is between 40-60 degrees. The field of view angle can be understood as the field of view that the user can see through the device. The field of view angle can be the angle between the edge of the image that the human eye can observe and the line connecting the center of the human pupil in the preset image projected by the display module.

[0038] The eye relief can also be understood as the distance between the pupil and the nearest diffractive light waveguide in the head-mounted display device worn by the user. The head-mounted display device can be a virtual reality helmet or augmented reality glasses. The binocular display module is a component of the head-mounted display device.

[0039] In this embodiment, the sampling device can scan and collect the preset image in the left-eye display module in real time in two-dimensional space to obtain the target detection image collected by the sampling device. The sampling device can also scan and collect the preset image in the right-eye display module in real time in two-dimensional space to obtain the target detection image collected by the sampling device.

[0040] S120, performing boundary detection processing on each of the target detection images to obtain eye box boundary information corresponding to each of the display modules.

[0041] In this embodiment, an image processing algorithm may be used to perform boundary detection processing on each of the target detection images to obtain eye box boundary information corresponding to each display module.

[0042] Optionally, performing boundary detection processing on each of the target detection images to obtain the eye box boundary information corresponding to each of the display modules includes: converting the target detection image into a grayscale image; performing foreground segmentation on the grayscale image based on a preset grayscale segmentation threshold to obtain a foreground image; and using the boundary of the foreground image as the eye box boundary information.

[0043] Specifically, for any target detection image, the target detection image is converted into a grayscale image; based on a preset grayscale segmentation threshold, the grayscale image is segmented foreground to obtain a foreground image. That is, in a grayscale image, if the brightness value of a pixel is lower than or equal to the preset grayscale segmentation threshold, the corresponding pixel can be set to 0 as a background image; if the brightness value of a pixel is higher than the preset grayscale segmentation threshold, the corresponding pixel is set to 255 as a foreground image. Through the grayscale segmentation threshold, the foreground image and the background image in the grayscale image can be segmented, and the boundary of the foreground image is used as the eye box boundary information. The method for extracting the boundary of the foreground image may include edge detection, morphological operations (such as corrosion, expansion, opening operation, closing operation, etc.), contour search, etc. In this embodiment, when extracting the boundary of the foreground image, coordinate calculation is also included, which can be used to accurately locate and describe the boundary of the foreground image.

[0044] Specifically, the contour detection function in the image processing library can be used to identify all contours of the foreground image, and the contour corresponding to the foreground image boundary can be screened out based on the geometric features of the contour (such as size, shape, position, etc.), and the contour points corresponding to the foreground image boundary can be extracted and traversed, and the coordinates of each contour point can be recorded, and the coordinates of these contour points are equivalent to the boundary coordinates of the foreground image. The eye box boundary information can include the boundary coordinates of the foreground image, and the boundary coordinates of the foreground image can include the vertex coordinates and eye point coordinates of the foreground image boundary.

[0045] S130: Determine a consistency detection result between the two eye box space regions based on a boundary comparison result between the two eye box boundary information.

[0046] In this embodiment, the consistency detection result between the two eye box space regions can be determined by comparing whether the boundary coordinates corresponding to the two eye box boundary information are consistent. If the boundary coordinates corresponding to the two eye box boundary information are consistent, it is considered that the consistency detection result is that the positions of the two eye box space regions are consistent; if the boundary coordinates corresponding to the two eye box boundary information are inconsistent, it is considered that the consistency detection result is that the positions of the two eye box space regions are inconsistent.

[0047] Optionally, the method of determining a consistency detection result between the two eye box space regions based on a boundary comparison result between the two eye box boundary information includes: obtaining first detection point coordinates and second detection point coordinates corresponding to each of the two eye box boundary information; performing a coordinate system conversion on the second detection point coordinates based on a coordinate system corresponding to the first detection point coordinates to obtain third detection point coordinates; and comparing the first detection point coordinates with the third detection point coordinates to determine a consistency detection result between the two eye box space regions.

[0048] In this embodiment, the first detection point coordinates and the second detection point coordinates corresponding to the two eye box boundary information are extracted. The first detection point coordinates and the second detection point coordinates may include the vertex coordinates and eye point coordinates of the respective foreground image boundaries. Based on the coordinate system corresponding to the first detection point coordinates, the coordinate system of the second detection point coordinates is converted into the coordinate system corresponding to the first detection point coordinates, and the third detection point coordinates of the second detection point coordinates in the coordinate system corresponding to the first detection point coordinates are obtained. The consistency detection result between the two eye box space regions can be determined by judging whether the first detection point coordinates and the third detection point coordinates match. Exemplarily, if the first detection point coordinates are used as a reference, the X-axis coordinates of the third detection point coordinates change to align with the first detection point coordinates. Alternatively, if the third detection point coordinates are used as a reference, the X-axis coordinates of the first detection point coordinates change to align with the third detection point coordinates. According to the aligned first detection point coordinates and the aligned third detection point coordinates, it can be seen whether the two coordinates are consistent, so that the consistency detection result between the two eye box space regions can be determined. If the deviation between the aligned first detection point coordinates and the aligned third detection point coordinates is less than a preset consistency threshold (or if the aligned first detection point coordinates are consistent with the aligned third detection point coordinates), the consistency detection result between the two eye box space regions is considered to be that the positions of the two eye box space regions are consistent. If the deviation between the aligned first detection point coordinates and the aligned third detection point coordinates is greater than or equal to the preset consistency threshold (or if the aligned first detection point coordinates are inconsistent with the aligned third detection point coordinates), the consistency detection result between the two eye box space regions is considered to be that the positions of the two eye box space regions are inconsistent.

[0049] Among them, the first detection point coordinates include the first upper left corner coordinate, the first upper right corner coordinate, the first lower left corner coordinate, the first lower right corner coordinate and the first eye point coordinate; the third detection point coordinates include the third upper left corner coordinate, the third upper right corner coordinate, the third lower left corner coordinate, the third lower right corner coordinate and the third eye point coordinate.

[0050] The eye point coordinates can be understood as the center coordinates of the boundary of the eye box space area, or as the center coordinates of the boundary of the foreground image. The eye point can be used to obtain the location of the user's eye pupil when the display performance is the best.

[0051] Optionally, the comparing the first detection point coordinates and the third detection point coordinates to determine the consistency detection result between the two eye box space areas includes: if the first upper left corner coordinate is consistent with the third upper left corner coordinate, the first upper right corner coordinate is consistent with the third upper right corner coordinate, the first lower left corner coordinate is consistent with the third lower left corner coordinate, the first lower right corner coordinate is consistent with the third lower right corner coordinate, and the first eye point coordinate is consistent with the third eye point coordinate, then the consistency detection result is that the positions of the two eye box space areas are consistent; if there is at least one set of coordinates inconsistent between the first upper left corner coordinate and the third upper left corner coordinate, the first upper right corner coordinate and the third upper right corner coordinate, the first lower left corner coordinate and the third lower left corner coordinate, the first lower right corner coordinate and the third lower right corner coordinate, and the first eye point coordinate and the third eye point coordinate, then the consistency detection result is that the positions of the two eye box space areas are inconsistent.

[0052] In this embodiment, if the aligned first upper left corner coordinate is consistent with the third upper left corner coordinate (or the deviation between the aligned first upper left corner coordinate and the aligned third upper left corner coordinate is less than a preset consistency threshold), the aligned first upper right corner coordinate is consistent with the aligned third upper right corner coordinate (or the deviation between the aligned first upper right corner coordinate and the aligned third upper right corner coordinate is less than a preset consistency threshold), the aligned first lower left corner coordinate is consistent with the aligned third lower left corner coordinate (or the deviation between the aligned first lower left corner coordinate and the aligned third lower left corner coordinate is less than a preset consistency threshold), the aligned first lower right corner coordinate is consistent with the aligned third lower right corner coordinate (or the deviation between the aligned first lower right corner coordinate and the aligned third lower right corner coordinate is less than a preset consistency threshold), and the aligned first eye point coordinate is consistent with the aligned third eye point coordinate (or the deviation between the aligned first eye point coordinate and the aligned third eye point coordinate is less than the preset consistency threshold), then the consistency detection result is that the positions of the two eye box space areas are consistent.

[0053] Among the following five groups of coordinates, the first upper left corner coordinate and the third upper left corner coordinate, the first upper right corner coordinate and the third upper right corner coordinate, the first lower left corner coordinate and the third lower left corner coordinate, the first lower right corner coordinate and the third lower right corner coordinate, and the first eye point coordinate and the third eye point coordinate, if at least one group of coordinates is inconsistent, the consistency detection result is that the positions of the two eye box space areas are inconsistent.

[0054] Among them, if the aligned first upper left corner coordinate is inconsistent with the aligned third upper left corner coordinate (or the deviation between the aligned first upper left corner coordinate and the aligned third upper left corner coordinate is greater than or equal to a preset consistency threshold), it is considered that the first upper left corner coordinate is inconsistent with the third upper left corner coordinate.

[0055] Among them, if the aligned first upper right corner coordinate is inconsistent with the aligned third upper right corner coordinate (or the deviation between the aligned first upper right corner coordinate and the aligned third upper right corner coordinate is greater than or equal to a preset consistency threshold), it is considered that the first upper right corner coordinate is inconsistent with the third upper right corner coordinate.

[0056] Among them, if the aligned first lower left corner coordinate is inconsistent with the aligned third lower left corner coordinate (or the deviation between the aligned first lower left corner coordinate and the aligned third lower left corner coordinate is greater than or equal to a preset consistency threshold), it is considered that the first lower left corner coordinate is inconsistent with the third lower left corner coordinate.

[0057] Among them, if the aligned first lower right corner coordinate is inconsistent with the aligned third lower right corner coordinate (or the deviation between the aligned first lower right corner coordinate and the aligned third lower right corner coordinate is greater than or equal to a preset consistency threshold), it is considered that the first lower right corner coordinate is inconsistent with the third lower right corner coordinate.

[0058] Among them, if the aligned first eye point coordinates are inconsistent with the aligned third eye point coordinates (or the deviation between the aligned first eye point coordinates and the aligned third eye point coordinates is greater than or equal to a preset consistency threshold), it is considered that the first eye point coordinates are consistent with the third eye point coordinates.

[0059] For example, Figure 3 A schematic diagram of the consistent positions of binocular eye box space areas provided by an embodiment of the present invention. Figure 3 After the two eye point coordinates in are aligned, the first eye point coordinate O'(x', y') (which can correspond to the left eye point coordinate) is consistent with the third eye point coordinate O(x, y) (which can correspond to the right eye point coordinate). The eye point coordinates can be understood as the best observation position under the theoretical exit pupil distance in the waveguide optical design.

[0060] For example, Figure 4 A schematic diagram of the inconsistent positions of binocular eye box space areas provided in an embodiment of the present invention. Figure 4 After the two eye point coordinates in are aligned, the first eye point coordinate O'(x', y') (which can correspond to the left eye point coordinate) is inconsistent with the third eye point coordinate O(x, y) (which can correspond to the right eye point coordinate). Among them, due to the assembly process and other reasons, the spatial area position of the binocular eye box is inconsistent. If the sampling is still carried out according to the theoretical design sampling position in the test sampling, the sampling result has no practical significance, and the difference between the left and right eyes of the same product, between the products of the same batch of products, and between different batches of products is large, which is not conducive to the control of product consistency.

[0061] Exemplarily, the sources of position consistency errors of the two eye box space regions are analyzed as follows:

[0062] First, the poor consistency of the diffraction waveguide grating introduces errors. The core of the diffraction waveguide optical display solution is that the diffraction waveguide coupling-in area and the diffraction waveguide coupling-out area have a sub-wavelength structure with periodic refractive index changes, where this sub-wavelength structure can be a rectangular grating, a blazed grating, a tilted grating, etc. Figure 5 As shown, Figure 5 A schematic diagram of a rectangular grating provided in an embodiment of the present invention. Figure 6 A schematic diagram of a blazed grating provided in an embodiment of the present invention. Figure 7 A schematic diagram of a tilted grating provided in an embodiment of the present invention.

[0063] Figure 8 Schematic diagram of the processing process of the diffraction optical waveguide provided in an embodiment of the present invention. The processing process is as follows: master preparation, material coating, nanoimprinting, curing and demolding. During the processing, based on the same master preparation, the nanoimprinting process is repeated until the template fails. In this process, due to the adhesion of the nanoimprint glue to the template, insufficient curing process, etc., the consistency of the grating obtained by each plate imprinting is poor during the demolding stage, resulting in inconsistent diffraction angles of signal light in different diffraction optical waveguides in the grating, and then the direction of the light coupled out of the grating is offset. Finally, in the Cartesian coordinate system, the position of the binocular eye box space area is inconsistent.

[0064] Second, the mismatch between the projector and the diffraction waveguide coupling area introduces errors. Taking the right-eye display module as an example, Fig. 9 A schematic diagram of coupling a projector and a diffractive optical waveguide provided in an embodiment of the present invention, Fig. 9 It includes a right-eye projector 1, a right-eye diffraction light waveguide 2, a right-eye diffraction light waveguide coupling-in region 21, and a right-eye diffraction light waveguide coupling-out region 22. Fig. 9 The example diagram shown here is a normal coupling example of vertical coupling. Fig.10A schematic diagram of coupling between another projector and a diffractive optical waveguide provided in an embodiment of the present invention. Fig.10 It also includes a right-eye projector 1, a right-eye diffraction light waveguide 2, a right-eye diffraction light waveguide coupling-in region 21, and a right-eye diffraction light waveguide coupling-out region 22, but Fig.10 This is an example of abnormal optical coupling. During the assembly process, due to the small size of the projector and the difficulty in strict alignment with the colorless optical coupling area, it is easy to Fig.10 situation.

[0065] The technical solution of the disclosed embodiment is to respectively collect preset images projected by each display module in the binocular display module through a sampling device to obtain a target detection image corresponding to each of the display modules; the sampling device is located on the boundary of the eye box space area corresponding to each of the display modules; a boundary detection process is performed on each of the target detection images to obtain the eye box boundary information corresponding to each of the display modules; and a consistency detection result between the two eye box space areas is determined based on the boundary comparison result between the two eye box boundary information. The disclosed embodiment is to respectively collect preset images in the binocular display module through a sampling device, perform boundary detection processing on each of the target detection images, and determine the consistency detection result between the two eye box space areas based on the boundary comparison result of the two eye box boundary information, so as to realize the consistency detection of the binocular eye box space area position, so as to discover and solve the problem of inconsistent binocular eye box space area position in the early stage of production, thereby improving product quality and enhancing user visual experience.

[0066] Fig.11 A schematic diagram of the structure of a binocular eye box space area position detection device provided by an embodiment of the present disclosure is shown in FIG. Fig.11 As shown, the device includes: a collection module 1110, a processing module 1120 and a consistency detection module 1130;

[0067] The acquisition module 1110 is used to respectively acquire the preset images projected by each display module in the binocular display module through a sampling device to obtain a target detection image corresponding to each display module; the sampling device is located on the boundary of the eye box space area corresponding to each display module;

[0068] The processing module 1120 is used to perform boundary detection processing on each of the target detection images to obtain eye box boundary information corresponding to each of the display modules;

[0069] The consistency detection module 1130 is used to determine the consistency detection result between the two eye box space regions based on the boundary comparison result between the two eye box boundary information.

[0070] The technical solution of the disclosed embodiment is to respectively collect the preset images projected by each display module in the binocular display module through the sampling device of the acquisition module, and obtain the target detection image corresponding to each of the display modules; the sampling device is located on the boundary of the eye box space area corresponding to each of the display modules; the processing module performs boundary detection processing on each of the target detection images to obtain the eye box boundary information corresponding to each of the display modules; the consistency detection result between the two eye box space areas is determined based on the boundary comparison result between the two eye box boundary information by the consistency detection module. In the disclosed embodiment, the sampling device respectively collects the preset images in the binocular display module, performs boundary detection processing on each of the target detection images, and determines the consistency detection result between the two eye box space areas based on the boundary comparison result of the two eye box boundary information, so as to realize the consistency detection of the binocular eye box space area position, so as to discover and solve the problem of inconsistent binocular eye box space area position in the early stage of production, thereby improving product quality and enhancing user visual experience.

[0071] The sampling device includes a humanoid eye lens and an image sensor; the binocular display module includes two display modules; each display module includes a projector, a diffraction light waveguide coupling-in region and a diffraction light waveguide coupling-out region.

[0072] Optionally, the acquisition module is specifically used to: project the preset image onto the corresponding diffraction light waveguide coupling-in area through each of the projectors, and transmit the preset image to the diffraction light waveguide coupling-in area through the diffraction light waveguide coupling-in area; and acquire the preset image projected from each of the diffraction light waveguide coupling areas to the corresponding eye box space area from the anthropomorphic eye lens through the image sensor to obtain a target detection image corresponding to each of the display modules.

[0073] Among them, the distance between the anthropomorphic eye lens and the diffractive light waveguide outcoupling area is the eye-friendly distance; the eye-friendly distance is the distance between the pupil and the nearest diffractive light waveguide outcoupling area in the head-mounted display device.

[0074] Optionally, the processing module is specifically used to: convert the target detection image into a grayscale image; perform foreground segmentation on the grayscale image based on a preset grayscale segmentation threshold to obtain a foreground image; and use the boundary of the foreground image as the eye box boundary information.

[0075] Optionally, the consistency detection module is specifically used to: obtain the first detection point coordinates and the second detection point coordinates corresponding to the two eye box boundary information; based on the coordinate system corresponding to the first detection point coordinates, perform coordinate system conversion on the second detection point coordinates to obtain the third detection point coordinates; compare the first detection point coordinates with the third detection point coordinates to determine the consistency detection result between the two eye box space areas.

[0076] Among them, the first detection point coordinates include the first upper left corner coordinate, the first upper right corner coordinate, the first lower left corner coordinate, the first lower right corner coordinate and the first eye point coordinate; the third detection point coordinates include the third upper left corner coordinate, the third upper right corner coordinate, the third lower left corner coordinate, the third lower right corner coordinate and the third eye point coordinate.

[0077] Optionally, the consistency detection module is specifically used for: if the first upper left corner coordinate is consistent with the third upper left corner coordinate, the first upper right corner coordinate is consistent with the third upper right corner coordinate, the first lower left corner coordinate is consistent with the third lower left corner coordinate, the first lower right corner coordinate is consistent with the third lower right corner coordinate, and the first eye point coordinate is consistent with the third eye point coordinate, then the consistency detection result is that the positions of the two eye box space areas are consistent; if there is at least one set of coordinates inconsistent between the first upper left corner coordinate and the third upper left corner coordinate, the first upper right corner coordinate and the third upper right corner coordinate, the first lower left corner coordinate and the third lower left corner coordinate, the first lower right corner coordinate and the third lower right corner coordinate, and the first eye point coordinate and the third eye point coordinate, then the consistency detection result is that the positions of the two eye box space areas are inconsistent.

[0078] The binocular eye box space area position detection device provided in the embodiments of the present disclosure can execute the binocular eye box space area position detection method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0079] Fig.12 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0080] like Fig.12As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0081] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0082] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as the method of binocular eye box space area position detection.

[0083] In some embodiments, the method binocular eye box space region position detection can be implemented as a computer program, which is tangibly contained in a computer readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method binocular eye box space region position detection described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the method binocular eye box space region position detection in any other appropriate manner (e.g., by means of firmware).

[0084] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0085] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0086] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0087] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0088] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0089] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0090] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the binocular eye box space area position detection method provided in any embodiment of the present application.

[0091] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0092] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A binocular eye box space area position detection method, characterized in that: include: The preset images projected by each display module in the binocular display module are collected by the sampling device respectively to obtain the target detection image corresponding to each display module; The sampling device is located on the boundary of the eye box space area corresponding to each display module; Performing boundary detection processing on each of the target detection images to obtain eye box boundary information corresponding to each of the display modules; Based on the boundary comparison result between the two eye box boundary information, the consistency detection result between the two eye box space regions is determined.

2. The method according to claim 1, characterized in that in, The sampling device includes a humanoid eye lens and an image sensor; the binocular display module includes two display modules; each display module includes a projector, a diffraction light waveguide coupling-in region and a diffraction light waveguide coupling-out region; The method of collecting the preset images projected by each display module in the binocular display module by using the sampling device to obtain the target detection image corresponding to each display module includes: Projecting the preset image onto the corresponding diffraction light waveguide coupling-in region through each of the projectors, and transmitting the preset image to the diffraction light waveguide coupling-out region through the diffraction light waveguide coupling-in region; The preset image of each diffraction light waveguide outcoupling area projected onto the corresponding eye box space area is collected from the humanoid eye lens through the image sensor to obtain a target detection image corresponding to each display module.

3. The method according to claim 2, characterized in that The distance between the humanoid eye lens and the diffractive light waveguide outcoupling area is the eye-friendly distance; the eye-friendly distance is the distance between the pupil and the nearest diffractive light waveguide outcoupling area in the head-mounted display device.

4. The method according to claim 1, characterized in that: The performing boundary detection processing on each of the target detection images to obtain the eye box boundary information corresponding to each of the display modules includes: Converting the target detection image into a grayscale image; Based on a preset grayscale segmentation threshold, performing foreground segmentation on the grayscale image to obtain a foreground image; The boundary of the foreground image is used as the eye box boundary information.

5. The method according to claim 1, characterized in that The step of determining the consistency detection result between the two eye box space regions based on the boundary comparison result between the two eye box boundary information includes: Obtaining the first detection point coordinates and the second detection point coordinates corresponding to the two eye box boundary information respectively; Based on the coordinate system corresponding to the first detection point coordinates, the coordinate system of the second detection point coordinates is converted to obtain the third detection point coordinates; The first detection point coordinates and the third detection point coordinates are compared to determine a consistency detection result between the two eye box space regions.

6. The method according to claim 5, characterized in that The first detection point coordinates include a first upper left corner coordinate, a first upper right corner coordinate, a first lower left corner coordinate, a first lower right corner coordinate, and a first eye point coordinate; the third detection point coordinates include a third upper left corner coordinate, a third upper right corner coordinate, a third lower left corner coordinate, a third lower right corner coordinate, and a third eye point coordinate; The comparing the first detection point coordinates with the third detection point coordinates to determine the consistency detection result between the two eye box space regions includes: If the first upper left corner coordinate is consistent with the third upper left corner coordinate, the first upper right corner coordinate is consistent with the third upper right corner coordinate, the first lower left corner coordinate is consistent with the third lower left corner coordinate, the first lower right corner coordinate is consistent with the third lower right corner coordinate, and the first eye point coordinate is consistent with the third eye point coordinate, then the consistency detection result is that the positions of the two eye box space areas are consistent; If at least one set of coordinates among the first upper left corner coordinate and the third upper left corner coordinate, the first upper right corner coordinate and the third upper right corner coordinate, the first lower left corner coordinate and the third lower left corner coordinate, the first lower right corner coordinate and the third lower right corner coordinate, and the first eye point coordinate and the third eye point coordinate is inconsistent, the consistency detection result is that the positions of the two eye box space areas are inconsistent.

7. A binocular eye box space area position detection device, characterized in that: include: An acquisition module, used to respectively acquire preset images projected by each display module in the binocular display module through a sampling device, and obtain a target detection image corresponding to each display module; The sampling device is located on the boundary of the eye box space area corresponding to each display module; A processing module, used for performing boundary detection processing on each of the target detection images to obtain eye box boundary information corresponding to each of the display modules; The consistency detection module is used to determine the consistency detection result between the two eye box space areas based on the boundary comparison result between the two eye box boundary information.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the binocular eye box spatial area position detection method as described in any one of claims 1-6.

9. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the binocular eye box space area position detection method as claimed in any one of claims 1 to 6 when executed by a computer processor.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the binocular eye box space area position detection method according to any one of claims 1 to 6.

Images