Ear region imaging

By performing millimeter wave or ultrasonic imaging of the ear area, combining facial images to generate mixed images of shared coordinate systems, it solves the problem that traditional measurement systems are difficult to accurately measure ear saddle points, and improves the fitting and virtual image presentation of head-mounted devices performance.

CN118974587APending Publication Date: 2024-11-15CTRL-LABS CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380031606.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-31
Filing Date
2023-03-28
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Traditional head-mounted device measurement systems are difficult to accurately measure the ear area, especially the ear saddle points, resulting in poor fit of the equipment, affecting the presentation of virtual images and user experience.

Method used

The ear area is imaged using non-visible light signals such as millimeter wave or ultrasound. The ear area image is generated through the transmitter and receiver, and the face image is combined to generate mixed images, sharing a common coordinate system to extract the ear saddle point and the eye. Size information to improve the fit of the device.

Benefits of technology

Improve the fit quality of the head-mounted device, enhance the presentation performance and user experience of virtual images, especially by accurately measuring the size of the ear area, and appropriately designing near-eye displays and eye tracking systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118974587A_ABST
    Figure CN118974587A_ABST
Patent Text Reader

Abstract

The transmitter transmits a transmit signal to the ear region. The receiver generates an image signal. The receiver generates an image signal in response to receiving the return signal. The return signal is a millimeter wave transmission signal reflected back from the ear region. An ear region image is generated in response to the image signal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates generally to optics and, more particularly, to imaging the ear region. Background Art

[0002] A head mounted device is a wearable electronic device that is typically worn on the head of a user. A head mounted device may include one or more electronic components for various applications such as gaming, aviation, engineering, medical, entertainment, and activity tracking. A head mounted device may include a display that presents virtual images to the wearer of the head mounted device. When a head mounted device includes a display, the head mounted device may be referred to as a head mounted display (HMD). A head mounted device may be fitted to a particular user to improve the performance or comfort of the head mounted device. Summary of the invention

[0003] According to a first aspect of the present disclosure, a system for imaging an ear region is provided, the system comprising: a transmitter configured to transmit a millimeter-wave transmission signal; a receiver configured to receive a return millimeter-wave signal; and processing logic configured to: drive the transmitter to transmit the millimeter-wave transmission signal to the ear region; receive an image signal from the receiver, wherein the image signal is generated by the receiver in response to receiving the return millimeter-wave signal, the return millimeter-wave signal being the millimeter-wave transmission signal reflected from the ear region; and generate an ear region image in response to the image signal.

[0004] In some embodiments, the system may also include: an optical imaging system configured to capture a facial image of a facial area by sensing visible light or non-visible light reflected from the facial area, the facial area including an eye area and a nose area, wherein the facial area overlaps with an ear area, and wherein the processing logic is further configured to: receive the facial image from the optical imaging system; and generate a hybrid face-ear image, the hybrid face-ear image including the facial image and the ear area image, wherein the facial image and the ear area image share a common coordinate system in the imaging space.

[0005] In some embodiments, the blended face-ear image may include dimensions between the user's ear saddle points and eye features of the eyes.

[0006] In some embodiments, the wavelength of the millimeter wave transmission signal can be between 1 millimeter (mm) and 10 mm.

[0007] In some embodiments, the ear region may include an ear saddle point.

[0008] In some embodiments, the transmitter may include a two-dimensional array of transmitter elements, and wherein driving the transmitter to transmit the millimeter wave transmit signal includes driving the transmitter elements to beam-form the millimeter wave transmit signal to direct the millimeter wave transmit signal to the ear region.

[0009] In some embodiments, the receiver may include a two-dimensional array of receiving elements.

[0010] In some embodiments, driving the transmitter to transmit the millimeter wave transmission signal may include driving the transmitter to sweep the millimeter wave transmission signal across the ear region.

[0011] In some embodiments, the system may further include: a second transmitter configured to transmit a second millimeter wave transmit signal; and a second receiver configured to receive a second return millimeter wave signal, wherein the second return millimeter wave signal is the second millimeter wave transmit signal reflected from the second ear region.

[0012] In some embodiments, the second ear region may be a left ear region, and the ear region may be a right ear region.

[0013] In some embodiments, the system may also include: a camera configured to capture one or more position images, wherein the processing logic is further configured to: receive the one or more position images from the camera; and identify the ear area of ​​a person based on one or more of the position images.

[0014] According to another aspect of the present disclosure, a system for imaging an ear region is provided, the system comprising: a transmitter configured to transmit a transmission signal, wherein the transmission signal is neither visible light nor invisible light; a receiver configured to receive a return signal; and processing logic configured to drive the transmitter to transmit the transmission signal to the ear region; receive an image signal from the receiver, wherein the image signal is generated by the receiver in response to receiving the return signal, the return signal being the transmission signal reflected from the ear region; and generate an ear region image in response to the image signal.

[0015] In some embodiments, the transmitted signal may be an ultrasonic signal.

[0016] According to another aspect of the present disclosure, a method is provided, the method comprising: driving a transmitter to transmit a millimeter-wave transmission signal to an ear region; receiving an image signal from a receiver, wherein the image signal is generated by the receiver in response to receiving a return millimeter-wave signal, the return millimeter-wave signal being the millimeter-wave transmission signal reflected from the ear region; and generating an ear region image in response to the image signal.

[0017] In some embodiments, the method may also include: capturing a facial image of a facial area of ​​a person by sensing visible light or non-visible light reflected from the facial area, the facial area including an eye area and a nose area, wherein the facial area overlaps with an ear area; and generating a hybrid face-ear image, the hybrid face-ear image including a facial image and an ear area image, wherein the facial image and the ear area image share a common coordinate system in the imaging space.

[0018] In some embodiments, the method may further include determining a size based on the hybrid face-ear image, wherein the size is between an ear saddle point and a pupil plane of the eye of the user.

[0019] In some embodiments, the wavelength of the millimeter wave transmission signal can be between 1 mm and 10 mm.

[0020] In some embodiments, the ear region may include an ear saddle point.

[0021] In some embodiments, the returning millimeter wave signal may propagate through the hair before encountering a receiver.

[0022] In some embodiments, the method may further include: identifying an ear region of the person based on the position image of the person.

[0023] It will be understood that any feature described herein as being suitable for incorporation into one or more aspects or embodiments of the present disclosure is intended to be generalizable to any aspect, all aspects, any embodiment, and all embodiments of the present disclosure. Other aspects of the present disclosure may be understood by those skilled in the art based on the specification, claims, and drawings of the present disclosure. The foregoing general description and the following detailed description are exemplary and illustrative only, and do not limit the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Non-limiting and non-exhaustive embodiments of the present invention are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified.

[0025] Figure 1 A top view of a user and an example system for imaging an ear region of the user in accordance with aspects of the present disclosure is shown.

[0026] Figure 2 A front view of a user including a facial area and an ear area is shown in accordance with aspects of the present disclosure.

[0027] Figure 3 The dimensions between the ear saddle points and the eye features are shown in accordance with aspects of the present disclosure.

[0028] Figure 4An example head mounted device including a frame and arms according to aspects of the present disclosure is shown.

[0029] Figure 5 An ear region imaging process according to aspects of the present disclosure is shown. DETAILED DESCRIPTION

[0030] Embodiments of imaging the ear region are described herein. In the following description, many specific details are listed to provide a thorough understanding of the present embodiment. However, those skilled in the relevant art will recognize that the techniques described herein can be implemented without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.

[0031] In this specification, reference to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing in various places in this specification do not necessarily all refer to the same embodiment. In addition, in one or more embodiments, these particular features, structures, or characteristics may be combined in any suitable manner.

[0032] In some embodiments of the present disclosure, the term "near eye" may be defined as including an element that is configured to be placed within 50 mm of the user's eye when using a near eye device. Thus, a "near eye optical element" or "near eye system" will include one or more elements that are configured to be placed within 50 mm of the user's eye.

[0033] In various aspects of the present disclosure, visible light can be defined as having a wavelength range of about 380 nanometers (nm) to 700nm. Non-visible light can be defined as light with a wavelength outside the visible light range, such as ultraviolet light and infrared light. Infrared light with a wavelength range of about 700nm to 1000000nm includes near infrared light. In various aspects of the present disclosure, near infrared light can be defined as having a wavelength range of about 700nm to 1600nm.

[0034] In various aspects of the present disclosure, the term "transparent" may be defined as having a light transmittance greater than 90%. In some aspects, the term "transparent" may be defined as a material having a visible light transmittance greater than 90%.

[0035] In some contexts, proper fitting of a head-mounted device improves the comfort and / or performance of features of the head-mounted device. For example, by properly fitting the head-mounted device to a particular user, the performance of eye tracking and virtual image presentation can be enhanced. In addition, proper fitting can help improve the quality of the virtual experience by shortening the user's adaptation period (if any). For a head-mounted device that is a head-mounted display, another potential advantage of improved fit is that a near-eye display with a smaller eye zone is optionally designed, which may be more effective than a near-eye display with a larger eye zone.

[0036] Achieving a good fit for the user may include achieving, for example, (1) physical comfort, (2) a properly positioned near-eye display (if any), (3) eye tracking functionality, and (4) visual comfort. Achieving physical comfort may include distributing any pressure due to the weight of the head-mounted device and stabilizing the head-mounted device even as the user moves. In order to properly position the near-eye display, one or both of the user's pupils will need to be located in the eye-adaptive zone to which the near-eye display presents virtual images. Eye tracking performance of the head-mounted device (if any) may be improved by an appropriate eye-adaptive distance between the user's eyes and components of the eye tracking system in the frame of the head-mounted device and / or temple arms. The user's visual comfort may require that the prescription lenses included in the head-mounted device be centered relative to the user's pupils.

[0037] To help provide a high-quality fit for a head-mounted device, it may be necessary to measure the dimensions of the user (e.g., nose shape, head width, and ear depth). Features of a particular user's ear may be particularly important if the head-mounted device is to rest or come to rest over the head of the user when worn. The ear saddle point is the point where the arm of the head-mounted device will rest when the user is wearing the head-mounted device. The ear saddle point may correspond to the inflection point where the ear hook of the arm of the head-mounted device contacts the ear. Conventional measurement systems (e.g., camera systems) provide acceptable measurements of the user's facial area to fit the head-mounted device, but because hair obscures the top and back of the ear (especially the ear saddle point), conventional measurement systems have difficulty accurately imaging / measuring the ear area. Therefore, the fit of the head-mounted device may be improved by an improved ear area imaging system.

[0038] Disclosed herein are embodiments of devices, systems, and methods for imaging the ear region. A transmitter transmits a transmission signal to the ear region. The transmission signal may be a millimeter-wave (also referred to as "mm-wave") transmission signal, an ultrasonic transmission signal, or other transmission signal that propagates through hair. A receiver receives return signals, which are transmission signals reflected / scattered back from the ear in the ear region. The receiver generates an image signal based on the return signal, and generates an ear region image in response to the image signal. The ear region image may be combined with a facial image of a user's facial region to generate a hybrid face-ear image. The hybrid face-ear image may share a common coordinate system, so that various dimensions of the user's eyes and face relative to the user may be extracted from the hybrid face-ear image for fitting the head-mounted device. In combination with Figures 1 to 5 These and other embodiments are described in further detail.

[0039] Figure 1 A top view of a user 190 and an example system 100 for imaging an ear region of the user in accordance with aspects of the present disclosure is shown. The example system 100 includes transducers 120A and 120B (collectively, transducers 120), optical imaging systems 130A and 130B (collectively, optical imaging systems 130), and processing logic 101. In some embodiments, the system 100 includes one transducer 120, and the processing logic 101 and one or more optical imaging systems 130 are optional. In various embodiments, the system 100 can include a single transducer 120 and a single optical imaging system 130, although any number of transducers 120 and optical imaging systems 130 can be combined in the systems of the present disclosure.

[0040] The transducer 120A includes a transmitter 121A configured to transmit a transmission signal 126A to an ear region 181A of the user 190, and a receiver 122A configured to receive a return signal 127A. The return signal 127A is the transmission signal 126A reflected / scattered from the ear region 181A. Similarly, the transducer 120B includes a transmitter 121B configured to transmit a transmission signal 126B to an ear region 181B of the user 190, and a receiver 122B configured to receive a return signal 127B. The return signal 127B is the transmission signal 126B reflected / scattered from the ear region 181B.

[0041] Transmitted signals 126A and 126B (collectively referred to as transmitted signals 126) penetrate the hair and reflect off the human skin. Therefore, if the hair blocks the skin of the ear, the transmitted signal 126 propagates through the hair, reflects / scatters off the skin of the ear, and then propagates through the hair back to the receiver 122 as a return signal 127. The ear saddle point 185 of the ear of the user 190 is indicated in the ear region 181A. Although not specifically illustrated, the ear that occupies the ear region also includes the ear saddle point. As described above, the ear saddle point can be an important point for properly fitting the head mounted device to a particular user.

[0042] The transmit signal 126 is neither visible light nor infrared light, as visible light and infrared light may be blocked by the hair covering the ears. In some embodiments, the transmit signal may be a millimeter wave transmit signal, which has a wavelength between 1 mm and 10 mm in the electromagnetic spectrum. For example, millimeter wave transducers / chips are commonly available from the autonomous vehicle industry or the airport security industry. In embodiments, the transmit signal 126 may be an ultrasonic or X-ray transmit signal.

[0043] In operation, one or more processing logics drive one or more transmitters 121 to transmit one or more transmit signals 126 to one or more ear regions 181. Receivers 122A and 122B receive return signals 127A and 127B, respectively, and generate image signals 129A and 129B in response to receiving return signals 127A and 127B (also respectively). Return signals 127A and 127B may be collectively referred to as return signals 127, and image signals 129A and 129B may be collectively referred to as image signals 129. Processing logic 101 receives image signals 129 from one or more receivers 122 of one or more transducers, and generates ear region images 104 in response to image signals 129.

[0044] In the illustrated embodiment, the processing logic 101 includes an ear image generation module 103 that receives the image signal 129 and generates the ear region image 104 by processing the image signal 129. When the system 100 includes one or more optical imaging systems 130, the optical imaging system 130 can capture a facial image 139 of the facial region of the user 190 by sensing visible light or non-visible light 143 reflected from the facial region.

[0045] Figure 2 A front view of a user 290 including a facial region 283 and ear regions 281A and 281B is shown in accordance with an embodiment of the present disclosure. Figure 1The optical imaging system 130 can be configured to generate one or more facial images 139 that image a facial region 283. In this particular example illustration, the facial region 283 includes an eye region 284 and a nose region 286 of the user 290. The features of the eyes and nose can help determine a comfortable fit of the head mounted device for a particular user 290. Figure 2 , the facial region 283 overlaps the ear regions 281A and 281B. The approximate location of the ear saddle point 285A is indicated in the right ear region 281A, and the approximate location of the ear saddle point 285B is indicated in the left ear region 281B.

[0046] Back to Figure 1 , the optical imaging system 100 is configured to capture a facial image 139 of the facial region 283 by sensing visible light or non-visible light reflected back from the facial region 283. In some embodiments, the illumination module 140 can be oriented to illuminate the facial region 283 with the illumination light 143. The illumination module 140 can include a light source, such as a light emitting diode (LED) or a laser light source. The illumination light 143 can be visible light or infrared light. The optical imaging system 130 can include one or more complementary metal-oxide semiconductor (CMOS) image sensors. In an embodiment where the illumination light 143 is infrared light, an infrared filter that receives a narrow band of infrared wavelengths can be placed on the image sensor to make it sensitive to the narrow band of infrared wavelengths of the illumination light 143 while rejecting visible light and wavelengths outside the narrow band. In the illustrated embodiment, the processing logic 101 is configured to selectively enable and disable (turn on and off) the illumination module 140 via the communication channel X1, so that the illumination of the user 190 with the illumination light 143 corresponds to the image capture of one or more optical imaging systems 130.

[0047] Processing logic 101 is configured to receive facial images 139A and 139B (collectively, facial images 139) of facial region 283. Facial images 139 may include columns and rows of pixels. Processing logic 101 receives one or more facial images 139 from optical imaging system 130. Processing logic 101 may generate a hybrid face-ear image 108 that includes one or more facial images 139 and one or more ear region images 104. Figure 1In the specific example illustration of , processing logic 101 includes a combined facial image generation module 105 that generates a combined facial image 106 that includes facial images 139A and 139B. In other embodiments, facial image 139 is routed directly to blended image generation module 107, and blended image generation module 107 generates blended face-ear image 108 based on one or more facial images 139 and ear region image 104. One or more facial images 139 may share a common coordinate system in imaging space with ear region image 104, so that blended image generation module 107 may generate blended face-ear image 108 in which locations in ear region image 104 are linked to locations in facial image 139.

[0048] although Figure 1 Each transducer 120 is shown having one transmitter 121 and one receiver 122, but embodiments of the transducer 120 include arrays of transmitters 121 that are used to implement beamforming of the transmit signal 126. In this way, the transmit signal 126 can be directed to multiple locations of the ear region 181 for imaging purposes. In embodiments, the transducer 120 includes a two-dimensional (2D) array of transmitters 121 that generates patterned illumination of the ear region 181 using the transmit signal 126. In embodiments, the transducer 120 includes a 2D array of transmitters 121 that scans the transmit signal 126 across the ear region 181. In embodiments, the transducer 120 includes a one-dimensional array of transmitters 121 that scans the transmit signal 126 across the ear region 181. In an embodiment, the receiver 122 includes a photodiode and the transmitter 121 includes a mechanical steering source for directing the transmit signal 126 within the ear region 181. The receiver 122 may include a 2D array of receiving elements.

[0049] Figure 3 The dimension 398 between the ear saddle point 385A and the eye feature is shown in accordance with aspects of the present disclosure. Figure 3The hair of person 390 is shown to potentially obstruct light-based imaging of ear saddle point 385 (located behind the ear in ear region 381A). However, imaging ear saddle point 385A using transmit signal 126 propagated through the hair provides accurate imaging of ear saddle point 385A, which can then be used to determine dimension 398 between ear saddle point 385A and eye features. For example, dimension 398 can be calculated from hybrid face-ear image 108 if hybrid face-ear image 108 includes a facial image of facial region 383 and ear region 381A that share a common coordinate system. Figure 3 In the illustrated embodiment, the eye feature is the corneal plane. In some embodiments, the eye feature is the pupil plane.

[0050] Figure 4 An example head mounted device 400 including a frame 402 and arms 404A and 404B according to an embodiment of the present disclosure is shown. The example of the head mounted device 400 shown is shown to include the frame 402, the temple arms 404A and 404B, and the near-eye optical elements 410A and 410B. In order to properly fit the head mounted device 400 to the person 390, the head mounted device 400 may be configured based on the Figure 3 The size 498 of the temple arm 404A is selected based on the size 398 of the head. The size 498 is measured from the inflection point 485 on the temple arm 404A located at the saddle point of the user's ear. The size 498 can be slightly longer than the size 398 to provide an eye relief between the user's eyes and the optical element 410A of the head mounted device 400. The head width size 497 of the frame 402 can also be selected for the user based on the head width measurement of the user derived from the hybrid face-ear image (e.g., image 108).

[0051] Figure 4An exploded view of an example of a near-eye optical element 410A is shown. The near-eye optical element 410A is shown as including an optically transparent layer 420A, an illumination layer 430A, and a display layer 440A. The display layer 440A can include a waveguide 448A configured to guide virtual images included in visible image light 441 to eyes of a user of the head-mounted device 400 located in an eyebox region of the head-mounted device 400. Thus, when proper fit of the head-mounted device 400 allows virtual images included in the visible image light 441 to be more accurately presented to the user's eyes positioned relative to the display layer 440A, the fit of the head-mounted device 400 enhances the user's experience when viewing these virtual images. In some embodiments, at least a portion of the electronic display of the display layer 440A is included in the frame 402 of the head-mounted device 400. The electronic display may include the following items for generating image light 441: a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a micro-LED display, a pico-projector, or a liquid crystal on silicon (LCOS) display.

[0052] When the head mounted device 400 includes a display, the head mounted device 400 may be considered a head mounted display. The head mounted device 400 may be considered an augmented reality (AR) head mounted display. Figure 4 A head mounted device 400 configured for use in an augmented reality (AR) or mixed reality (MR) context is shown, and the disclosed embodiments may also be used for other implementations of head mounted displays (e.g., virtual reality head mounted displays). In addition, some embodiments of the present disclosure may be used for head mounted devices that do not include a display.

[0053] The lighting layer 430A is shown as including a plurality of in-field illuminators 426. The in-field illuminators 426 are described as being "in-field" because they are in the field of view (FOV) of the user of the head mounted device 400. In an embodiment, the in-field illuminators 426 may be located in the same FOV as the display of the head mounted device 400 viewed by the user. In some aspects of the present disclosure, the in-field illuminators 426 are configured to emit near-infrared light that aids eye tracking. Each in-field illuminator 426 may be a micro-LED, an edge-emitting LED, a vertical cavity surface emitting laser (VCSEL) diode, or a superluminescent diode (SLED). In some embodiments, the illuminator 426 is not in the field. Conversely, in some embodiments, the illuminator 426 may be outside the field.

[0054] like Figure 4 As shown, the frame 402 is coupled to the temple arms 404A and 404B for securing the head mounted device 400 to the user's head. The example head mounted device 400 may also include support hardware contained in the frame 402 and / or the temple arms 404A and 404B. The hardware of the head mounted device 400 may include any of the following: processing logic, a wired data interface and / or a wireless data interface for sending and receiving data, a graphics processor, and one or more memories for storing data and computer executable instructions. In one example, the head mounted device 400 may be configured to receive wired power, and / or may be configured to be powered by one or more batteries. In addition, the head mounted device 400 may be configured to receive wired data and / or wireless data including video data.

[0055] The optically transparent layer 420A is shown as being disposed between the illumination layer 430A and the eye-facing side 409 of the near-eye optical element 410A. The optically transparent layer 420A can receive infrared illumination light emitted by the illumination layer 430A and pass the infrared illumination light to illuminate the user's eyes. As described above, the optically transparent layer 420A can also be transparent to visible light, such as scene light received from the environment and / or image light 441 received from the display layer 440A. In some examples, the optically transparent layer 420A has a curvature for focusing light (e.g., display light and / or scene light) to the user's eyes. Therefore, in some examples, the optically transparent layer 420A can be referred to as a lens. In some aspects, the optically transparent layer 420A has a thickness and / or curvature corresponding to the user's specifications. In other words, the optically transparent layer 420A can be a prescription lens. However, in other examples, the optically transparent layer 420A can be a non-prescription lens. When optically transparent layer 420 has optical power, the improved fit of head mounted device 400 allows the center of the lens to correspond with the user's pupil location so that image light 441 and / or scene light 491 are properly focused onto the pupil.

[0056] Figure 5 An ear region imaging process 500 is shown in accordance with aspects of the present disclosure. The order in which some or all of the multiple process blocks appear in process 500 should not be considered limiting. In contrast, one of ordinary skill in the art having the benefit of the present disclosure will appreciate that some of the multiple process blocks may be performed in various orders not shown, or even in parallel. In some embodiments, Figure 1 The processing logic 101 executes the entire process or a part of the process 500 .

[0057] At process block 505, the transmitter is driven to transmit a millimeter wave transmit signal (e.g., transmit signal 126) toward the ear region. The wavelength of the millimeter wave transmit signal may be between 1 mm and 10 mm. The ear region image may include an ear saddle point. In an embodiment, the transmitter includes a two-dimensional array of transmitter elements, and driving the transmitter to transmit the millimeter wave transmit signal includes: driving the transmitter elements to beamform the millimeter wave transmit signals to direct the millimeter wave transmit signals to the ear region.

[0058] At process block 510, an image signal is received from a receiver (eg, receiver 122). The image signal is generated by the receiver in response to receiving a return millimeter wave signal. The return millimeter wave signal is a millimeter wave transmit signal that is reflected back from the ear region.

[0059] At process block 515, an ear region image is generated in response to the image signal.

[0060] An embodiment of process 500 also includes capturing a facial image of a facial region of a person by sensing visible light or non-visible light reflected from the facial region of the person, and generating a hybrid face-ear image, the hybrid face-ear image including the facial image and the ear region image. The facial region may include an eye region and a nose region. The facial image may share a common coordinate system with the ear region image in an imaging space. Process 500 may also include determining a size based on the hybrid face-ear image, wherein the size is between the ear saddle point of the user and the corneal plane of the eye.

[0061] In an embodiment of process 500, the ear region of the person is identified based on the position image of the person before executing process block 505. This can allow the millimeter wave transmission signal to be directed to the location for imaging the ear. In an embodiment, the position image of the person is captured using a camera included in one or more optical imaging systems 130 so that the ear region (e.g., 181) of the person can be determined relative to transducer 120.

[0062] Embodiments of the present invention may include an artificial reality system, or be implemented in conjunction with an artificial reality system. Artificial reality is a form of reality that has been adjusted in some way before being presented to a user, and may include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or some combination and / or derivative thereof. Artificial reality content may include fully generated content or generated content combined with captured (e.g., real-world) content. Artificial reality content may include video, audio, tactile feedback, or some combination thereof, and any of them may be presented in a single channel or multiple channels (e.g., stereoscopic video that produces a three-dimensional effect for the viewer). In addition, in some embodiments, artificial reality may also be associated with applications, products, accessories, services, or some combination thereof, which are used, for example, to create content in artificial reality, and / or otherwise used in artificial reality (e.g., performing activities in artificial reality). Artificial reality systems that provide artificial reality content can be implemented on a variety of platforms, including a head-mounted display (HMD) connected to a host computer system, a standalone HMD, a mobile device or computing system, or any other hardware platform capable of providing artificial reality content to one or more viewers.

[0063] The term "processing logic" in the present disclosure (e.g., processing logic 101) may include the following items for performing the operations disclosed herein: one or more processors, microprocessors, multi-core processors, application-specific integrated circuits (ASICs) and / or field programmable gate arrays (FPGAs). In some embodiments, a memory (not shown) is integrated into the processing logic to store instructions for performing operations and / or storing data. The processing logic may also include analog or digital circuits for performing operations according to embodiments of the present disclosure.

[0064] The "memory" or "memory" described in the present disclosure may include one or more volatile or non-volatile memory architectures. The "memory" or "memory" may be removable and non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, program modules, or other data). Example memory technologies may include random access memory (RAM), read only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory, CD-ROM, digital versatile disk (DVD), high-resolution multimedia / data storage disk or other optical storage device, magnetic cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, or any other non-transmission medium that can be used to store information for access by a computing device.

[0065] The network may include any network or network system, such as but not limited to: peer-to-peer networks; Local Area Network (LAN); Wide Area Network (WAN); public networks such as the Internet; private networks; cellular networks; wireless networks; wired networks; combined wireless and wired networks; and satellite networks.

[0066] The communication channel may include or be routed through one or more wired or wireless communications utilizing IEEE 802.11 protocols, Bluetooth, Serial Peripheral Interface (SPI), Inter-Integrated Circuit (IC), or a combination thereof. 2C), universal serial port (USB), controller area network (CAN), cellular data protocol (e.g., 3G, 4G, long term evolution (LTE), 5G), optical communication network, Internet Service Providers (ISP), peer-to-peer network, local area network (LAN), wide area network (WAN), public network (e.g., the Internet), private network, satellite network or others.

[0067] The computing device may include a desktop computer, a laptop computer, a tablet computer, a phablet, a smartphone, a feature phone, a server computer, or other computing device. The server computer may be located remotely in a data center, or may be stored locally.

[0068] The above explained processes are described in the form of computer software and hardware. The described techniques may constitute machine executable instructions contained in a tangible or non-transitory machine (e.g., computer) readable storage medium, which, when executed by a machine, will cause the machine to perform the described operations. In addition, these processes may be embodied in hardware such as an application specific integrated circuit ("ASIC") or other.

[0069] Tangible, non-transitory machine-readable storage media include any mechanism that provides (i.e., stores) information in a form accessible to a machine (e.g., a computer, a network device, a personal digital assistant, a manufacturing tool, any device having a set of one or more processors, etc.). For example, machine-readable storage media include recordable / non-recordable media (e.g., read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

[0070] The above description of the multiple embodiments of the invention shown (including what is described in the Abstract) is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Although specific embodiments and examples of the invention are described herein for illustrative purposes, various modifications may be made within the scope of the invention as will be appreciated by those skilled in the relevant art.

[0071] These modifications may be made to the present invention in light of the above detailed description. The terms used in the appended claims should not be interpreted as limiting the present invention to the specific embodiments disclosed in the specification. In contrast, the scope of the present invention will be determined entirely by the appended claims, which will be interpreted according to established claim interpretation principles.

Claims

1. A system for imaging an ear region, the system comprising: A transmitter, the transmitter being configured to transmit a millimeter wave transmission signal; a receiver configured to receive a return millimeter wave signal; as well as Processing logic, the processing logic is configured to: Driving the transmitter to transmit the millimeter wave transmission signal toward the ear area; receiving an image signal from the receiver, wherein the image signal is generated by the receiver in response to receiving the return millimeter wave signal, the return millimeter wave signal being a millimeter wave transmission signal reflected from the ear region; as well as An ear region image is generated in response to the image signal.

2. The system according to claim 1, further comprising: an optical imaging system configured to capture a facial image of a facial region by sensing visible light or non-visible light reflected from the facial region, the facial region comprising an eye region and a nose region, wherein the facial region overlaps the ear region, and wherein the processing logic is further configured to: receiving the facial image from the optical imaging system; and generating a hybrid face-ear image, the hybrid face-ear image comprising the face image and the ear region image, wherein the face image and the ear region image share a common coordinate system in an imaging space; and preferably The hybrid face-ear image includes the dimensions between the ear saddle point and the eye features of the user's eyes.

3. The system according to claim 1 or 2, wherein: The wavelength of the millimeter wave transmission signal is between 1 mm and 10 mm.

4. A system according to claim 1, 2 or 3, wherein: The ear region includes an ear saddle point.

5. A system according to any one of the preceding claims, wherein: The transmitter includes a two-dimensional array of transmitter elements, and wherein driving the transmitter to transmit the millimeter wave transmit signal includes driving the transmitter elements to beamform the millimeter wave transmit signal to direct the millimeter wave transmit signal to the ear region.

6. A system according to any one of the preceding claims, wherein: The receiver includes a two-dimensional array of receiving elements.

7. A system according to any one of the preceding claims, wherein: Driving the transmitter to transmit the millimeter wave transmission signal includes: driving the transmitter to scan the millimeter wave transmission signal across the ear area.

8. The system according to any one of the preceding claims, further comprising: a second transmitter, the second transmitter being configured to transmit a second millimeter wave transmit signal; as well as a second receiver configured to receive a second returned millimeter wave signal, the second returned millimeter wave signal being the second millimeter wave transmitted signal reflected from the second ear region; and preferably Wherein, the second ear area is the left ear area, and the ear area is the right ear area.

9. The system according to any one of the preceding claims, further comprising: A camera configured to capture one or more location images, wherein the processing logic is further configured to: receiving the one or more location images from the camera; and An ear region of a person is identified based on one or more of the location images.

10. A system for imaging an ear region, the system comprising: A transmitter, the transmitter being configured to transmit a transmission signal, wherein the transmission signal is neither visible light nor invisible light; a receiver configured to receive the return signal; and Processing logic, the processing logic is configured to: driving the transmitter to transmit the transmission signal toward the ear area; receiving an image signal from the receiver, wherein the image signal is generated by the receiver in response to receiving the return signal, the return signal being a transmitted signal reflected from the ear region; and An ear region image is generated in response to the image signal.

11. The system according to claim 10, wherein: The transmission signal is an ultrasonic signal.

12. A method comprising: The transmitter is driven to transmit a millimeter wave signal to the ear area; receiving an image signal from a receiver, wherein the image signal is generated by the receiver in response to receiving a return millimeter wave signal, the return millimeter wave signal being a millimeter wave transmission signal reflected from the ear region; as well as An ear region image is generated in response to the image signal.

13. The method according to claim 12, further comprising: capturing a facial image of a facial region of a person by sensing visible light or non-visible light reflected from the facial region, the facial region including an eye region and a nose region, wherein the facial region overlaps the ear region; and generating a hybrid face-ear image, the hybrid face-ear image comprising the face image and the ear region image, wherein the face image and the ear region image share a common coordinate system in an imaging space; and preferably The method further comprises: A size is determined based on the hybrid face-ear image, wherein the size is between an ear saddle point and a pupil plane of the eye of the user.

14. The method according to claim 12 or 13, wherein: The wavelength of the millimeter wave transmission signal is between 1 mm and 10 mm.

15. The method according to claim 12, 13 or 14, wherein: The ear region includes an ear saddle point; and / or preferably wherein the returning millimeter wave signal propagates through hair before encountering the receiver; and / or preferably Wherein, the method further comprises: An ear region of a person is identified based on the position image of the person.