Head-mounted display device

Abstract

The utility model provides a head-mounted display device, and the device comprises a passive radar which is used for measuring an electromagnetic signal in an environment, so as to generate electromagnetic environment data; the control device is in communication connection with the passive radar and is used for acquiring the electromagnetic environment data and generating an electromagnetic image according to the electromagnetic environment data; and the display device is in communication connection with the control device and is used for acquiring and displaying the electromagnetic image so as to realize visual expression of the electromagnetic environment. According to the head-mounted display device, the passive radar is arranged, so that the head-mounted display device can monitor the electromagnetic environment and visually display the electromagnetic environment, the use function of the head-mounted display device is expanded, and the attraction to consumers is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent wearable devices, in particular to a head-mounted display device. BACKGROUND

[0002] The propagation law of electromagnetic waves in space is extremely complex and invisible, and can only be detected by professional monitoring equipment. How to use visual technical means to intuitively display various electromagnetic wave propagation signals and radiation intensity in a place has been an important problem to be solved in the field of electromagnetic environment visualization technology.

[0003] A head-mounted display device (HMD) can use computer-generated additional information to enhance or expand the real-world scene seen by the user, greatly changing the way humans interact with computers or the external world. This device combines various technologies from different research fields and is quickly applied to entertainment, scientific research, simulation training, telemedicine and other fields.

[0004] In the current technology, the head-mounted display device cannot monitor and visually display the electromagnetic environment, resulting in a single function of the head-mounted display device and insufficient attraction to consumers. Practical new type content

[0005] The embodiment of the present application provides a head-mounted display device, by setting a passive radar, so that the head-mounted display device can monitor and visually display the electromagnetic environment, thereby expanding the use function of the head-mounted display device and improving the attraction to consumers.

[0006] In a first aspect, a head-mounted display device is provided, comprising: a passive radar configured to measure electromagnetic signals in an environment to generate electromagnetic environment data; a control device in communication with the passive radar, configured to obtain the electromagnetic environment data and generate an electromagnetic image based on the electromagnetic environment data; and a display device in communication with the control device, configured to obtain and display the electromagnetic image to realize visual expression of the electromagnetic environment.

[0007] The head-mounted display device provided in the embodiments of the present application is provided with a passive radar, the passive radar can measure electromagnetic signals in an environment to generate electromagnetic environment data. A control device can generate an electromagnetic image according to the electromagnetic environment data, and display the electromagnetic image through a display device, so as to realize visual expression of the electromagnetic environment. The head-mounted display device provided in the embodiments of the present application can monitor and visually display the electromagnetic environment, and can help the user to interact with the electromagnetic environment in real time and dynamically master the change of the electromagnetic environment. In this way, the use function of the head-mounted display device is expanded, the head-mounted display device can be applied to more scenes, and the interest and playability of the product and the attraction to consumers are improved.

[0008] In some examples, the electromagnetic image can visually express the electromagnetic environment through any expression form such as text, graphics or table, which is not specially limited in the present application. For example, the electromagnetic image can include an electromagnetic field distribution diagram, a frequency spectrum diagram or an electromagnetic situation diagram.

[0009] In some examples, the electromagnetic image can include part or all of the electromagnetic environment data.

[0010] In some examples, the control device acquires the electromagnetic environment data generated by the passive radar in real time, and generates the electromagnetic image in real time according to the electromagnetic environment data, and the display device acquires and displays the electromagnetic image in real time. Through the above setting, real-time display and dynamic refreshing of the electromagnetic image can be realized, which helps the user to timely obtain information of the electromagnetic environment and dynamically master the change of the electromagnetic environment.

[0011] In a possible implementation, the head-mounted display device further includes: a visual sensor, which is in communication connection with the control device and is configured to collect image data in a target visual field; and the control device is specifically configured to acquire the electromagnetic environment data and the image data in real time, and to superimpose the electromagnetic environment data and the image data in real time to generate the electromagnetic image.

[0012] Through the above setting, the electromagnetic environment data can be fused with the real observation scene, so that the user can immerse in the observation environment to watch the information (for example, part or all of the electromagnetic environment data) of the electromagnetic environment, thereby improving the efficiency of visual expression, improving the viewing effect of the user, and helping the user to more intuitively obtain the information of the electromagnetic environment. Through the above setting, the interest and playability of the head-mounted display device can be increased, which is beneficial to improve the attraction of the product to consumers.

[0013] In some examples, the visual sensor can be a camera, a visible light camera or a camera sensor, for example, the visual sensor can be an RGB camera or an infrared camera, but is not limited thereto.

[0014] In some examples, the electromagnetic environment data can be superimposed or fused with the image data by VST technology to generate the electromagnetic image.

[0015] In a possible implementation, the head-mounted display device further comprises: an active radar, which is in communication connection with the control device, and is configured to measure position data of an object in the target field of view relative to the active radar; and the control device is specifically configured to: acquire the position data in real time, and superimpose the electromagnetic environment data, the image data and the position data in real time to generate the electromagnetic image.

[0016] By the above arrangement, the distribution of the electromagnetic environment in the three-dimensional space can be completely described, the three-dimensional visualization of the electromagnetic environment is realized, and the multi-view observation demand of the user for the overall distribution and the detailed features of the electromagnetic environment (for example, electromagnetic field) is met. The correctness, accuracy and completeness of the electromagnetic environment expression are improved.

[0017] In some examples, the active radar can be any one of a laser radar, a structured light depth sensor or a time of flight (TOF) sensor.

[0018] In some examples, the position data can be three-dimensional coordinate information, for example, coordinate information of the object in a reference coordinate system of the active radar.

[0019] In some examples, the position information can include distance information and angle information of the object relative to the active radar.

[0020] In a possible implementation, the active radar emits a plurality of electromagnetic wave signals with different angles to the target field of view when in operation, and receives a plurality of reflected electromagnetic wave signals reflected by the object in the target field of view, so as to acquire the position data of the object in the target field of view relative to the active radar.

[0021] In a possible implementation, the electromagnetic image comprises a three-dimensional model of the target field of view.

[0022] In a possible implementation, the display device is a see-through display device.

[0023] Since the see-through display device can make the natural reflection light of the environmental object normally pass through, the user can view the surrounding environment and things through the see-through display device without affecting the wearer's line of sight. The see-through display device can also allow external light to pass through so as to send the electromagnetic image and the background light source into the wearer's eyes at the same time, and realize modification and enhancement of the background image. That is, the head-mounted display device provided by the embodiments of the present application can be an AR device, and the user's eyes can directly obtain external environmental information through the display device without the help of a camera.

[0024] In a possible implementation, the passive radar is detachably connected with the main body of the head-mounted display device, and when the passive radar and the main body are separated from each other, the passive radar and the control device are connected through a wireless network.

[0025] By being separately arranged, the passive radar can also be arranged at a remote position such as a top of an unmanned aerial vehicle or a high-rise building, electromagnetic environment data at a remote place is obtained through the passive radar, and visualization expression of the electromagnetic environment of an area that the user cannot reach is realized, thereby enabling the head-mounted display device to be applied to more scenes and improving product use performance.

[0026] In some examples, detachable connection between the passive radar and the main body of the head-mounted display device can be achieved through clamping or screwing, but is not limited thereto.

[0027] In a possible implementation, the electromagnetic environment data includes at least one of the following data: number of radiation sources, spatial direction of radiation sources, type of radiation sources, electromagnetic wave intensity, electromagnetic wave amplitude, electromagnetic wave frequency and direction of arrival.

[0028] In a possible implementation, the head-mounted display device further includes a wireless communication module, which is in communication connection with the control device and is configured to send the electromagnetic image to an external electronic device.

[0029] Through the above arrangement, the electromagnetic environment can be visualized by means of the external electronic device, thereby enabling the head-mounted display device to be applied to more scenes and improving product use performance.

[0030] In a possible implementation, the head-mounted display device is an augmented reality device, a virtual reality device or a mixed reality device. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a wearing schematic view of the head-mounted display device provided by the embodiments of the present application.

[0032] Figure 2Fig. 1 is a schematic diagram of a modular structure of a main body provided by an embodiment of the present application.

[0033] Figure 3 Fig. 2 is a block diagram of a method for realizing electromagnetic environment visualization by a head-mounted display device provided by an embodiment of the present application.

[0034] Figure 4 Fig. 3 is a schematic diagram of an application scenario of a head-mounted display device provided by an embodiment of the present application.

[0035] Figure 5 Fig. 4 is a schematic diagram of another application scenario of a head-mounted display device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0036] The embodiments of the present application are described in detail below with reference to the accompanying drawings. The embodiments described below are examples for explaining the present application and should not be understood as limiting the present application.

[0037] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connecting" should be understood in a broad sense, for example, can be fixedly connected, or detachably connected, or integrally connected; can be mechanically connected, or electrically connected or can communicate with each other; can be directly connected, or indirectly connected through an intermediate medium, can be the internal connection of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0038] In the description of the present application, it should be understood that the terms "first", "second" and the like are only for the purpose of description and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features limited by "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise explicitly specified and limited.

[0039] In the description of the present application, it should be understood that the terms "up", "down", "side", "front", "back" and the like indicate the orientation or positional relationship based on the installation, and are only for the purpose of facilitating the description of the present application and simplifying the description, and do not indicate or imply that the devices or elements indicated must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present application.

[0040] For the convenience of understanding the embodiments of the present application, the following first introduces the electromagnetic environment, visualization, active radar, passive radar, head-mounted display device and video see-through and other related terms involved in the embodiments of the present application.

[0041] Electromagnetic environment: the general term of all electromagnetic phenomena existing in a given place or space. The electromagnetic environment is composed of three elements of space, time and frequency spectrum. It can be simply understood as an electromagnetic field phenomenon, that is, the electromagnetic induction and interference phenomenon universally existing in the environment. In actual life, the electromagnetic environment can be affected by various factors, including natural and man-made electromagnetic emissions. For example, power transmission lines, radio broadcasting, television, microwave communication, household appliances such as microwave ovens, and various industrial high-frequency equipment can all constitute a complex electromagnetic environment.

[0042] Visualization: visualization technology is a technology that emerged in the 1980s. Its meaning is to display complex data in the form of graphics (images) that people can easily understand and accept in a highly immersive display environment, providing a visual interactive means for people to calculate and simulate data. The visualization of electromagnetic environment is to analyze the type, attribute and distribution of various electromagnetic signals, and to use visualization methods such as graphics, images and analysis reports to show the electromagnetic situation.

[0043] Active radar: refers to a radar that actively transmits signals and detects targets by using the coherence of target echoes and transmitted signals.

[0044] Passive radar: a passive radar that does not send any signal, but detects targets by receiving electromagnetic wave signals emitted by the target itself.

[0045] Head-mounted display device (HMD): refers to a display device that can be worn on the head. HMD generally uses "near-eye optical system" to display graphical images and other multimedia information on a display screen (such as a lens) at a distance of a few centimeters from the eyeball, and can achieve different visual effects such as augmented reality (AR), virtual reality (VR) and mixed reality (MR) by sending optical signals to the user's eyes, providing users with a full range of immersive experience.

[0046] Video see through (VST): the VST technology can realize seamless fusion interaction between a virtual scene and a real object. The principle is that a miniature camera installed on an HMD takes an image of an external real environment, a computer superimposes information or an image signal to be added on a video signal of the camera through calculation processing, a virtual scene generated by the computer is fused with a real scene through a video signal fusion device, and finally the display system on the HMD presents the user.

[0047] In view of the technical problems described in the background art, the embodiment of the present application provides a head-mounted display device, which can monitor and visually display an electromagnetic environment by setting a passive radar, can help a user to interact with the electromagnetic environment in real time, and dynamically master the changes of the electromagnetic environment. Thus, the use function of the head-mounted display device is expanded, the head-mounted display device can be applied to more scenes, and the interest and playability of the product and the attraction to consumers are improved.

[0048] Figure 1 is a wearing schematic diagram of the head-mounted display device 200 provided by the embodiment of the present application. As shown in Figure 1 The embodiment of the present application provides a head-mounted display device 200, which is a display device worn on the head of a wearer to provide an immersive experience different from a television, a mobile phone and the like. The head-mounted display device 200 can be a VR device, an AR device or an MR device, and the form of the head-mounted display device 200 can be glasses, an eye cover, a helmet or a mask, but is not limited thereto.

[0049] As shown in Figure 1 When the head-mounted display device 200 is worn on the head of a user, the eyes of the user can see the images presented on the display screen of the head-mounted display device 200. When the display screen is transparent, the eyes of the user can also see the real objects in front through the display screen. The application scenarios of the head-mounted display device 200 mainly involve interactions with virtual scenes, for example, an application that can recognize gestures and facial expressions of a person and interact in real time with a certain communication protocol at a high code rate.

[0050] As shown in Figure 1 The head-mounted display device 200 provided by the embodiment of the present application includes a main body part 210 and a wearing part 220. When a user wears the head-mounted display device 200, the main body part 210 is located in front of the eyes of the user to provide a display picture for the eyes of the user, and the wearing part 220 is used to be fixed on the head or the ear of the user to realize reliable wearing of the head-mounted display device 200. In Figure 1In some examples, the wearing part 220 is in the form of a strap. In other examples, the wearing part 220 can also be in the form of a glasses leg, a headband, or a helmet to hold the main part 210 on the head of the user. When the wearing part 220 is in the form of a helmet, the wearing part 220 and the main part 210 can also be in an integrated structure.

[0051] Figure 2 is a schematic diagram of a modular structure of the main part 210 provided by an example of the present application. As shown in Figure 2 the main part 210 includes a housing 211, a passive radar 212, a control device 213, and a display device 214. In some examples, the main part 210 can also include a visual sensor 215, an active radar 216, and a wireless communication module 217, etc.

[0052] The housing 211 can be a mounting base or a rack of various components of the head-mounted display device 200. The passive radar 212, the control device 213, the display device 214, the visual sensor 215, the active radar 216, and the wireless communication module 217, etc. can be mounted on the housing 211 or inside the housing 211. The material of the housing 211 can be any material, such as metal, plastic, polymer, etc. The housing 211 can include rigid components or flexible components, so that the housing 211 has sufficient structural strength and better wearing comfort.

[0053] In some examples, the housing 211 is also provided with an external interface for connecting with an external device, such as a universal serial bus (USB) interface or a lightning interface. The head-mounted display device 200 is connected with the external device through the external interface, so that the electrical signals are transmitted between the functional circuit inside the electronic device and the external device, to realize the corresponding interface functions, such as charging function or data transmission function, etc. For example, the external interface can be a Type-C interface.

[0054] The passive radar 212 can be mounted on the housing 211, for example, in a corresponding mounting slot. The passive radar 212 measures the electromagnetic signals in the environment during operation to generate electromagnetic environment data. The passive radar 212 is in communication connection with the control device 213, and can send the generated electromagnetic environment data to the control device 213 in real time.

[0055] In some examples, the passive radar 212 measures electromagnetic signals in the environment, which can be electromagnetic waves emitted by a radiation source in the environment, and through signal sorting and other processing steps, the amplitude, direction of arrival, frequency and other electromagnetic environment data of the electromagnetic waves are measured.

[0056] In some examples, the electromagnetic environment data includes at least one of the following data: number of radiation sources, spatial orientation of radiation sources, type of radiation sources, electromagnetic wave intensity, electromagnetic wave amplitude, electromagnetic wave frequency, and direction of arrival.

[0057] In some examples, the passive radar 212 can determine the type of radiation source corresponding to the electromagnetic wave according to the electromagnetic wave intensity, electromagnetic wave frequency, modulation mode and other electromagnetic wave information, combined with a preset algorithm. For example, the preset algorithm can include a preset mapping relationship table or an artificial intelligence (AI) model. The type of radiation source may, for example, include any electronic device type such as a base station, a router, a mobile phone, a tablet computer, a notebook computer, a smart watch (bracelet), etc., which is not specifically limited in the present application.

[0058] For example, the passive radar 212 has a local storage of a radiation source type library, which includes a correspondence between a plurality of radiation source types and electromagnetic wave information. The passive radar 212 matches the measured electromagnetic wave intensity, electromagnetic wave amplitude, electromagnetic wave frequency and other information with the radiation source type library to identify the type of radiation source. For example, it is known that the electromagnetic wave #A in a certain environment corresponds to an electromagnetic wave frequency of 5MHz, and the local radiation source type library corresponds to the type #B of the radiation source type with a frequency of 5MHz. Therefore, a matching relationship is established, i.e. the type of radiation source corresponding to the electromagnetic wave #A is determined to be type #B.

[0059] The control device 213 is installed on the housing 211 or can be installed inside the housing 211. The control device 213 is in communication connection with the passive radar 212, for obtaining the electromagnetic environment data generated by the passive radar 212, and generating an electromagnetic image according to the electromagnetic environment data. The control device 213 can obtain the electromagnetic environment data generated by the passive radar 212 in real time, and generate the electromagnetic image in real time.

[0060] The control device 213 can include a hardware device with data information processing function and a program required to drive the hardware device to work. Of course, the control device 213 can also be only a hardware device with data processing capability, or only a program running in a hardware device. In some examples, the control device 213 can include a circuit board and a system on chip (SoC) disposed on the circuit board. The control device 213 can include at least one storage medium and at least one central processing unit.

[0061] The storage medium can include a data storage device. The data storage device can be a non-transitory storage medium or a transitory storage medium. For example, the data storage device can include one or more of a magnetic disk, a read-only memory (ROM), or a random access memory (RAM). The storage medium also includes at least one set of instructions stored in the data storage device. The instructions are computer program codes, which can include programs, routines, objects, components, data structures, processes, modules, and the like that perform the method of data superimposition processing provided in the present specification.

[0062] The at least one central processor can be communicatively connected to the at least one storage medium. The at least one central processor is used to execute the at least one set of instructions described above. When the control device is running, the at least one central processor reads the at least one set of instructions and performs the generation steps (processes) of the electromagnetic image according to the instructions of the at least one set of instructions. The central processor can be in the form of one or more processors, and in some embodiments, the central processor can include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physics processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of executing one or more functions, or the like, or any combination thereof.

[0063] The display device 214 is mounted on the housing 211, and the display device 214 is communicatively connected to the control device 213, for receiving and displaying the electromagnetic image. For example, the display device 214 receives and displays the electromagnetic image in real time.

[0064] When the head-mounted display device 200 is worn on the head of the user, the display apparatus 214 is located in front of the eyes of the user. The display apparatus 214 can include a display screen and components such as imaging lenses, for providing a display picture for the user, for example, to realize a virtual picture. The number of display apparatuses 214 can be two, respectively corresponding to the left eye and the right eye of the user. The two display apparatuses 214 can achieve the purpose of stereoscopic display through staggered display, picture exchange or parallax fusion. In other examples, the number of display apparatuses 214 can also be one, and simultaneously corresponding to the left eye and the right eye of the user, at this time, the user's eyes can simultaneously watch the electromagnetic image displayed on the display apparatus 214.

[0065] The head-mounted display device 200 provided by the embodiment of the present application is provided with the passive radar 212, which can measure electromagnetic signals in the environment to generate electromagnetic environment data. The control apparatus 213 can generate an electromagnetic image according to the electromagnetic environment data, and display the electromagnetic image through the display apparatus 214, so as to realize the visual expression of the electromagnetic environment. The head-mounted display device 200 provided by the embodiment of the present application can monitor and visually display the electromagnetic environment, and can help the user to interact with the electromagnetic environment in real time and dynamically master the changes of the electromagnetic environment. Thus, the use function of the head-mounted display device 200 is expanded, so that the head-mounted display device 200 can be applied to more scenes, and the interest and playability of the product are improved, and the attraction to consumers is improved.

[0066] In some examples, the electromagnetic image can visually express the electromagnetic environment through any expression form such as text, graphics or table, which is not specially limited by the present application. For example, the electromagnetic image can include an electromagnetic field distribution diagram, a frequency spectrum diagram or an electromagnetic situation diagram.

[0067] In some examples, the electromagnetic image can include part or all of the data of the electromagnetic environment data.

[0068] In some examples, the control apparatus 213 acquires the electromagnetic environment data generated by the passive radar 212 in real time, and generates the electromagnetic image in real time according to the electromagnetic environment data, and the display apparatus 214 acquires and displays the electromagnetic image in real time.

[0069] Through the above setting, the real-time display and dynamic refreshing of the electromagnetic image can be realized, which helps the user to timely know the information of the electromagnetic environment and dynamically master the changes of the electromagnetic environment.

[0070] In some examples, the display apparatus 214 is a see-through display apparatus.

[0071] Because transmissive display devices allow naturally reflected light from environmental objects to pass through normally, users can view their surroundings and objects without obstructing their vision. Transmissive display devices also allow external light to pass through, enabling the simultaneous delivery of electromagnetic images and background light into the wearer's eyes, thus modifying and enhancing the background image. In other words, the head-mounted display device 200 provided in this application embodiment can be an AR device, allowing the user's eyes to directly obtain external environmental information through the display device 214 without the need for a camera.

[0072] In some examples, the passive radar 212 is detachably connected to the main body 210 of the head-mounted display device 200, for example, to the housing 211. When the passive radar 212 is separated from the main body 210, the passive radar 212 is connected to the control device 213 via a wireless network.

[0073] By implementing a separate configuration, the passive radar 212 can also be placed at distant locations such as drones or the top of tall buildings. The passive radar 212 can acquire electromagnetic environment data from a distance, enabling a visual representation of the electromagnetic environment in areas inaccessible to the user. This allows the head-mounted display device 200 to be applied in more scenarios, improving product performance.

[0074] In some examples, the passive radar 212 and the main body 210 (e.g., housing 211) of the head-mounted display device 200 can be detachably connected by means of snap-fit ​​or screw-fit, but are not limited thereto.

[0075] like Figure 2 As shown, the head-mounted display device 200 provided in this embodiment of the application also includes a vision sensor 215 disposed on the housing 211. The vision sensor 215 is communicatively connected to the control device 213 and is used to acquire image data within the target's field of view. The control device 213 is specifically used to acquire electromagnetic environment data and image data in real time, and to overlay the electromagnetic environment data and image data in real time to generate the electromagnetic image.

[0076] By implementing the above settings, electromagnetic environment data can be integrated with real-world observation scenarios, allowing users to immerse themselves in the observation environment and view electromagnetic environment information (such as part or all of the electromagnetic environment data). This improves the efficiency of visualization, enhances the user's viewing experience, and helps users obtain electromagnetic environment information more intuitively. Furthermore, these settings increase the fun and playability of the head-mounted display device 200, thereby increasing the product's appeal to consumers.

[0077] In some examples, the vision sensor 215 may be a camera, a visible light camera, or a camera sensor, such as an RGB camera or an infrared camera, but is not limited to these.

[0078] In some examples, the target field of view may be the sensing range of the vision sensor 215; or, the sensing range of the vision sensor 215 may include the target field of view. Alternatively, the target field of view may include the sensing range of the vision sensor 215. The target field of view can be set and changed according to usage requirements.

[0079] In some examples, the electromagnetic environment data can be overlaid or fused with image data using the aforementioned VST technology to generate the electromagnetic image.

[0080] like Figure 2 As shown, the head-mounted display device 200 provided in this application embodiment also includes an active radar 216 disposed on the housing 211. The active radar 216 is communicatively connected to the control device 213 and is used to measure the position data of objects within the target's field of view relative to the active radar. Specifically, the control device 213 is used to acquire the position data in real time and to overlay the electromagnetic environment data, image data, and position data in real time to generate the aforementioned electromagnetic image.

[0081] The above settings enable a complete description of the electromagnetic environment's distribution in three-dimensional space, achieving a three-dimensional visualization of the electromagnetic environment and meeting users' needs for multi-view observation of the overall distribution and detailed features of the electromagnetic environment (e.g., electromagnetic fields). This improves the accuracy, precision, and completeness of the electromagnetic environment representation.

[0082] The active radar 216 can measure the position data of the objects in the target field of view relative to the active radar 216 when in operation. The position data can be, for example, three-dimensional coordinate information. That is, the active radar 216 can measure the coordinate information of the objects in the target field of view in the reference coordinate system of the active radar 216 when in operation. The active radar 216 can be used to measure the distance and angle, calculate the distance and angle of the objects relative to the active radar 216 by emitting microwave signals, receiving reflected signals of the microwave signals reflected by the objects, and calculating the time difference. Specifically, the active radar 216 can include a transmitter and a receiver. The transmitter can emit electromagnetic wave signals outwardly. Specifically, the transmitter can emit electromagnetic wave signals at multiple different emission angles outwardly. The receiver can receive electromagnetic wave signals reflected by the objects in the target field of view. Specifically, the receiver can receive multiple reflected electromagnetic wave signals reflected by the objects in the target field of view. The active radar 216 can determine the distance of the objects at the emission angle of the current electromagnetic wave signal relative to the active radar 216 based on the time difference between each electromagnetic wave signal and its corresponding reflected electromagnetic wave signal, and determine the three-dimensional coordinate information of the objects at the emission angle relative to the active radar 216 based on the emission angle of the current electromagnetic wave signal.

[0083] In some examples, the target field of view can be the sensing range of the active radar 216; or, the sensing range of the active radar 216 can include the target field of view. Alternatively, the target field of view can include the sensing range of the active radar 216. The target field of view can be set and changed according to the use requirements.

[0084] In some examples, the active radar 216 can be used for three-dimensional space scanning, thereby establishing a three-dimensional model of the target field of view. Specifically, the active radar 216 can obtain the three-dimensional coordinate information of the objects at different positions in the target field of view relative to the active radar 216, thereby obtaining the relative coordinate information between the objects in the target field of view. When the position and attitude of the active radar 216 change, the control device 213 can select a fixed point in the target field of view as a reference point, calculate the three-dimensional coordinate information of the objects at different positions relative to the reference point according to the three-dimensional coordinate information of the objects at different positions relative to the active radar 216 obtained by the active radar 216 at different times, thereby establishing a three-dimensional model of the objects in the target field of view. The active radar 216 can collect position data in real time, and the control device 213 can perform three-dimensional modeling in real time.

[0085] In some examples, the stereoscopic image can include three-dimensional model data within the target field of view. As mentioned above, the lidar 200 can be used for three-dimensional space scanning. The control device 213 can determine relative position relationships of objects within the target field of view based on the position data, thereby constructing a three-dimensional model of the target field of view. The control device 213 can further superimpose the aforementioned electromagnetic environment data, image data, and the three-dimensional model, to obtain a three-dimensional (3D) electromagnetic image, i.e., an electromagnetic image containing a three-dimensional model within the target field of view, thereby enabling three-dimensional visualization of the electromagnetic environment.

[0086] In some examples, the active radar 216 can be any one of a lidar, a structured light depth sensor, or a time of flight (TOF) sensor, for example.

[0087] As shown in Figure 2 The head-mounted display device 200 provided by the embodiments of the present application further includes a wireless communication module 217 disposed in the housing 211, to achieve wireless communication with other devices. The wireless communication module 217 can include a wireless fidelity (Wi-Fi) module, a bluetooth (BT) module, etc., but is not limited thereto.

[0088] In some examples, the wireless communication module 217 is in communication connection with the control device 213, for sending the electromagnetic image to an external electronic device.

[0089] With the above arrangement, the electromagnetic environment can be visualized by means of the external electronic device, thereby enabling the head-mounted display device 200 to be applied to more scenarios and improving the product use performance.

[0090] In some examples, the head-mounted display device 200 can further include one or more of a battery for powering the above-mentioned modules, an eye tracking system for identifying the user's line of sight, a speaker for outputting audio, a microphone for obtaining user voice instructions, and various types of sensor functional modules, without limitation.

[0091] Figure 3 is a method block diagram of the electromagnetic environment visualization method implemented by the head-mounted display device 200 provided by the embodiments of the present application. As shown in Figure 3 The electromagnetic environment visualization method generally includes the following steps S1-S5.

[0092] Step S1, the passive radar 212 receives electromagnetic signals of the radiation source, and measures electromagnetic environment data such as the amplitude, direction of arrival, frequency, etc. of the electromagnetic wave through signal sorting. Further, the passive radar 212 can also determine the type of the radiation source corresponding to the electromagnetic wave according to the electromagnetic wave information such as the electromagnetic wave intensity, electromagnetic wave frequency, modulation mode, etc., in combination with a preset algorithm. For example, the preset algorithm can include a preset formula, a mapping relationship table or an AI model. The type of the radiation source may, for example, include any electronic device type such as a base station, a router, a mobile phone, a tablet computer, a notebook computer, a smart watch (bracelet), etc., which is not particularly limited in the present application.

[0093] In some examples, the passive radar 212 has a local storage of a radiation source type library including a plurality of corresponding relationships between radiation source types and electromagnetic wave information. The passive radar 212 matches the measured information such as the electromagnetic wave intensity, electromagnetic wave amplitude, electromagnetic wave frequency, etc. with the radiation source type library to identify the type of the radiation source. For example, it is known that the electromagnetic wave #A in a certain environment corresponds to an electromagnetic wave frequency of 5MHz, and the local radiation source type library corresponds to the type #B of the radiation source type with a frequency of 5MHz. Therefore, a matching relationship is established, i.e. the type of the radiation source corresponding to the electromagnetic wave #A is determined as type #B.

[0094] The passive radar 212 reports or sends the measured electromagnetic environment data to the control device 213. The electromagnetic environment data includes at least one of the number of radiation sources, the spatial orientation of the radiation source, the type of the radiation source, the electromagnetic wave intensity, the electromagnetic wave amplitude, the electromagnetic wave frequency and the direction of arrival, etc., but is not limited thereto.

[0095] Step S2, the active radar 216 includes a signal transmitting unit and a signal receiving unit. The signal transmitting unit is used to transmit electromagnetic wave signals outward, for example, to transmit electromagnetic wave signals with different transmission angles outward. The receiver can receive electromagnetic wave signals reflected back by objects in the target field of view, and can calculate position data such as the direction and distance of the target (object) relative to the active radar 216. The radar echo signal contains effective information for target identification, and common means such as waveform matching and feature point extraction are currently used. Further, the active radar 216 can also determine the type of the target corresponding to the target according to the measured position data in combination with a preset algorithm. For example, the preset algorithm can include a preset formula, a mapping relationship table or an AI model. The target type may, for example, include various types of home products such as sofas, chairs, tables, beds, refrigerators or walls, etc., which is not particularly limited in the present application.

[0096] In some examples, the active radar 216 has a local target type library stored therein, which includes a plurality of target types and a plurality of corresponding position data. The active radar 216 matches the measured position data with the target type library to identify a corresponding target type. For example, it is known that position data #A in a certain environment matches the position data corresponding to type #B in the local target type library, and a matching relationship is established, i.e., it is determined that the target type corresponding to the position data #A is type #B.

[0097] The active radar 216 reports or sends the measured position data to the control device 213. The position data includes at least one of angle information, distance information, three-dimensional coordinate information, and the like of the object, but is not limited thereto.

[0098] In step S3, the visual sensor 215 acquires surrounding environment information and performs a series of processing on the image, including but not limited to image stitching, noise removal, etc., to obtain image data of the target field of view. At this time, the head-mounted display device 200 provided by the embodiments of the present application can be a VR device or an MR device.

[0099] In other examples, the display device 214 of the head-mounted display device 200 can be a see-through display device. At this time, the head-mounted display device 200 provided by the embodiments of the present application can be an AR device, and the user's eyes can directly obtain external environment information through the display device 214 without the aid of the visual sensor 215, i.e., the head-mounted display device 200 can not be provided with the visual sensor 215.

[0100] The visual sensor 215 reports or sends the acquired image data to the control device 213.

[0101] In step S4, the control device 213 can include a virtual engine module and a visualization display module. The virtual engine module is mainly used for model generation, space estimation, target fusion, etc. Its functions can be seen but are not limited to the following examples: after the A, B information (i.e., position data) output by the active radar 216 and the C, D information (i.e., electromagnetic environment data) output by the passive radar are input into the virtual engine module, a corresponding target model is generated, the spatial position of the model and the arrangement of the electromagnetic waves are calculated, and finally the A, B, C, D objects and electromagnetic wave signals are reasonably arranged according to the depth and other spatial information.

[0102] The visualization display module is mainly responsible for virtual-real fusion and visualization reconstruction. Its specific functions include but are not limited to virtual-real occlusion, distortion calculation, picture quality (PQ) adjustment, image rendering, etc. The objects of the actual physical environment and the objects generated by the virtual engine are effectively fused, and the electromagnetic beams are correctly arranged according to the spatial calculation results, and finally the electromagnetic image to be displayed is generated.

[0103] The control device 213 sends the generated electromagnetic image to the display device 214.

[0104] In step S5, the display device 214 receives the electromagnetic image from the control device 213 in real time and displays the electromagnetic image in real time.

[0105] Figure 4 This is a schematic diagram illustrating an application scenario of the head-mounted display device 200 provided in this application embodiment. The head-mounted display device 200 provided in this application embodiment can be used to assist in base station layout. Figure 4 As shown, a communication system needs to be built in Gymnasium 100 and the base station layout needs to be optimized. The number and placement of base stations within Gymnasium 100 can be adjusted. For example, Gymnasium 100 currently includes four base stations: base station 110, base station 120, base station 130, and base station 140. These four base stations are distributed in the four corners of Gymnasium 100. The following steps can be used to assist in the base station layout:

[0106] Step 1: Select a base station layout method, for example... Figure 4 The base station layout shown in the image.

[0107] Step 2, move the head-mounted display device 200 to a certain position, for example... Figure 4 The image shows the central location of stadium 100;

[0108] Step 3: Activate the passive radar 212 of the head-mounted display device 200 to measure electromagnetic environment data, and report the measured electromagnetic environment data to the control device 213. The electromagnetic environment data may include electromagnetic signal indicators such as signal strength and direction.

[0109] Step 4: The control device 213 generates a virtual image based on the electromagnetic environment data. The virtual image may include, for example, text or image models.

[0110] Step 5: The vision sensor 215 of the head-mounted display device 200 acquires image data (i.e., real image) of the stadium 100 and reports the image data to the control device 213. The control device 213 fuses and overlays the virtual image from step 4 with the image data to generate an electromagnetic image. The display device 214 then visualizes and displays the electromagnetic image.

[0111] Step 6: Repeat steps 1-5 until the electromagnetic signal distribution within the stadium 100 meets the requirements, and the base station layout is completed.

[0112] The gym is crowded, and the signal of mobile phones and other devices is often poor. The head-mounted display device 200 provided by the embodiment of the application can distribute the signal of the gym on the display device of the device to reproduce stereoscopically, thereby assisting the staff to layout the base station. The scheme is simple to operate, and the electromagnetic signal index can be displayed in real time by wearing the head-mounted display device 200, which is beneficial to improving the efficiency of the staff to layout the base station.

[0113] Figure 5 FIG. 2 is another application scenario of the head-mounted display device 200 provided by the embodiment of the application. The head-mounted display device 200 provided by the embodiment of the application can also be used to search for a radiation source object (such as a mobile terminal such as a mobile phone and a smart watch). As shown in FIG. 2, the radiation source 400 can radiate electromagnetic signals, and the radiation source 400 is blocked by the shelter 300 (such as a sofa). At this time, the head-mounted display device 200 can be used to search for the radiation source 400 according to the following steps. Figure 5

[0114] Step 1, the passive radar 212 of the head-mounted display device 200 measures electromagnetic environment data, and reports the measured electromagnetic environment data to the control device 213. The electromagnetic environment data includes, for example, the direction of the radiation source 400 (that is, the direction of the incoming electromagnetic wave) and other electromagnetic signal indicators.

[0115] Step 2, the active radar 216 scans the contour of the shelter 300, that is, measures the position data of the shelter 300 relative to the active radar 216, such as the direction and distance, and reports the position data to the control device 213.

[0116] Step 3, the control device 213 identifies the type of the shelter 300 (for example, a sofa) according to the position information, and generates an electromagnetic image to be displayed in combination with the electromagnetic environment data in step 1.

[0117] Step 4, the display device 214 visually displays the electromagnetic image. The electromagnetic image includes, for example, the direction and position of the shelter 300, and the direction and position of the radiation source 400 (behind the shelter 300).

[0118] The head-mounted display device 200 provided by the embodiment of the application can also be used to identify hidden electromagnetic radiation bodies. The passive radar 212 can identify the direction of the hidden electromagnetic radiation bodies, and the active radar 216 can perform targeted scanning, thereby solving the problem of insufficient identification capability of the existing head-mounted display device for the hidden electromagnetic radiation bodies.

[0119] ​Due to the ability to monitor and visualize the electromagnetic environment, the head-mounted display device 200 provided by the embodiments of the present application can also rely on this function to expand other functions such as signal detection, target identification, target tracking, and the like. In addition, the head-mounted display device 200 provided by the embodiments of the present application can also be applied to the field of 3D reconstruction, and used to assist in 3D reconstruction.

[0120] The above merely provides a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A head-mounted display device, comprising: The head-mounted display device further comprises: a visual sensor, in communication connection with the control device, for collecting image data within a target field of view; the control device is specifically configured to acquire the electromagnetic environment data and the image data in real time, and to superimpose the electromagnetic environment data and the image data in real time to generate the electromagnetic image. The head-mounted display device further comprises:

2. The head-mounted display device of claim 1, wherein, an active radar, in communication connection with the control device, for measuring position data of objects within the target field of view relative to the active radar; the control device is specifically configured to acquire the position data in real time, and to superimpose the electromagnetic environment data, the image data and the position data in real time to generate the electromagnetic image. The active radar emits a plurality of electromagnetic wave signals of different angles to the target field of view during operation, and receives a plurality of reflected electromagnetic wave signals reflected by objects within the target field of view, thereby acquiring the position data of the objects within the target field of view relative to the active radar.

3. The head-mounted display device of claim 2, wherein, The electromagnetic image comprises a three-dimensional model of the target field of view. The display device is a see-through display device. The passive radar is detachably connected with a main body part of the head-mounted display device, and the passive radar and the control device are in communication connection through a wireless network when the passive radar and the main body part are separated from each other.

4. The head-mounted display device of claim 3, wherein, The electromagnetic environment data comprises at least one of the following data: number of radiation sources, spatial orientation of radiation sources, type of radiation sources, electromagnetic wave intensity, electromagnetic wave amplitude, electromagnetic wave frequency and direction of arrival.

5. The head-mounted display device of claim 3, wherein, The head-mounted display device further comprises:

6. The head-mounted display device of claim 1, wherein, a wireless communication module, in communication connection with the control device, for sending the electromagnetic image to an external electronic device.

7. The head-mounted display device of any of claims 1-6, wherein, The head-mounted display device is an augmented reality device, a virtual reality device or a mixed reality device.

8. The head-mounted display device of any one of claims 1-6, wherein, ​ 9. The head-mounted display device of any one of claims 1-6, wherein, ​ ​ 10. The head-mounted display device of any one of claims 1-6, wherein, ​

Images