Space image processing method, intelligent head-mounted device, image processing device and medium

Abstract

The invention discloses a space image processing method, intelligent head-mounted equipment, image processing equipment and a medium, and relates to the technical field of intelligent head-mounted equipment, the method is applied to the intelligent head-mounted equipment, the method comprises an infrared positioning module and an inertia measurement module, and the method comprises the following steps: outputting infrared light through the infrared positioning module, the image processing device receives the infrared light through the infrared camera, and positions the intelligent head-mounted device according to the infrared light to obtain position information; acquiring inertial data of the intelligent head-mounted equipment through an inertial measurement module; and outputting the inertial data to an image processing device, so that the image processing device renders a preset display scene according to the position information and the inertial data to obtain a rendered picture, and displaying the rendered picture through a display module. According to the invention, algorithms such as image processing are set in the external image processing equipment, so that the size and weight of the intelligent head-mounted equipment are reduced while the space experience is ensured.

Description

Technical Field

[0001] This application relates to the field of smart head-mounted device technology, and in particular to a spatial image processing method, a smart head-mounted device, an image processing device, and a medium. Background Technology

[0002] With the development of technology, smart head-mounted devices, with their unique immersive spatial experience, bring users a unique interactive experience. By integrating sophisticated sensors, high-performance processors, high-definition displays, and advanced display technologies, smart head-mounted devices enable users to participate in virtual environments as if they were actually there, and are widely used in entertainment, education, healthcare, industrial design, and many other fields.

[0003] Traditional smart head-mounted devices with spatial experience typically require high-performance processors, large-capacity memory, high-speed storage media, and multiple high-precision sensors to achieve high-precision positioning and tracking, complex graphics rendering, and smooth interactive experience. In addition, to ensure a high-quality visual experience, the devices also need to be equipped with displays and speakers or bone conduction headphones for sound output.

[0004] Due to the complexity and high requirements of the aforementioned hardware, traditional smart head-mounted devices generally suffer from excessive weight. Furthermore, the stacking of numerous hardware components increases the weight and size of the device, making prolonged wear uncomfortable and impacting the user experience. Simultaneously, high-performance hardware consumes significant power, shortening battery life and limiting continuous use. Therefore, how to reduce the size of smart head-mounted devices while maintaining a comfortable user experience has become a pressing technical challenge in the current smart head-mounted device industry.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this application is to provide a spatial image processing method, a smart head-mounted device, an image processing device, and a medium, aiming to solve the technical problem of reducing the size of the device while ensuring a good spatial experience.

[0007] To achieve the above objectives, this application proposes a spatial image processing method applied to a smart head-mounted device, the smart head-mounted device comprising: an infrared positioning module and an inertial measurement module, the method comprising:

[0008] The infrared positioning module outputs infrared light, which is received by an image processing device via an infrared camera. The device then uses the infrared light to locate the smart head-mounted device and obtain its position information.

[0009] The inertial measurement module acquires the inertial data of the smart head-mounted device;

[0010] The inertial data is output to the image processing device, so that the image processing device can render a preset display scene based on the position information and the inertial data to obtain a rendered image, and display the rendered image through a preset display module.

[0011] In one embodiment, the smart head-mounted device further includes a display module, and after the step of outputting the inertial data to the image processing device, the method further includes:

[0012] Receive the rendered image output by the image processing device;

[0013] The rendered image is displayed through the display module.

[0014] In one embodiment, the smart head-mounted device further includes: a prompting module, wherein after the step of outputting the inertial data to the image processing device, the method further includes:

[0015] Receive the prompt information output by the image processing device;

[0016] The prompting module outputs the prompting information.

[0017] In one embodiment, prior to the step of outputting infrared light through the infrared positioning module, the method further includes:

[0018] Obtain the shooting frequency of the infrared camera of the image processing device;

[0019] The flashing frequency of the infrared positioning module is set according to the shooting frequency.

[0020] Furthermore, to achieve the above objectives, this application also proposes a spatial image processing method, which is applied to an image processing device, the image processing device including an infrared camera, and the method comprising:

[0021] The infrared camera receives infrared light output from the smart head-mounted device.

[0022] The location information is obtained by locating the smart head-mounted device based on the infrared light.

[0023] Receive inertial data output by the smart head-mounted device;

[0024] The preset display scene is rendered based on the location information and the inertial data to obtain a rendered image, and the rendered image is output to the preset display module for display.

[0025] In one embodiment, the step of rendering a preset display scene based on the location information and the inertial data to obtain a rendered image includes:

[0026] The location information is applied to the virtual camera in the display scene;

[0027] The shooting angle of the virtual camera is set according to the inertial data;

[0028] The captured image from the virtual camera is obtained, and the captured image is the rendered image.

[0029] In one embodiment, after the step of receiving the inertial data output by the smart head-mounted device, the method further includes:

[0030] The inertial data is preprocessed to obtain intermediate data;

[0031] Motion features are obtained by extracting features from the intermediate data;

[0032] When the motion characteristics match preset abnormal characteristics, a prompt message corresponding to the abnormal characteristics is output to the smart head-mounted device, so that the smart head-mounted device can output the prompt message through the prompt module.

[0033] In addition, to achieve the above objectives, this application also proposes an image processing apparatus, the apparatus comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the spatial image processing method as described above.

[0034] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the spatial image processing method described above.

[0035] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the spatial image processing method described above.

[0036] This application provides a spatial image processing method. This application establishes a communication connection between a head-mounted device and an external image processing device. The head-mounted device is equipped with only an infrared positioning module and an inertial measurement module. The infrared positioning module outputs infrared light, and the inertial measurement module measures inertial data and outputs it to the image processing device. The image processing device locates the smart head-mounted device based on the infrared light, renders the scene based on the positioning information and inertial data to obtain a rendered image, and then displays it through a preset display module. This method can achieve spatial image display while reducing the number of hardware modules in the smart head-mounted device.

[0037] In summary, this application sets the spatial image rendering and processing part in an external image processing device. In the smart head-mounted device, only an infrared positioning module and an inertial measurement module are needed to view the spatial image. Compared with existing smart head-mounted devices for spatial image experience, this application reduces the number of hardware components in the smart head-mounted device by setting the image processing and other algorithms in an external image processing device, thereby ensuring the spatial experience while reducing the size and weight of the smart head-mounted device. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating an embodiment of the spatial image processing method of this application.

[0041] Figure 2 This is a schematic diagram of the structure of a smart head-mounted device according to an embodiment of this application;

[0042] Figure 3 This is a flowchart illustrating Embodiment 2 of the spatial image processing method of this application;

[0043] Figure 4 This is a schematic diagram of the structure of a smart tablet according to an embodiment of this application;

[0044] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the spatial image processing method in the embodiments of this application.

[0045] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0046] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0047] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0048] The main solution of this application embodiment is as follows: infrared light is output through the infrared positioning module, so that the image processing device receives the infrared light through the infrared camera and locates the smart head-mounted device according to the infrared light to obtain position information; the inertial measurement module obtains the inertial data of the smart head-mounted device; the inertial data is output to the image processing device, so that the image processing device renders a preset display scene according to the position information and the inertial data to obtain a rendered image, and displays the rendered image through a preset display module.

[0049] With the development of technology, smart head-mounted devices, with their unique immersive spatial experience, bring users a unique interactive experience. By integrating sophisticated sensors, high-performance processors, high-definition displays, and advanced display technologies, smart head-mounted devices enable users to participate in virtual environments as if they were actually there, and are widely used in entertainment, education, healthcare, industrial design, and many other fields.

[0050] Traditional smart head-mounted devices with spatial experience typically require high-performance processors, large-capacity memory, high-speed storage media, and multiple high-precision sensors to achieve high-precision positioning and tracking, complex graphics rendering, and smooth interactive experience. In addition, to ensure a high-quality visual experience, the devices also need to be equipped with displays and speakers or bone conduction headphones for sound output.

[0051] Due to the complexity and high requirements of the aforementioned hardware, traditional smart head-mounted devices generally suffer from excessive weight. Furthermore, the stacking of numerous hardware components increases the weight and size of the device, making prolonged wear uncomfortable and impacting the user experience. Simultaneously, high-performance hardware consumes significant power, shortening battery life and limiting continuous use. Therefore, how to reduce the size of smart head-mounted devices while maintaining a comfortable user experience has become a pressing technical challenge in the current smart head-mounted device industry.

[0052] To address the aforementioned issues, this application provides a spatial image processing method. This method establishes a communication connection between a head-mounted device and an external image processing device. The head-mounted device is equipped with only an infrared positioning module and an inertial measurement module. The infrared positioning module outputs infrared light, and the inertial measurement module measures inertial data, which is then output to the image processing device. The image processing device positions the smart head-mounted device based on the infrared light, renders the scene based on the positioning information and inertial data to obtain a rendered image, and then displays it through a pre-set display module. This method reduces the number of hardware modules in the smart head-mounted device while simultaneously enabling the display of spatial images.

[0053] In summary, this application sets the spatial image rendering and processing part in an external image processing device. In the smart head-mounted device, only an infrared positioning module and an inertial measurement module are needed to view the spatial image. Compared with existing smart head-mounted devices for spatial image experience, this application reduces the number of hardware components in the smart head-mounted device by setting the image processing and other algorithms in an external image processing device, thereby ensuring the spatial experience while reducing the size and weight of the smart head-mounted device.

[0054] In this embodiment, for ease of description, the following description uses a smart head-mounted device as the execution subject.

[0055] Based on this, embodiments of this application provide a spatial image processing method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the spatial image processing method of this application.

[0056] In this embodiment, the spatial image processing method includes steps S10 to S30:

[0057] Step S10: The infrared positioning module outputs infrared light so that the image processing device can receive the infrared light through the infrared camera and locate the smart head-mounted device based on the infrared light to obtain location information.

[0058] It should be noted that in this embodiment, the method is applied to a smart head-mounted device. To improve the comfort and reduce the weight of the smart head-mounted device, it can be designed in the shape of glasses. Please refer to [reference needed]. Figure 2 , Figure 2 This is a schematic diagram of the structure of a smart head-mounted device according to an embodiment of this application, such as... Figure 2 As shown, the overall structure of the smart head-mounted device is similar to that of glasses. A ring of infrared lights (i.e., infrared positioning modules) is set around the frame, and an inertial measurement module is set in the middle of the frame.

[0059] Alternatively, the smart head-mounted device can be designed to resemble a clip-on eyeglasses, allowing it to be attached to ordinary eyeglasses. The clip-on eyeglasses can also have infrared positioning and inertial measurement modules placed in easily accessible locations, further reducing the size of the smart head-mounted device.

[0060] The above two are merely two feasible structures for the smart head-mounted device of this application. Any other head-mounted device that can install an infrared positioning module and an inertial measurement module can achieve the technical effects of this embodiment. Therefore, no specific limitation is made to the structure of the smart head-mounted device.

[0061] It should be noted that, in this embodiment, the infrared positioning module is a key component of the smart head-mounted device, responsible for emitting infrared light. This infrared light has a specific frequency and intensity so that the image processing device can accurately capture it.

[0062] In this embodiment, the infrared positioning module outputs infrared light at a specific frequency and intensity. The infrared camera in the image processing device can receive the infrared light from the smart head-mounted device. The image processing device uses the received infrared light information, combined with parameters such as the camera's angle of view and focal length, to calculate the position of the smart head-mounted device in three-dimensional space through a specific positioning algorithm. This position information can be represented in the form of X, Y, and Z coordinates.

[0063] Furthermore, in one feasible implementation, before step S10 described above, the method may further include steps S01 to S02:

[0064] Step S01: Obtain the shooting frequency of the infrared camera of the image processing device;

[0065] Step S02: Set the flashing frequency of the infrared positioning module according to the shooting frequency.

[0066] In this embodiment, the camera on the image processing device, used to receive infrared light, has a shooting frequency (i.e., the number of frames captured per second) that determines the accuracy and smoothness of the dynamic information it can capture. During the startup or initialization phase of the image processing device, the system obtains the actual shooting frequency of the infrared camera through internal sensors or by querying the device specification table. This frequency is usually fixed, but may vary depending on the device model or user settings. The module on the smart head-mounted device, used to emit infrared light, has a flashing frequency that determines the probability and accuracy of the infrared light being captured by the camera. After obtaining the shooting frequency of the infrared camera, the system sets the flashing frequency of the infrared positioning module based on this frequency. Typically, to ensure that the camera can capture a sufficient number of infrared light points, the flashing frequency is set to an integer multiple or close to the shooting frequency. To achieve precise frequency synchronization, clock synchronization technology or signal processing technology can be used to ensure time synchronization between the infrared positioning module and the infrared camera.

[0067] The above steps improve the collaborative efficiency between the infrared positioning module and the infrared camera, reducing positioning errors caused by frequency mismatch. Simultaneously, they minimize the flickering of the infrared lights when the camera is not capturing images, thus lowering the power consumption of the infrared positioning module.

[0068] Step S20: Obtain the inertial data of the smart head-mounted device through the inertial measurement module;

[0069] It should be noted that, in this embodiment, the inertial measurement module is another key component of the smart head-mounted device, which typically includes sensors such as accelerometers and gyroscopes. These sensors can measure the device's acceleration, angular velocity, and other motion state information in real time.

[0070] In this embodiment, when the user is wearing the smart head-mounted device, the inertial measurement module acquires the inertial data of the smart head-mounted device in real time.

[0071] Step S30: The inertial data is output to the image processing device so that the image processing device can render the preset display scene according to the position information and the inertial data to obtain a rendered image, and display the rendered image through a preset display module.

[0072] It should be noted that in this embodiment, the smart head-mounted device communicates with the image processing device, and the connection method can be wireless or through a USB Type-C to DP cable or other cable specifications.

[0073] In this embodiment, after acquiring inertial data, the image processing device receives the inertial data from the smart head-mounted device through a communication connection and fuses it with the previously acquired position information. Based on the fused data, the image processing device uses a rendering engine to render a preset display scene. The rendering process may include multiple stages such as lighting calculation, shadow calculation, and texture mapping, ultimately generating a two-dimensional image (rendered screen) containing the position and dynamic information of the smart head-mounted device. The rendered screen is displayed through a preset display module (such as a screen, projector, etc.) for user viewing and interaction.

[0074] Furthermore, in one feasible implementation, the smart head-mounted device further includes a display module, and after step S30 described above, the method may further include steps S31 to S32:

[0075] Step S31: Receive the rendered image output by the image processing device;

[0076] It should be noted that, in this embodiment, the display module can be a display module set in a smart head-mounted device, which realizes a three-dimensional visual effect through the two screens of the eyes.

[0077] In practical applications, humans typically observe objects with both eyes simultaneously. Due to the distance between the visual axes of the two eyes (approximately 65 millimeters), the left and right eyes receive different visual images when viewing objects at a certain distance; this difference is called binocular parallax. The brain combines the information from the two images through eye movements and adjustments, creating a sense of depth. To allow people to see 3D images, the left and right eyes need to see different images, creating a gap between them to simulate the actual viewing experience and thus create a 3D effect. In the two displays of smart glasses, there is a slight horizontal parallax between the images displayed on the screens; that is, the image position of the same object is slightly different on the left and right eye screens. This difference is key to simulating the different images received by the two eyes when viewing an object. When an observer views these screens with both eyes, their brain automatically fuses and analyzes the images on the two screens. Due to the slight parallax between the images, the brain can perceive the depth information of the object, thus creating a sense of depth. When rendering for binocular parallax, images can be captured using a dual-lens virtual camera and then output to the corresponding screens.

[0078] In addition, when displaying rendered images through a screen or projector, the content seen by the user will vary depending on the user's angle when viewing the same three-dimensional object. Therefore, even if the displayed image is two-dimensional, the user can experience a three-dimensional spatial effect by changing the displayed image when the user changes their perspective.

[0079] In this embodiment, the image processing device generates a rendered image based on location information and inertial data. This rendered image is then transmitted to the smart head-mounted device via a communication interface.

[0080] Step S32: Display the rendered screen through the display module.

[0081] In this embodiment, the display module built into the smart head-mounted device, such as a micro OLED screen, LCD screen, or projection display system, is responsible for presenting the received rendered image to the user. After receiving the rendered image, the smart head-mounted device sends it to the display module for display. The user can intuitively see the rendered image by wearing the smart head-mounted device.

[0082] By following the steps described above, the dual display modules of the smart head-mounted device enable users to experience a three-dimensional visual experience, enhancing visual immersion and thus improving the user experience.

[0083] Furthermore, in one feasible implementation, the smart head-mounted device further includes a prompting module, and after step S30 above, the method may further include steps S301 to S302:

[0084] Step S301: Receive the prompt information output by the image processing device;

[0085] It should be noted that, in this embodiment, the prompting module built into the smart head-mounted device can be an audio prompt (such as a speaker or headphone jack), a visual prompt (such as an LED light or a small display screen), or a tactile prompt (such as a vibration motor).

[0086] In this embodiment, the image processing device generates a prompt message while generating the rendered image, or based on certain specific conditions (such as user operation, system state changes, etc.). This information is transmitted to the smart head-mounted device through a communication interface.

[0087] For example, when a user is using a smart head-mounted device for learning, if they become fatigued or their eyes are too close to the screen, a prompt message can be sent to remind the user to rest or change their posture.

[0088] Step S302: Output the prompt information through the prompt module.

[0089] In this embodiment, upon receiving a prompt message, the processor or control unit of the smart head-mounted device sends it to the corresponding prompt module for output. For example, if the prompt message is an audio message, it is played through a speaker; if it is a visual message, it is displayed through flashing LEDs or a small display screen.

[0090] Through the steps described above, the smart head-mounted device can receive and output prompts generated by the image processing device. This technological expansion not only enriches the interaction methods of the smart head-mounted device but also improves the user's perception and responsiveness during use. It also demonstrates the potential of smart head-mounted devices in providing multi-dimensional information output and enhancing user experience.

[0091] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. In this embodiment, for ease of description, the following description uses an image processing device as the execution subject. Based on this, please refer to Figure 3 The image processing device includes an infrared camera, and the method includes steps A10 to A40:

[0092] Step A10: Receive infrared light output from the smart head-mounted device via the infrared camera;

[0093] It should be noted that in this embodiment, the image processing device can be various types of devices, such as a smart tablet. Please refer to [reference needed]. Figure 4 , Figure 4 This is a schematic diagram of the structure of a smart tablet according to an embodiment of this application, such as... Figure 4 As shown, a binocular infrared camera is installed above the smart tablet to locate the smart head-mounted device. The image processing device can also be a smart projector, etc.

[0094] The above are merely feasible structures for the image processing device of this application, and no specific limitations are made here on the structure of the image processing device.

[0095] In this embodiment, the image processing device receives infrared light signals emitted by the smart head-mounted device via its equipped infrared camera. These infrared light signals are generated by the infrared positioning module of the smart head-mounted device flashing at a preset frequency, and are used to assist the image processing device in positioning.

[0096] Step A20: Use the infrared light to locate the smart head-mounted device and obtain location information;

[0097] In this embodiment, the image processing device uses an image processing algorithm to locate the smart head-mounted device based on the received infrared light signal and calculate its precise location information.

[0098] Step A30: Receive inertial data output by the smart head-mounted device;

[0099] In this embodiment, the image processing device receives inertial data from the smart head-mounted device, including acceleration, angular velocity, etc., to help improve positioning accuracy and stability. The smart head-mounted device is typically equipped with an inertial measurement unit (IMU) capable of measuring and outputting inertial data in real time. The image processing device receives this data via wireless communication (such as Bluetooth, Wi-Fi, etc.) or wired communication.

[0100] Step A40: Render the preset display scene according to the location information and the inertial data to obtain a rendered image, and output the rendered image to the preset display module for display.

[0101] In this embodiment, the image processing device renders a preset display scene based on the acquired position information and inertial data, generating a realistic virtual image, which is then displayed through a preset display module. The rendering process involves graphics algorithms such as 3D modeling, lighting calculation, and texture mapping. The display module can be a built-in display screen or projector of the image processing device, or it can be an external display device or projection device connected to it. Through rendering, the user can see a virtual scene corresponding to the position of the smart head-mounted device on the display module.

[0102] As an example, specifically, an inertial measurement module can employ an inertial measurement unit (IMU), which typically consists of three single-axis accelerometers and three single-axis gyroscopes; some IMUs also include a magnetometer. Based on Newton's second law, the motion state of an object is reflected by measuring the acceleration signals of the object along three independent axes of the carrier coordinate system. When the object's acceleration changes, the accelerometers generate corresponding output signals. Based on the gyroscopic effect, the angular velocity signal of the object relative to the navigation coordinate system is measured. The angular velocity of the object is calculated by measuring the induced torque, thereby determining the object's rotational state. Gyroscopes can be divided into two types: optical gyroscopes and MEMS (Micro-Electro-Mechanical Systems) gyroscopes. MEMS gyroscopes are widely used due to their miniaturization, low power consumption, and cost-effectiveness. When a user wears glasses with an IMU, the IMU begins to operate, acquiring the user's acceleration and angular velocity data in real time. This data is transmitted in digital signal form to external image processing equipment for further processing. Image processing equipment performs filtering, noise reduction, and calibration on the acquired raw data to improve its accuracy and precision. Using gyroscope data, it calculates the attitude of the inertial navigation system (INS), i.e., derives the quaternion or rotation matrix representing the attitude. Based on the INS attitude, the specific force components of the carrier coordinate system measured by the accelerometer are transformed to the navigation coordinate system. In the navigation coordinate system, the specific force equation is solved by integration to remove the influence of Earth's gravitational acceleration, thus obtaining the carrier's velocity relative to the Earth. Further, in the position reference coordinate system, the position information is obtained by integrating the carrier velocity. By fusing data from the accelerometer and gyroscope, and possibly magnetometer data (if the IMU includes a magnetometer), algorithms such as complementary filtering, AHRS (Attitude and Heading Reference System), or Kalman filtering are employed to obtain more accurate and stable attitude information. After attitude calculation, the user's attitude information, including position, velocity, and attitude angles, is obtained. Based on the calculated user attitude, subsequent image rendering can be better performed, resulting in more accurate image display.

[0103] Furthermore, in a feasible implementation, the step A40 above, which involves rendering a preset display scene based on the location information and the inertial data to obtain a rendered image, may include steps A41 to A43:

[0104] Step A41: Apply the location information to the virtual camera in the display scene;

[0105] Step A42: Set the shooting angle of the virtual camera according to the inertial data;

[0106] Step A43: Obtain the captured image from the virtual camera, which is the rendered image.

[0107] In this embodiment, after acquiring the position information and inertial data of the smart head-mounted device, the actual position information of the smart head-mounted device is mapped into the virtual space of the display scene as the position of the virtual camera. Through coordinate transformation, the position of the smart head-mounted device in the three-dimensional physical space is converted into coordinates in the virtual space. These coordinates will serve as the position of the virtual camera for subsequent rendering calculations. Then, using the inertial data output by the smart head-mounted device (such as acceleration, angular velocity, etc.), the shooting angle of the virtual camera is calculated and set. By integrating the inertial data (such as integrating the angular velocity to obtain the rotation angle) and combining it with the initial state, the current orientation of the virtual camera can be calculated. This orientation will serve as the shooting angle of the virtual camera to determine the viewpoint of the rendered image. After determining the position and shooting angle of the virtual camera, the rendering engine calculates and acquires the image captured by the virtual camera. The rendering engine performs real-time three-dimensional rendering calculations based on the position and shooting angle of the virtual camera, as well as information such as objects, lighting, and textures in the display scene, to generate a realistic virtual image.

[0108] In addition, more factors can be considered during the rendering process, such as lighting, shadows, texture mapping, and anti-aliasing, to further improve the quality and realism of the rendered image. Furthermore, various optimization techniques can be used to optimize rendering performance, such as Level of Detail (LOD) technology and Frustum Culling.

[0109] Furthermore, in one feasible implementation, after step A30 described above, the method may further include steps A31 to A33:

[0110] Step A31: Preprocess the inertial data to obtain intermediate data;

[0111] Step A32: Extract the features from the intermediate data to obtain motion features;

[0112] Step A33: When the motion feature matches the preset abnormal feature, output the prompt information corresponding to the abnormal feature to the smart head-mounted device, so that the smart head-mounted device can output the prompt information through the prompt module.

[0113] In this embodiment, after receiving inertial data, the received inertial data is preprocessed to eliminate noise, fill in missing values, etc., to obtain more accurate and reliable intermediate data. Preprocessing may include filtering (such as Kalman filtering, low-pass filtering, etc.) to smooth the data, and using interpolation algorithms to fill in missing data points. Then, features that can characterize the motion state of the smart head-mounted device, i.e., motion features, are extracted from the intermediate data. Feature extraction may include statistical methods (such as mean, variance, standard deviation, etc.), time-domain analysis (such as rate of change of acceleration, rate of change of angular velocity, etc.), or frequency-domain analysis (such as Fourier transform), etc. The extracted motion features are compared with preset abnormal features. When a match is found, it is considered that the smart head-mounted device may be in an abnormal state, and corresponding prompt information is output to it. Anomaly detection may include threshold judgment (such as acceleration exceeding a certain threshold), pattern recognition (such as using machine learning algorithms to identify abnormal patterns), or statistical analysis (such as outlier detection algorithms), etc. The image processing device sends the corresponding prompt information for the abnormal features to the smart head-mounted device. After receiving the prompt information, the smart head-mounted device outputs the prompt information to the user in a visual, tactile, or auditory manner through its built-in prompt module (such as LED lights, vibrators, speakers, etc.).

[0114] As an example, during learning using a smart head-mounted device, the user wears the device, and the image processing device renders the image based on the device's positioning and inertial data. The rendered image is then displayed on a screen, projector, or the head-mounted device's display module. Simultaneously, the head-mounted device's built-in inertial sensors continuously monitor the user's head movement data, including acceleration and angular velocity. After receiving this inertial data, the image processing device preprocesses and extracts features, identifying abnormal head movement patterns, such as prolonged stillness or slight, irregular swaying. These features may be related to a daydreaming state. When the image processing device identifies a movement pattern matching preset daydreaming characteristics, it determines that the user may be daydreaming. The image processing device then sends a daydreaming alert to the head-mounted device via a wireless communication module. Upon receiving the alert, the head-mounted device uses its built-in notification module (such as a slight vibration, a brief beep, or flashing LED lights) to remind the user of their daydreaming state. The head-mounted device can also record the number and duration of daydreams during the learning process, serving as a reference for evaluating learning progress and efficiency. Based on the state of daydreaming, the image processing device can generate personalized learning suggestions, such as adjusting study time, increasing rest intervals, or conducting attention training, to help users improve learning efficiency.

[0115] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the spatial image processing method of this application. Any simple transformations based on this technical concept are all within the protection scope of this application.

[0116] This application provides a smart head-mounted device / image processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the spatial image processing method in Embodiment 1 above.

[0117] The following is for reference. Figure 5 It shows a structural schematic diagram of a smart head-mounted device / image processing device suitable for implementing embodiments of this application. Figure 5 The smart head-mounted device / image processing device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0118] like Figure 5 As shown, the intelligent head-mounted device / image processing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the intelligent head-mounted device / image processing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the smart head-mounted device / image processing device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show smart head-mounted devices / image processing devices with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0119] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0120] The intelligent head-mounted device / image processing device provided in this application, employing the spatial image processing method described in the above embodiments, can solve the technical problem of reducing the device's size while ensuring a good spatial experience. Compared with the prior art, the beneficial effects of the intelligent head-mounted device / image processing device provided in this application are the same as those of the spatial image processing method described in the above embodiments, and other technical features of this intelligent head-mounted device / image processing device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0121] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0122] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0123] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the spatial image processing method described in the above embodiments.

[0124] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0125] The aforementioned computer-readable storage medium may be included in a smart head-mounted device / image processing device; or it may exist independently and not assembled into a smart head-mounted device / image processing device.

[0126] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the smart head-mounted device, the smart head-mounted device causes the following: the infrared positioning module outputs infrared light, which is received by an image processing device via an infrared camera, and the image processing device locates the smart head-mounted device based on the infrared light to obtain position information; the inertial measurement module acquires inertial data of the smart head-mounted device; the inertial data is output to the image processing device, which renders a preset display scene based on the position information and the inertial data to obtain a rendered image, and the rendered image is displayed through a preset display module.

[0127] When one or more of the above programs are executed by the image processing device, the image processing device: receives infrared light output from the smart head-mounted device through the infrared camera; locates the smart head-mounted device based on the infrared light to obtain position information; receives inertial data output by the smart head-mounted device; renders a preset display scene based on the position information and the inertial data to obtain a rendered image, and outputs the rendered image to a preset display module for display.

[0128] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0130] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0131] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described spatial image processing method. This solves the technical problem of reducing the size of a smart head-mounted device while ensuring a good spatial experience. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the spatial image processing method provided in the above embodiments, and will not be repeated here.

[0132] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the spatial image processing method described above.

[0133] The computer program product provided in this application solves the technical problem of reducing the size of smart head-mounted devices while ensuring a good spatial experience. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the spatial image processing method provided in the above embodiments, and will not be repeated here.

[0134] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A spatial image processing method, characterized in that, The method is applied to a smart head-mounted device, the smart head-mounted device including: an infrared positioning module and an inertial measurement module, the method including: The infrared positioning module outputs infrared light, which is received by an image processing device via an infrared camera. The device then uses the infrared light to locate the smart head-mounted device and obtain its position information. The inertial measurement module acquires the inertial data of the smart head-mounted device; The inertial data is output to the image processing device, so that the image processing device can render a preset display scene based on the position information and the inertial data to obtain a rendered image, and display the rendered image through a preset display module.

2. The spatial image processing method as described in claim 1, characterized in that, The smart head-mounted device further includes a display module, and after the step of outputting the inertial data to the image processing device, the method further includes: Receive the rendered image output by the image processing device; The rendered image is displayed through the display module.

3. The spatial image processing method as described in claim 1, characterized in that, The smart head-mounted device further includes a prompting module, and after the step of outputting the inertial data to the image processing device, the method further includes: Receive the prompt information output by the image processing device; The prompting module outputs the prompting information.

4. The spatial image processing method as described in claim 1, characterized in that, Before the step of outputting infrared light through the infrared positioning module, the method further includes: Obtain the shooting frequency of the infrared camera of the image processing device; The flashing frequency of the infrared positioning module is set according to the shooting frequency.

5. A spatial image processing method, characterized in that, The method is applied to an image processing device, the image processing device including an infrared camera, and the method includes: The infrared camera receives infrared light output from the smart head-mounted device. The location information is obtained by locating the smart head-mounted device based on the infrared light. Receive inertial data output by the smart head-mounted device; The preset display scene is rendered based on the location information and the inertial data to obtain a rendered image, and the rendered image is output to the preset display module for display.

6. The spatial image processing method as described in claim 5, characterized in that, The step of rendering a preset display scene based on the location information and the inertial data to obtain a rendered image includes: The location information is applied to the virtual camera in the display scene; The shooting angle of the virtual camera is set according to the inertial data; The captured image from the virtual camera is obtained, and the captured image is the rendered image.

7. The spatial image processing method as described in claim 5, characterized in that, After the step of receiving the inertial data output by the smart head-mounted device, the method further includes: The inertial data is preprocessed to obtain intermediate data; Motion features are obtained by extracting features from the intermediate data; When the motion characteristics match the preset abnormal characteristics, the system outputs the corresponding prompt information to the smart head-mounted device, so that the smart head-mounted device can output the prompt information through the prompt module.

8. A smart head-mounted device, characterized in that, The smart head-mounted device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the spatial image processing method as described in any one of claims 1 to 4.

9. An image processing device, characterized in that, The image processing device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the spatial image processing method as described in any one of claims 5 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the spatial image processing method as described in any one of claims 1 to 4 or 5 to 7.

Images