Imaging detection method and system of head-mounted display equipment

Abstract

The invention relates to the related field of display equipment, discloses an imaging detection method and system of head-mounted display equipment, is suitable for mixed reality head-mounted display equipment, and realizes real-time virtual synchronization of environment data and self motion under the cooperative action of an image sensor, an attitude sensor and the like. In this way, the purpose of enhancing and displaying visual interaction experience through mixed reality is achieved, and more immersive virtual overlapping experience is achieved compared with the prior art by means of mapping correction of imaging and correction of visual centering through the view angle shaking angle. The image content can be output more accurately so as to achieve positioning coverage of a real scene, and the optimal output state effect is achieved in cooperation with visual angle positioning.

Description

Technical Field

[0001] The present invention relates to the field related to display devices, and in particular to an imaging detection method and system for a head-mounted display device. Background Art

[0002] Mixed reality (MR) is a hybrid technology solution based on virtual reality and augmented reality. In mixed reality, users can not only see the real world, but also see virtual content generated by computers. By combining the two, a more immersive real-world interaction effect can be achieved.

[0003] In the prior art, when a mixed reality wearable device is used, it collects environmental data through sensor devices and cameras, identifies and judges environmental objects and related content, and then generates content to be output and outputs it through a display projection device. In this process, posture sensors can perform motion calculations through relevant readings such as inertial posture, thereby achieving synchronization with the movement of real users. However, the superposition of virtual and reality in the prior art mostly adopts the camera ranging method, and due to the differences in wearing habits and visual habits of different users, the fixed-mode scaling superposition method has large errors. Summary of the invention

[0004] The object of the present invention is to provide an imaging detection method and system for a head mounted display device to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: An imaging detection method for a head mounted display device, comprising: Acquire environmental spatial data within the scanning field of view in real time through a sensor device group, and perform high-precision spatial modeling based on the environmental spatial data to obtain an environmental twin model, wherein the environmental spatial data includes environmental image data and laser ranging data; Based on the posture sensing device group, the current motion posture data of the display device is obtained and updated synchronously in real time, and the motion integral calculation is performed through the motion posture data to obtain the spatial motion stroke within the time period, wherein the motion posture data includes posture orientation data and motion inertia data; Acquire the user's viewing angle, and map the display image based on the viewing angle to synchronize the user's viewing angle with the display viewing angle of the display device, wherein the viewing angle is used to represent the user's visual center deflection angle between two time nodes; The user's pupil is tracked, and the user's visual center is obtained through an initial request. The spatial perspective difference between the visual center and the current posture orientation data is calculated, and the display output field of view is calibrated based on the spatial perspective difference. The spatial perspective difference is used to characterize the spatial orientation angle between the user's visual center and the display output center of the display device. The visual center is used to characterize the user's head-on viewing angle.

[0006] As a further solution of the present invention, the step of obtaining the user's viewing angle and mapping the display image based on the viewing angle to synchronize the user's viewing angle with the display viewing angle of the display device specifically includes: When it is detected that the wearing state of the display device changes, a viewing angle range synchronization request is generated, and after determining to obtain the visual center of the user, a projection display distance between the user's pupil and the display device is detected; In response to the range synchronization request, a pair of synchronization reference nodes are randomly selected based on the environment twin model, and the pair of synchronization reference nodes are displayed and marked in the environment twin model; When the visual center of the display device coincides with the synchronization reference node, the current posture orientation data is recorded, and the posture inclination difference of the posture orientation data between a pair of synchronization reference nodes is calculated, that is, the viewing angle shaking angle of the corresponding display device; The visual width under the corresponding viewing angle shaking angle is calculated based on the projection display distance, and the display scaling ratio is calculated through the actual display distance of a pair of synchronous reference nodes, and the output picture of the display device is synchronously scaled based on the display scaling ratio.

[0007] As a further solution of the present invention: it also includes a user characterization correction step, specifically including: When tracking the user's pupil, generate a view initialization request to guide the user's view center to coincide with the display center of the display device. If the user's pupil angle does not coincide with the display center of the display device, guide the user to perform feature correction. Generate multiple sets of correction guide points distributed in multiple plane quadrants of the display range, and record the pupil angles of the user at the multiple correction guide points when the user feedback visual centers coincide; Based on the relative position relationship between the multiple correction guide points and the corresponding position relationship between the multiple pupil angles, a fitting calculation is performed to obtain the user's pupil movement mapping model, which is used to characterize the actual visual center change corresponding to when the user's pupil rotates a certain angle.

[0008] As a further solution of the present invention: it also includes an imaging resource optimization step: Acquire the user's visual center in real time through pupil tracking, and set the visual center as the first imaging center; A plurality of annular display areas are established based on the first imaging center as the center of the circle. The plurality of annular display areas correspond to a plurality of display output schemes respectively. The display output resolution and content refresh frequency of the plurality of annular display areas along the radial direction gradually decrease.

[0009] As a further solution of the present invention: the imaging resource optimization step further includes: Acquire the environmental spatial data within the scanning field of view in real time, and update the difference data based on the environmental twin model; Establishing a second imaging center based on the object area updated by the difference data, wherein the second imaging center is different from the first imaging center, and the second imaging center does not include a plurality of accompanying annular display areas; A display output scheme consistent with that of the first imaging center is adopted for the corresponding display data content of the second imaging center.

[0010] An embodiment of the present invention is to provide an imaging detection system for a head mounted display device, comprising: An environmental synchronization module is used to obtain environmental spatial data within the scanning field of view in real time through a sensor device group, and to perform high-precision spatial modeling based on the environmental spatial data to obtain an environmental twin model. The environmental spatial data includes environmental image data and laser ranging data. A motion synchronization module is used to obtain the current motion posture data of the display device based on the posture sensing device group and perform real-time synchronous update, and perform motion integral calculation based on the motion posture data to obtain the spatial motion stroke within a time period, wherein the motion posture data includes posture orientation data and motion inertia data; A perspective synchronization module, used to obtain a user's perspective shaking angle, and to map a display image based on the perspective shaking angle to synchronize the user's perspective with the display perspective of the display device, wherein the perspective shaking angle is used to represent a user's visual center deflection angle between two time nodes; The field of view calibration module is used to track the user's pupil, obtain the user's visual center through an initial request, calculate the spatial perspective difference between the visual center and the current posture orientation data, and calibrate the display output field of view based on the spatial perspective difference. The spatial perspective difference is used to characterize the spatial orientation angle between the user's visual center and the display output center of the display device. The visual center is used to characterize the user's head-on viewing angle.

[0011] As a further solution of the present invention: the viewing angle synchronization module specifically includes: A viewing distance detection unit, configured to generate a viewing angle range synchronization request when detecting that a wearing state of the display device has changed, and after determining to obtain the user's visual center, detect and obtain the projection display distance between the user's pupil and the display device; A reference marking unit, configured to respond to the range synchronization request, randomly select a pair of synchronization reference nodes based on the environment twin model, and display mark the pair of synchronization reference nodes in the environment twin model; A rotation angle acquisition unit, used for recording the current posture orientation data when the visual center of the display device coincides with the synchronization reference node, and calculating the posture inclination difference of the posture orientation data between a pair of synchronization reference nodes, that is, the viewing angle shaking angle of the corresponding display device; The field of view synchronization unit is used to calculate the visual width under the corresponding viewing angle shaking angle based on the projection display distance, and calculate the display scaling ratio through the actual display distance of a pair of synchronization reference nodes, and perform perspective synchronization scaling on the output picture of the display device based on the display scaling ratio.

[0012] As a further solution of the present invention: it also includes a characterization correction module, specifically including: A correction initialization unit, used to generate a view angle initialization request when tracking the user's pupil, so as to guide the user's view angle center to coincide with the display center of the display device, and to guide the user to perform a characteristic correction if the user's pupil angle does not coincide with the display center of the display device at this time; A correction marking unit, used to generate multiple groups of correction guide points distributed in multiple plane quadrants of the display range, and record the pupil angles of the user at the multiple correction guide points when the user feedbacks that the visual centers coincide; The correction mapping unit is used to perform fitting calculations based on the relative position relationship between multiple correction guide points and the corresponding position relationship between multiple pupil angles to obtain the user's pupil movement mapping model, wherein the pupil movement mapping model is used to characterize the actual visual center change corresponding to the user's pupil rotating at a certain angle.

[0013] As a further solution of the present invention: it also includes a resource optimization module, specifically including: A first center determination unit, configured to obtain the user's visual center in real time through pupil tracking, and set the visual center as a first imaging center; The display optimization management unit is used to establish multiple annular display areas based on the first imaging center as the center of the circle, and the multiple annular display areas correspond to multiple display output schemes respectively, and the display output resolution and content refresh frequency of the multiple annular display areas along the radial direction gradually decrease.

[0014] As a further solution of the present invention: the resource optimization module further includes: An updating synchronization unit is used to acquire the environmental spatial data within the scanning field of view in real time and update the difference data based on the environmental twin model; A second center determination unit, configured to establish a second imaging center based on the object area updated by the difference data, wherein the second imaging center is different from the first imaging center and does not include a plurality of accompanying annular display areas; The display resource management unit is used to adopt a display output scheme consistent with that of the first imaging center for the corresponding display data content of the second imaging center.

[0015] Compared with the prior art, the beneficial effects of the present invention are: a head-mounted display device suitable for mixed reality, with the cooperation of image sensors and posture sensors, etc., realizes real-time virtual synchronization of environmental data and its own movement, thereby achieving the purpose of enhancing the visual interaction experience with display through mixed reality, and through the mapping correction of imaging by the perspective shaking angle and the correction of visual centering, it has a more immersive virtual overlapping experience compared with the prior art, can output picture content more accurately to achieve positioning coverage of real scenes, and achieve the best output state effect in conjunction with perspective positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention is a flowchart of an imaging detection method for a head-mounted display device.

[0017] Figure 2 A flowchart of the step of synchronizing a user's viewing angle with a display viewing angle of a display device in an imaging detection method for a head-mounted display device.

[0018] Figure 3 The present invention is a block diagram of the composition of an imaging detection system of a head-mounted display device. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0021] like Figure 1 The imaging detection method of a head mounted display device provided by an embodiment of the present invention comprises the following steps: S10, acquiring environmental spatial data within the scanning field of view in real time through a sensor device group, and performing high-precision spatial modeling based on the environmental spatial data to obtain an environmental twin model, wherein the environmental spatial data includes environmental image data and laser ranging data; S20, based on the posture sensing device group, obtaining the current motion posture data of the display device and performing real-time synchronous update, and performing motion integral calculation based on the motion posture data to obtain the spatial motion stroke within the time period, wherein the motion posture data includes posture orientation data and motion inertia data; S30, obtaining a user's viewing angle, and mapping a display image based on the viewing angle to synchronize the user's viewing angle with the display viewing angle of the display device, wherein the viewing angle is used to represent a visual center deflection angle of the user between two time nodes; S40, tracking the pupil of the user, obtaining the user's visual center through an initial request, calculating the spatial perspective difference between the visual center and the current posture orientation data, and calibrating the display output field of view based on the spatial perspective difference, the spatial perspective difference is used to characterize the spatial orientation angle between the user's visual center and the display output center of the display device, and the visual center is used to characterize the user's head-on viewing angle.

[0022] In this embodiment, an imaging detection method for a head-mounted display device is provided, which is applicable to a head-mounted display device of mixed reality. With the cooperation of an image sensor and a posture sensor, etc., real-time virtual synchronization of environmental data and its own motion is realized, so as to achieve the purpose of enhancing the visual interaction experience with the display through mixed reality, and the mapping correction of the imaging by the perspective shaking angle and the correction of the visual center make it have a more immersive virtual overlapping experience compared with the prior art, and can output the picture content more accurately to achieve the positioning coverage of the real scene, and cooperate with the perspective positioning to achieve the best output state effect; Mixed reality MR is a hybrid technology solution based on virtual reality and augmented reality. In the mixed reality, the user can not only see the real world, but also see the virtual content generated by the computer. Through the combination of the two, a more immersive real world interaction effect is achieved; At present, mixed reality devices mainly include two types, one is a fully virtual mixed reality display device with full field of view coverage. This mixed reality display device completely collects image data through a camera module, performs visual isolation, and then completely provides visual interaction by the display device; the other is a semi-hybrid type, in which the user can still directly see the real world. Compared with the former, the latter is due to In order to still be able to directly see the original external environment, it can provide more realistic feedback in terms of perception, while the former can obtain a better sense of immersion due to the partition effect; generally speaking, in the prior art, when a semi-hybrid wearable device is used, environmental data will be collected through sensing devices and cameras, etc., environmental objects and related content will be identified and judged, and then the content to be output will be generated and output through a display projection device. In this process, a posture sensor, etc. can perform motion calculation through relevant readings such as inertial posture, so as to achieve synchronization with the motion of the real user. However, in the prior art, the superposition of virtual and real mostly adopts the method of camera ranging, and because of the different wearing habits and visual habits of different users, the zoom superposition method with a fixed mode has a large error; in this embodiment, a motion correction method is adopted, and by cooperating with the user, the user's fixed-point motion angle information is mapped with the display content, so that the user's viewing angle obtained in the display device can be highly overlapped with the real viewing angle. The initial request here is used to represent sending an initialization request to the user. At this time, the user will realize his usual viewing angle under normal conditions, so that the user's standard visual center can be detected, which can be used to perform field of view correction management on the output image.

[0023] like Figure 2 As shown, as another preferred embodiment of the present invention, the step of obtaining the user's viewing angle, mapping the display image based on the viewing angle, and synchronizing the user's viewing angle with the display viewing angle of the display device specifically includes: S41, when it is detected that the wearing state of the display device changes, a viewing angle range synchronization request is generated, and after determining to obtain the visual center of the user, a distance between the pupil of the user and the projection display of the display device is detected; S42, in response to the range synchronization request, randomly selecting a pair of synchronization reference nodes based on the environment twin model, and displaying and marking the pair of synchronization reference nodes in the environment twin model; S43, when the visual center of the display device coincides with the synchronization reference node, the current posture orientation data is recorded, and the posture inclination difference of the posture orientation data between a pair of synchronization reference nodes is calculated, that is, the viewing angle shaking angle of the corresponding display device; S44, calculating the visual width at the corresponding viewing angle shaking angle based on the projection display distance, calculating the display scaling ratio through the actual display distance of a pair of synchronous reference nodes, and performing viewing angle synchronous scaling on the output picture of the display device based on the display scaling ratio.

[0024] In this embodiment, the steps of mapping based on the viewing angle shaking are further explained. When a change in the wearing state of the device is detected, the correction step is started. First, the pupil distance between the device and the user's pupil after wearing is required as an important parameter for subsequent mapping; two synchronization reference points with a certain distance between them are randomly generated, and the user is informed through content interaction. When the user moves his head to make the synchronization reference point overlap with the visual center, the current posture orientation data is marked. By calculating the plane angle of the user's movement between the two synchronization reference points, the angle information of the convergence in the user's observation can be obtained (with the user's eyeball as the starting point of the angle). If you want to achieve an effect of complete overlap with reality, the two synchronization reference points in the display device should also have the same angle relative to the pupil, so as to achieve an overlapping effect in the user's realization. Therefore, the visual width is calculated by the viewing angle shaking angle and the pupil distance.

[0025] As another preferred embodiment of the present invention, it also includes a user characterization correction step, specifically including: When tracking the user's pupil, generate a view initialization request to guide the user's view center to coincide with the display center of the display device. If the user's pupil angle does not coincide with the display center of the display device, guide the user to perform feature correction. Generate multiple sets of correction guide points distributed in multiple plane quadrants of the display range, and record the pupil angles of the user at the multiple correction guide points when the user feedback visual centers coincide; Based on the relative position relationship between the multiple correction guide points and the corresponding position relationship between the multiple pupil angles, a fitting calculation is performed to obtain the user's pupil movement mapping model, which is used to characterize the actual visual center change corresponding to when the user's pupil rotates a certain angle.

[0026] In this embodiment, specifically for individual users, some users may have strabismus due to physical reasons. Therefore, the eye scanning tracking of related display devices in the prior art still has serious adaptation problems. For a small number of users with strabismus, the system cannot accurately identify the user's accurate visual center position, which will cause output errors in image content. The solution adopted here is to randomly generate correction guide points distributed in multiple quadrants within the output viewing angle range of the display device, and the users align them one by one and provide feedback. Finally, the system calculates the strabismus deviation of the user's pupil when it moves in different directions based on the actual positions of several correction guide points and the relative position of the user's pupil, and uses it as a pupil movement mapping model of the user's characteristics for automatic correction during pupil tracking in subsequent use.

[0027] As another preferred embodiment of the present invention, it also includes an imaging resource optimization step: Acquire the user's visual center in real time through pupil tracking, and set the visual center as the first imaging center; A plurality of annular display areas are established based on the first imaging center as the center of the circle. The plurality of annular display areas correspond to a plurality of display output schemes respectively. The display output resolution and content refresh frequency of the plurality of annular display areas along the radial direction gradually decrease.

[0028] Furthermore, the imaging resource optimization step further includes: Acquire the environmental spatial data within the scanning field of view in real time, and update the difference data based on the environmental twin model; Establishing a second imaging center based on the object area updated by the difference data, wherein the second imaging center is different from the first imaging center, and the second imaging center does not include a plurality of accompanying annular display areas; A display output scheme consistent with that of the first imaging center is adopted for the corresponding display data content of the second imaging center.

[0029] In this embodiment, the limited computing resources and energy consumption of the real device are managed and optimized. The computing resource usage and energy consumption are reduced by weakening the display quality outside the visual focus area. At the same time, when a new object content appears in the field of view, it is often easy to cause the user's attention to shift, causing the user to quickly shift the visual viewing object. Therefore, for the corresponding content of the difference data update, the speed of this process is much higher than the speed of switching the line of sight when browsing the object normally. In order to ensure the user's fast perspective switching, the corresponding second imaging center should also adopt complete resource scheduling to ensure the output resolution and refresh frame rate, etc.

[0030] like Figure 3 As shown, the present invention also provides an imaging detection system for a head mounted display device, which comprises: The environment synchronization module 100 is used to obtain the environment space data within the scanning field of view in real time through the sensor device group, and perform high-precision space modeling based on the environment space data to obtain the environment twin model, wherein the environment space data includes environment image data and laser ranging data; The motion synchronization module 200 is used to obtain the current motion posture data of the display device based on the posture sensing device group and perform real-time synchronous update, and perform motion integral calculation based on the motion posture data to obtain the spatial motion stroke within a time period, wherein the motion posture data includes posture orientation data and motion inertia data; A view synchronization module 300 is used to obtain a user's view shaking angle, and to map a display image based on the view shaking angle to synchronize the user's view with the display view of the display device, wherein the view shaking angle is used to represent a user's visual center deflection angle between two time nodes; The field of view calibration module 400 is used to track the pupil of the user, obtain the user's visual center through an initial request, calculate the spatial perspective difference between the visual center and the current posture orientation data, and calibrate the display output field of view based on the spatial perspective difference. The spatial perspective difference is used to characterize the spatial orientation angle between the user's visual center and the display output center of the display device. The visual center is used to characterize the user's head-on viewing angle.

[0031] As another preferred embodiment of the present invention, the view synchronization module 300 specifically includes: The viewing distance detection unit 310 is used to generate a viewing angle range synchronization request when detecting that the wearing state of the display device has changed, and after determining to obtain the user's visual center, detect and obtain the projection display distance between the user's pupil and the display device; A reference marking unit 320 is used to respond to the range synchronization request, randomly select a pair of synchronization reference nodes based on the environment twin model, and display mark the pair of synchronization reference nodes in the environment twin model; The rotation angle acquisition unit 330 is used to record the current posture orientation data when the visual center of the display device coincides with the synchronization reference node, and calculate the posture inclination difference of the posture orientation data between a pair of synchronization reference nodes, that is, the viewing angle shaking angle of the corresponding display device; The field of view synchronization unit 340 is used to calculate the visual width under the corresponding viewing angle shaking angle based on the projection display distance, and calculate the display scaling ratio through the actual display distance of a pair of synchronization reference nodes, and perform perspective synchronization scaling on the output picture of the display device based on the display scaling ratio.

[0032] As another preferred embodiment of the present invention, it also includes a characterization correction module, specifically including: A correction initialization unit, used to generate a view angle initialization request when tracking the user's pupil, so as to guide the user's view angle center to coincide with the display center of the display device, and to guide the user to perform a characteristic correction if the user's pupil angle does not coincide with the display center of the display device at this time; A correction marking unit, used to generate multiple groups of correction guide points distributed in multiple plane quadrants of the display range, and record the pupil angles of the user at the multiple correction guide points when the user feedbacks that the visual centers coincide; The correction mapping unit is used to perform fitting calculations based on the relative position relationship between multiple correction guide points and the corresponding position relationship between multiple pupil angles to obtain the user's pupil movement mapping model, wherein the pupil movement mapping model is used to characterize the actual visual center change corresponding to the user's pupil rotating at a certain angle.

[0033] As another preferred embodiment of the present invention, it also includes a resource optimization module, specifically including: A first center determination unit, configured to obtain the user's visual center in real time through pupil tracking, and set the visual center as a first imaging center; The display optimization management unit is used to establish multiple annular display areas based on the first imaging center as the center of the circle, and the multiple annular display areas correspond to multiple display output schemes respectively, and the display output resolution and content refresh frequency of the multiple annular display areas along the radial direction gradually decrease.

[0034] As another preferred embodiment of the present invention, the resource optimization module further includes: An updating synchronization unit is used to acquire the environmental spatial data within the scanning field of view in real time and update the difference data based on the environmental twin model; A second center determination unit, configured to establish a second imaging center based on the object area updated by the difference data, wherein the second imaging center is different from the first imaging center and does not include a plurality of accompanying annular display areas; The display resource management unit is used to adopt a display output scheme consistent with that of the first imaging center for the corresponding display data content of the second imaging center.

[0035] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0036] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure in the specification and examples. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0037] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An imaging detection method for a head mounted display device, characterized in that: Include: Acquire environmental spatial data within the scanning field of view in real time through a sensor device group, and perform high-precision spatial modeling based on the environmental spatial data to obtain an environmental twin model, wherein the environmental spatial data includes environmental image data and laser ranging data; Based on the posture sensing device group, the current motion posture data of the display device is obtained and updated synchronously in real time, and the motion integral calculation is performed through the motion posture data to obtain the spatial motion stroke within the time period, wherein the motion posture data includes posture orientation data and motion inertia data; Acquire the user's viewing angle, and map the display image based on the viewing angle to synchronize the user's viewing angle with the display viewing angle of the display device, wherein the viewing angle is used to represent the user's visual center deflection angle between two time nodes; The user's pupil is tracked, and the user's visual center is obtained through an initial request. The spatial perspective difference between the visual center and the current posture orientation data is calculated, and the display output field of view is calibrated based on the spatial perspective difference. The spatial perspective difference is used to characterize the spatial orientation angle between the user's visual center and the display output center of the display device. The visual center is used to characterize the user's head-on viewing angle.

2. The imaging detection method of a head mounted display device according to claim 1, characterized in that: The step of obtaining the user's viewing angle and mapping the display image based on the viewing angle to synchronize the user's viewing angle with the display viewing angle of the display device specifically includes: When it is detected that the wearing state of the display device changes, a viewing angle range synchronization request is generated, and after determining to obtain the visual center of the user, a projection display distance between the user's pupil and the display device is detected; In response to the range synchronization request, a pair of synchronization reference nodes are randomly selected based on the environment twin model, and the pair of synchronization reference nodes are displayed and marked in the environment twin model; When the visual center of the display device coincides with the synchronization reference node, the current posture orientation data is recorded, and the posture inclination difference of the posture orientation data between a pair of synchronization reference nodes is calculated, that is, the viewing angle shaking angle of the corresponding display device; The visual width under the corresponding viewing angle shaking angle is calculated based on the projection display distance, and the display scaling ratio is calculated through the actual display distance of a pair of synchronous reference nodes, and the output picture of the display device is synchronously scaled based on the display scaling ratio.

3. The imaging detection method of a head mounted display device according to claim 2, characterized in that: It also includes user characterization correction steps, including: When tracking the user's pupil, generate a view initialization request to guide the user's view center to coincide with the display center of the display device. If the user's pupil angle does not coincide with the display center of the display device, guide the user to perform feature correction. Generate multiple sets of correction guide points distributed in multiple plane quadrants of the display range, and record the pupil angles of the user at the multiple correction guide points when the user feedback visual centers coincide; Based on the relative position relationship between the multiple correction guide points and the corresponding position relationship between the multiple pupil angles, a fitting calculation is performed to obtain the user's pupil movement mapping model, which is used to characterize the actual visual center change corresponding to when the user's pupil rotates a certain angle.

4. The imaging detection method of a head mounted display device according to claim 1, characterized in that: Also included are steps for optimizing imaging resources: Acquire the user's visual center in real time through pupil tracking, and set the visual center as the first imaging center; A plurality of annular display areas are established based on the first imaging center as the center of the circle. The plurality of annular display areas correspond to a plurality of display output schemes respectively. The display output resolution and content refresh frequency of the plurality of annular display areas along the radial direction gradually decrease.

5. The imaging detection method of a head mounted display device according to claim 4, characterized in that: The imaging resource optimization step further comprises: Acquire the environmental spatial data within the scanning field of view in real time, and update the difference data based on the environmental twin model; Establishing a second imaging center based on the object area updated by the difference data, wherein the second imaging center is different from the first imaging center, and the second imaging center does not include a plurality of accompanying annular display areas; A display output scheme consistent with that of the first imaging center is adopted for the corresponding display data content of the second imaging center.

6. An imaging detection system for a head mounted display device, characterized in that: Include: An environmental synchronization module is used to obtain environmental spatial data within the scanning field of view in real time through a sensor device group, and to perform high-precision spatial modeling based on the environmental spatial data to obtain an environmental twin model. The environmental spatial data includes environmental image data and laser ranging data. A motion synchronization module is used to obtain the current motion posture data of the display device based on the posture sensing device group and perform real-time synchronous update, and perform motion integral calculation based on the motion posture data to obtain the spatial motion stroke within a time period, wherein the motion posture data includes posture orientation data and motion inertia data; A perspective synchronization module, used to obtain a user's perspective shaking angle, and to map a display image based on the perspective shaking angle to synchronize the user's perspective with the display perspective of the display device, wherein the perspective shaking angle is used to represent a user's visual center deflection angle between two time nodes; The field of view calibration module is used to track the user's pupil, obtain the user's visual center through an initial request, calculate the spatial perspective difference between the visual center and the current posture orientation data, and calibrate the display output field of view based on the spatial perspective difference. The spatial perspective difference is used to characterize the spatial orientation angle between the user's visual center and the display output center of the display device. The visual center is used to characterize the user's head-on viewing angle.

7. The imaging detection system of a head mounted display device according to claim 6, characterized in that: The perspective synchronization module specifically includes: A viewing distance detection unit, configured to generate a viewing angle range synchronization request when detecting that a wearing state of the display device has changed, and after determining to obtain the visual center of the user, detect and obtain the projection display distance between the user's pupil and the display device; A reference marking unit, configured to respond to the range synchronization request, randomly select a pair of synchronization reference nodes based on the environment twin model, and display mark the pair of synchronization reference nodes in the environment twin model; A rotation angle acquisition unit, used for recording the current posture orientation data when the visual center of the display device coincides with the synchronization reference node, and calculating the posture inclination difference of the posture orientation data between a pair of synchronization reference nodes, that is, the viewing angle shaking angle of the corresponding display device; The field of view synchronization unit is used to calculate the visual width under the corresponding viewing angle shaking angle based on the projection display distance, and calculate the display scaling ratio through the actual display distance of a pair of synchronization reference nodes, and perform perspective synchronization scaling on the output picture of the display device based on the display scaling ratio.

8. The imaging detection system of a head mounted display device according to claim 7, characterized in that: It also includes a feature correction module, including: A correction initialization unit, used to generate a view angle initialization request when tracking the user's pupil, so as to guide the user's view angle center to coincide with the display center of the display device, and to guide the user to perform a characteristic correction if the user's pupil angle does not coincide with the display center of the display device at this time; A correction marking unit, used to generate multiple groups of correction guide points distributed in multiple plane quadrants of the display range, and record the pupil angles of the user at the multiple correction guide points when the user feedbacks that the visual centers coincide; The correction mapping unit is used to perform fitting calculations based on the relative position relationship between multiple correction guide points and the corresponding position relationship between multiple pupil angles to obtain the user's pupil movement mapping model, wherein the pupil movement mapping model is used to characterize the actual visual center change corresponding to the user's pupil rotating at a certain angle.

9. The imaging detection system of a head mounted display device according to claim 6, characterized in that: It also includes resource optimization modules, including: A first center determination unit, configured to obtain the user's visual center in real time through pupil tracking, and set the visual center as a first imaging center; The display optimization management unit is used to establish multiple annular display areas based on the first imaging center as the center of the circle, and the multiple annular display areas correspond to multiple display output schemes respectively, and the display output resolution and content refresh frequency of the multiple annular display areas along the radial direction gradually decrease.

10. The imaging detection system of a head mounted display device according to claim 9, characterized in that: The resource optimization module also includes: An updating synchronization unit is used to acquire the environmental spatial data within the scanning field of view in real time and update the difference data based on the environmental twin model; A second center determination unit, configured to establish a second imaging center based on the object area updated by the difference data, wherein the second imaging center is different from the first imaging center and does not include a plurality of accompanying annular display areas; The display resource management unit is used to adopt a display output scheme consistent with that of the first imaging center for the corresponding display data content of the second imaging center.

Images