Augmented reality display device and display method thereof, vehicle, mobile terminal, and medium
By judging the display method of the augmented reality display device that the user intends to watch, the control screen is always in the field of view when the user drives and rides a vehicle, solving the problem of screen occlusion and edge cutting, improving the user experience, and improving the viewing effect through anti-shake processing .
Patent Information
- Application Number
- PCT/CN2024/090997
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-06
- Filing Date
- 2024-04-30
- Publication Date
- 2024-11-14
AI Technical Summary
In mobile scenarios such as user driving and riding vehicles, the display method of augmented reality display device usually causes the screen to move with the user, causing information to be blocked, affecting the user's viewing experience, and the prior art is difficult to effectively solve the problem of picture edge cutting.
By judging the relative motion information between the user's wearing part and the movable carrier, determine whether the user intends to view the display screen. If so, the control screen will follow the movement of the wear part in the 3D moving space. If otherwise, the control screen will be displayed in the set area to avoid Information is blocked from each other and anti-shake processing is performed to improve the user experience.
Make sure that the display screen is always in the user's field of view, avoid information being blocked, improve user's viewing experience, make full use of the 3D mobile space to display the picture information provided by different devices, and prevent picture shaking and dizziness.
Smart Images

Figure CN2024090997_14112024_PF_FP_ABST
Abstract
Description
Augmented reality display device and display method thereof, vehicle, mobile terminal, and medium
[0001] This application claims priority to Chinese patent application No. 202310497935.4 filed on May 6, 2023, with the invention name “Augmented reality display device and display method thereof, vehicle, mobile terminal, medium”. The entire contents of the above Chinese patent application are incorporated into this application by reference. Technical Field
[0002] The present application relates to the field of augmented reality technology, and in particular to an augmented reality display device and a display method thereof, a vehicle, a mobile terminal, and a medium. Background Art
[0003] Augmented reality display devices mainly use augmented reality (AR) technology to fuse and display computer-generated virtual world information with real-world information. Users can view the displayed images by wearing augmented reality display devices (such as AR glasses).
[0004] With the continuous development of augmented reality technology, the application scenarios of augmented reality display devices have also increased. For example, users can watch navigation information or entertainment videos through augmented reality display devices while driving a vehicle. However, in mobile scenarios such as users driving a vehicle, the current display method of augmented reality display devices usually controls the screen to be in the user's field of view, that is, the screen will always move with the user's movement. For example, if the user moves to the left, the screen will also move to the left synchronously. The screen moves with the user, which may cause the information displayed by some devices originally on the left to be blocked by the synchronously moving screen, making it impossible for the user to see the information displayed on the screen or the information displayed by the device originally on the left. In order to avoid this problem, the information displayed on the screen can only be minimized, but this will greatly reduce the user experience.
[0005] While current display methods can control the image to move freely relative to the user in certain directions in non-mobile scenarios, such as indoor movie viewing, this approach can easily cause image clipping when applied to mobile scenarios, impacting the user's viewing experience. Specifically, the field of view (FOV) of an augmented reality display device in mobile scenarios is limited. If the image is controlled to move freely relative to the user when the user makes slight movements (such as changing posture), it is likely that a portion of the image will fall outside the FOV (image clipping), making it impossible for the user to view this portion.
[0006] Accordingly, this field requires a new technical solution to solve the above problems.
[0007] Summary of the Invention
[0008] In order to overcome the above-mentioned defects, the present application is proposed to provide an augmented reality display device and its display method, vehicle, mobile terminal, and medium that solve or at least partially solve the technical problem of improving the display effect of the augmented reality display device in mobile scenarios such as when the user is driving a vehicle, so as to improve the user experience.
[0009] In a first aspect, a display method of an augmented reality display device is provided, comprising:
[0010] When a user is wearing the augmented reality display device and is in the 3D moving space corresponding to the movable carrier, determining whether the user intends to view the display image of the augmented reality display device in the 3D moving space based on information about the relative movement between the wearing part and the movable carrier;
[0011] If yes, controlling the display image to move in the 3D moving space following the movement of the wearing part, so that the display image is always located within the user's field of view determined by the position of the wearing part;
[0012] If not, the display image is controlled to be displayed within a set area of the 3D moving space, where the set area is an area corresponding to the user's visual field determined by the position of the wearing part when there is no relative motion between the wearing part and the movable carrier.
[0013] In one technical solution of the display method of the above-mentioned augmented reality display device, the step of “determining whether the user intends to view the display image of the augmented reality display device in the 3D mobile space” specifically includes:
[0014] According to the relative motion information, obtaining the relative displacement generated by the relative translational motion and / or the relative rotational amplitude generated by the relative rotational motion between the wearing part and the movable carrier along the spatial coordinate axis;
[0015] Whether the user intends to view the display image is determined based on the relative displacement and / or the relative rotation amplitude.
[0016] In one technical solution of the display method of the above-mentioned augmented reality display device, the step of “determining whether the user intends to view the display image based on the relative displacement and / or the relative rotation amplitude” specifically includes:
[0017] Obtaining preset amplitude thresholds and / or displacement thresholds corresponding to each spatial coordinate axis;
[0018] Determining whether the relative rotation amplitude of at least one spatial coordinate axis is greater than a corresponding amplitude threshold and / or whether the relative displacement of at least one spatial coordinate axis is greater than a corresponding displacement threshold;
[0019] If so, determining that the user intends to view the display image;
[0020] If not, it is determined that the user does not intend to view the display image.
[0021] In one technical solution of the display method of the above-mentioned augmented reality display device, the preset amplitude threshold and displacement threshold corresponding to each spatial coordinate axis are obtained respectively by the following methods:
[0022] Obtaining first historical information of the current user using the augmented reality display device;
[0023] Training a first threshold model according to the first historical information, and using the trained first threshold model to obtain an amplitude threshold and a displacement threshold corresponding to each spatial coordinate axis matched with the current user;
[0024] The first historical information includes information about the relative movement between the wearing part and the movable carrier and corresponding real viewing results, where the real viewing results include whether the current user has viewed the display image and whether the current user has not viewed the display image.
[0025] In one technical solution of the display method of the above-mentioned augmented reality display device, in the process of controlling the display image to move in the 3D moving space in accordance with the movement of the wearing part, the method further includes: performing anti-shake processing on the display image;
[0026] And / or, in the process of controlling the display image to be displayed within a set area of the 3D mobile space, the method further includes:
[0027] The visual field image within the user's field of view is anti-shake processed, and the user's field of view is determined by the position of the wearing part. The visual field image includes at least a portion of the display image and / or the image displayed in the 3D mobile space by other devices other than the current augmented reality display device.
[0028] In one technical solution of the display method of the above-mentioned augmented reality display device, in the process of controlling the display image to be displayed within a set area of the 3D moving space, the method further includes:
[0029] determining, based on the position of the wearing part, whether the user intends to review the displayed image;
[0030] If yes, re-controlling the display image to move in the 3D moving space following the movement of the wearing part, so that the display image is always located within the user's field of view determined by the position of the wearing part;
[0031] If not, continue to control the display image to be displayed within the set area of the 3D moving space.
[0032] In one technical solution of the display method of the above-mentioned augmented reality display device, the step of "determining whether the user intends to look back at the displayed image based on the position of the wearing part" specifically includes:
[0033] Determine the current user's field of view based on the location of the wearing part;
[0034] According to the field of view area corresponding to the current user field of view in the 3D mobile space and the set area, it is determined whether the user intends to look back at the displayed image.
[0035] In one technical solution of the display method of the above-mentioned augmented reality display device, the step of "determining whether the user intends to look back at the displayed image based on the field of view area corresponding to the current user field of view in the 3D moving space and the set area" specifically includes:
[0036] Determining whether the visual field area overlaps with the set area;
[0037] If so, determining that the user intends to review the displayed image;
[0038] If not, it is determined that the user does not intend to review the displayed image.
[0039] In one technical solution of the display method of the augmented reality display device, the step of "determining whether the field of view area overlaps with the set area" includes: obtaining a degree of regional overlap between the field of view area and the set area, and determining whether the degree of regional overlap is greater than a preset overlap threshold; if so, determining that there is overlap; if not, determining that there is no overlap;
[0040] Alternatively, the step of "determining whether the field of view area overlaps with the set area" includes: determining whether the degree of overlap of the areas is greater than a preset overlap threshold and whether the duration reaches a preset first duration threshold; if so, determining that there is overlap; if not, determining that there is no overlap.
[0041] In one technical solution of the display method of the above-mentioned augmented reality display device, the preset overlap threshold and the preset first duration threshold are respectively obtained by the following methods:
[0042] Obtaining second historical information of the current user using the augmented reality display device;
[0043] Training a second threshold model based on the second historical information, and using the trained second threshold model to respectively obtain a coincidence threshold and a first duration threshold that match the current user;
[0044] Among them, the second historical information includes the position of the wearing part and the corresponding real review results in the process of controlling the display screen to be displayed in the set area of the 3D mobile space, and the real review results include whether the current user has reviewed the display screen and whether the current user has not reviewed the display screen.
[0045] In one technical solution of the display method of the above-mentioned augmented reality display device, the step of “determining whether the user intends to look back at the displayed image based on the position of the wearing part” further includes:
[0046] According to the position of the wearing part, information on the relative movement between the wearing part and the movable carrier is obtained;
[0047] Based on the information, it is determined whether the user intends to review the displayed image.
[0048] In one technical solution of the display method of the above-mentioned augmented reality display device, the step of "determining, based on the information, whether the user intends to review the displayed image" specifically includes: determining, based on the information, whether the wearing portion and the movable carrier are in a stationary state without relative motion; if so, determining that the user intends to review the displayed image; if not, determining that the user does not intend to review the displayed image;
[0049] Alternatively, the step of "determining whether the user intends to look back at the display screen based on the information" specifically includes: determining, based on the information, whether the wearing part and the movable carrier are in a stationary state without relative movement and whether the duration reaches a preset second duration threshold; if so, determining that the user intends to look back at the display screen; if not, determining that the user does not intend to look back at the display screen.
[0050] In one technical solution of the display method of the above-mentioned augmented reality display device, the preset second duration threshold is obtained by:
[0051] Obtain third historical information of the current user using the augmented reality display device;
[0052] Training a third threshold model according to the third historical information, and using the trained third threshold model to obtain a second duration threshold that matches the current user;
[0053] Among them, the third historical information includes the position of the wearing part in the process of controlling the display screen to be displayed in the set area of the 3D mobile space and the corresponding real review results, and the real review results include whether the current user has reviewed the display screen and whether the current user has not reviewed the display screen.
[0054] In one technical solution of the display method of the above-mentioned augmented reality display device, in the process of controlling the display image to move in the 3D moving space in accordance with the movement of the wearing part, and / or in the process of controlling the display image to be displayed in a set area of the 3D moving space, the method further includes:
[0055] In response to the received display exception processing instruction, forcibly controlling the display image to be displayed within the set area,
[0056] The display image is then controlled to move in the 3D moving space following the movement of the wearing part.
[0057] In one technical solution of the display method of the above-mentioned augmented reality display device, the method further includes determining whether the user is wearing the augmented reality display device by the following method:
[0058] determining whether first information fed back by a screen information output device used in conjunction with the augmented reality display device is received, and whether second information fed back by the augmented reality display device after detecting that the augmented reality display device is worn on the wearing portion is received, wherein the first information is generated by the screen information output device after detecting that communication is established with the augmented reality display device;
[0059] If yes, it is determined that the user is wearing an augmented reality display device;
[0060] If not, it is determined that the user is not wearing the augmented reality display device.
[0061] In one technical solution of the display method of the augmented reality display device, the method further comprises obtaining information on the relative motion between the wearing part and the movable carrier by: obtaining the information on the relative motion based on sensor data of the augmented reality display device and / or sensor data of the movable carrier;
[0062] And / or the step of “controlling the display image to move in the 3D moving space in accordance with the movement of the wearing part” includes: controlling the display image to move in the 3D moving space in accordance with the movement of the wearing part based on sensor data of the augmented reality display device and / or sensor data of the movable carrier, so that the display image is always located within the user's field of view determined by the position of the wearing part;
[0063] And / or, the step of "controlling the display screen to be displayed within the set area of the 3D mobile space" specifically includes: controlling the display screen to be displayed within the set area of the 3D mobile space based on the sensor data of the augmented reality display device and / or the sensor data of the movable carrier.
[0064] In a second aspect, an augmented reality display device is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the method described in any one of the technical solutions of the display method of the above-mentioned augmented reality display device.
[0065] In one technical solution of the above-mentioned augmented reality display device, the augmented reality display device is augmented reality glasses, eye mask or helmet.
[0066] In a third aspect, a vehicle is provided, comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, and the program codes are suitable for being loaded and run by the processor to execute the method described in any one of the technical solutions of the display method of the above-mentioned augmented reality display device.
[0067] In a fourth aspect, a mobile terminal is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the method described in any one of the technical solutions of the display method of the above-mentioned augmented reality display device.
[0068] In a fifth aspect, a computer-readable storage medium is provided, which stores a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the display method of the above-mentioned augmented reality display device.
[0069] The above one or more technical solutions of this application have at least one or more of the following beneficial effects:
[0070] In a technical solution for implementing the method provided by the present application, it is possible to accurately determine whether the user intends to view (wants to view) the display screen of the augmented reality display device based on the information of the relative movement between the wearing part and the movable carrier. If it is determined that the user wants to view, the display screen can be controlled to move in the 3D mobile space corresponding to the movable carrier to follow the movement of the wearing part, so that the display screen is always located within the user's field of view determined by the position of the wearing part. This control method can ensure that the display screen completely follows the user, ensuring that the user can clearly and completely view the content of the display screen. In addition, even if the user makes slight movements, as long as it is determined that he wants to view the display screen, the screen can continue to be controlled to move completely in accordance with the movement of the wearing part, avoiding the problem of screen clipping. If it is determined that the user does not want to view, the display screen can be controlled to be displayed within a set area of the 3D mobile space to avoid interfering with the user. In this way, the problem of information blocking each other in the prior art can also be avoided, so that the 3D mobile space can be fully utilized to display the screen information provided by different devices.
[0071] In another technical solution for implementing the method provided in the present application, the display image can be anti-shake processed in the process of controlling the display image of the augmented reality display device, thereby preventing the user from not being able to see the image clearly or even feeling dizzy while watching the display image. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] The disclosure of this application will become more easily understood with reference to the accompanying drawings. Those skilled in the art will readily appreciate that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Among them:
[0073] FIG1 is a schematic flow chart of main steps of a display method of an augmented reality display device according to an embodiment of the present application;
[0074] FIG2 is a flowchart showing the main steps of a method for determining whether a user intends to view a display image according to an embodiment of the present application;
[0075] 3 is a flow chart showing the main steps of a method for controlling screen display based on a determination result of whether a user intends to review a displayed screen according to an embodiment of the present application;
[0076] FIG4 is a schematic diagram of a processor and a storage device in an augmented reality display device according to an embodiment of the present application. DETAILED DESCRIPTION
[0077] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the scope of protection of the present application.
[0078] In the description of this application, "processor" may include hardware, software, or a combination of the two. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. Computer-readable storage media include any suitable medium that can store program code, such as a magnetic disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" represents all possible combinations of A and B, such as only A, only B, or A and B.
[0079] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, and based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.
[0080] The user personal information processed by this application will vary depending on the specific product / service scenario and must be based on the specific scenario in which the user uses the product / service. This may involve the user's account information, device information, driving information, vehicle information, or other related information. This application will treat the user's personal information and its processing with a high degree of diligence.
[0081] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that comply with industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.
[0082] An embodiment of a display method for an augmented reality display device is described below.
[0083] Referring to FIG1 , FIG1 is a flow chart illustrating the main steps of a display method for an augmented reality display device according to an embodiment of the present application. As shown in FIG1 , the display method for an augmented reality display device in the embodiment of the present application mainly includes the following steps S101 to S104.
[0084] Step S101: When a user wears an augmented reality display device and is in a 3D moving space corresponding to a movable carrier, information on the relative movement between the wearing part and the movable carrier is obtained.
[0085] A movable carrier refers to an object that can carry a user and can be moved. Movable carriers include but are not limited to vehicles, airplanes, ships, bicycles and other means of transportation.
[0086] 3D mobile space refers to a local space that is formed in the 3D physical space according to the position of the movable carrier and can at least surround the movable carrier. This local space will move as the movable carrier moves. For a movable carrier with a shell (such as a car), the space surrounded by the shell can be used as the 3D mobile space; for a movable carrier without a shell (such as a bicycle), the space within a preset range (such as 2 meters) from the position of the movable carrier can be used as the 3D mobile space. It should be noted that those skilled in the art can flexibly set the size of the 3D mobile space according to actual needs, and the embodiments of this application do not specifically limit it.
[0087] Relative motion refers to a motion in which the position of the wearing part changes relative to the position of the movable carrier, and the information of the relative motion may be the amount of change in position.
[0088] Step S102: Based on the relative motion information, determine whether the user intends to watch the display screen of the augmented reality display device in the 3D mobile space; if the user intends to watch, go to step S103; otherwise, go to step S104.
[0089] Specifically, the magnitude of the relative motion information can be used to determine whether the user intends to view the display screen (whether the user has the intention to view the display screen). If the relative motion information is large, it is determined that the user intends to view the display screen; otherwise, it is determined that the user does not intend to view the display screen.
[0090] The augmented reality display device can display a picture in a 3D mobile space. At this time, the display picture to be displayed and controlled by this application is the single picture currently displayed.
[0091] The augmented reality display device can also display multiple different pictures at the same time in the 3D mobile space, and these pictures are distributed at different positions in the 3D mobile space. In this case, the display picture to be displayed and controlled in this application can be a part or all of the above-mentioned multiple different pictures. For example, the central area of the user's field of view can be determined first, and then the picture located in the above-mentioned central area can be obtained as the display picture to be displayed and controlled in this application according to the position of each picture. It should be noted that when the augmented reality display device displays multiple different pictures at the same time, those skilled in the art can flexibly select the picture to be displayed and controlled according to actual needs. The embodiment of the present application does not make specific limitations on this, as long as the picture to be controlled can be selected from multiple pictures.
[0092] Step S103: Control the display image to follow the movement of the wearing part within the 3D moving space, so that the display image always remains within the user's field of view, which is determined by the position of the wearing part. The user's field of view refers to the range covered by the user's line of sight. When determining the user's field of view, the wearing part can be used as the origin to re-determine the 3D position of each spatial point in the 3D moving space, and the user's field of view can be determined based on the 3D position of each spatial point. Under the control of this step, the display image cannot move freely relative to the wearing part, and therefore, under the control of this step, the augmented reality display device is in a zero degrees of freedom (DOF) control mode for the display image.
[0093] Step S104: Controlling the display image to be displayed within a set area in the 3D moving space. The set area is the area corresponding to the user's field of view, determined by the position of the wearing part when there is no relative motion between the wearing part and the movable carrier. The set area can be the central area of the user's field of view. When controlling the display image to be displayed within the set area, the center of the display image, the center of the set area, and the center of the user's field of view can all coincide. Under the control of this step, the display image can move freely in at least one direction relative to the wearing part, and thus, under the control of this step, the augmented reality display device is in a multi-degree-of-freedom control mode for the display image.
[0094] In some embodiments, when the augmented reality display device is powered on and the display screen is first started, step S103 may be executed first, and then the process goes to step S101 and the subsequent steps S102 to S104 are executed normally. Alternatively, step S104 may be executed first, and then the process goes to step S101 and the subsequent steps S102 to S104 are executed normally.
[0095] Based on the method described in steps S101 to S104 above, it is possible to adaptively switch between the zero-degree-of-freedom control mode and the multi-degree-of-freedom control mode according to whether the user intends to view the display screen. In the zero-degree-of-freedom control mode, the display screen will completely follow the movement of the wearing part, so that the user can clearly and completely view the content of the display screen. In the multi-degree-of-freedom control mode, the display screen will not completely follow the movement of the wearing part, so that when the user wants to view other things (such as viewing the scenery outside the car window), the display screen will not interfere with the user. In the multi-degree-of-freedom control mode, different areas of the 3D mobile space can also be fully utilized to display different information, avoiding the problem of information displayed in different areas being blocked by each other due to the movement of the display screen with the wearing part. In addition, during the use of the augmented reality display device, even if the wearing part moves slightly, as long as it is determined that the user has no intention of viewing the display screen, the display screen will be controlled to completely follow the movement of the wearing part, avoiding the occurrence of screen edge cutting.
[0096] In an application scenario based on the above method embodiment, a user can use an augmented reality display device in a vehicle and the device is worn on the user's head. Specifically, after detecting that the user is wearing the augmented reality display device on their head, the display screen is first controlled to move in the 3D moving space of the vehicle following the movement of the head, and in this process, the information of the relative movement between the head and the movable carrier is detected in real time. When the user changes their sitting position or the vehicle turns, the information of the relative movement between the head and the movable carrier is relatively small, indicating that the user still wants to view the display screen, and thus the display screen continues to be controlled to move in the 3D moving space of the vehicle following the movement of the head. When the user tilts his head to look at the scenery outside the window, the information of the relative movement between the head and the movable carrier is relatively large, indicating that the user does not want to view the display screen at this time, and thus the display screen is controlled to be displayed in a set area of the 3D moving space to avoid interfering with the user's line of sight.
[0097] The following describes steps S101 to S104 of the above method embodiment respectively.
[0098] 1. Explain step S101.
[0099] In some implementations of the above step S101, it may be determined whether the user has worn the augmented reality display device in the following manner, so that other steps are performed after confirming that the user has worn the augmented reality display device.
[0100] Specifically, it can be determined whether a first message fed back by a screen information output device used in conjunction with the augmented reality display device has been received, and whether a second message fed back by the augmented reality display device after detecting that the device is being worn at the wearing location has been received. The first message can be generated after the screen information output device detects that communication has been established with the augmented reality display device. If both the first and second messages are received, it is determined that the user is wearing the augmented reality display device; otherwise, it is determined that the user is not wearing the augmented reality display device.
[0101] The image information output device may be a device mounted on a removable carrier, and the device may communicate with the augmented reality display device via wired or wireless communication, for example, via a USB interface or via Bluetooth.
[0102] The augmented reality display device may be pre-installed with a detection component to detect whether it is worn at the wearing part. For example, the augmented reality display device may be provided with an infrared sensor to detect whether the user is within a set range. If so, the augmented reality display device is worn at the wearing part; otherwise, the augmented reality display device is not worn at the wearing part.
[0103] In some implementations of step S101, the information about the relative motion between the wearing part and the movable carrier can be obtained based on sensor data from the augmented reality display device, or the information about the relative motion can be obtained based on sensor data from the movable carrier. Furthermore, the information can be obtained based on sensor data from both the augmented reality display device and the movable carrier.
[0104] For example, when it is impossible to obtain the above information based solely on the sensor data of the augmented reality display device or the movable carrier, the information can be obtained based on the sensor data of both. For another example, since the sensor data of both the augmented reality display device and the movable carrier may contain some errors to a greater or lesser extent, in order to avoid reducing the accuracy of the above information due to these errors, the information can be obtained based on the sensor data of both. For example, relative motion information can be obtained based on the sensor data of both devices, and then the obtained information can be averaged, and the average value can be used as the final relative motion information.
[0105] In this embodiment, the sensor data includes but is not limited to measurement data of an inertial measurement unit (IMU).
[0106] 2. Explain step S102.
[0107] 2 , in some implementations of the above step S102 , it may be determined whether the user intends to view the display image through the following steps S1021 to S1022 .
[0108] Step S1021: According to the relative motion information, the relative displacement generated by the relative translation motion and / or the relative rotation amplitude generated by the relative rotation motion between the wearing part and the movable carrier along the spatial coordinate axis are obtained.
[0109] The spatial coordinate axes may be coordinate axes of a 3D moving space, including three coordinate axes: an X-axis, a Y-axis, and a Z-axis. The wearing portion and the movable carrier may perform relative translational motion along at least one coordinate axis, or may perform relative rotational motion along at least one coordinate axis.
[0110] Step S1022: Determine whether the user intends to view the display image based on the relative displacement and / or relative rotation amplitude.
[0111] Specifically, the user's intention to view the displayed image can be determined based on the magnitude of the relative displacement and / or relative rotation. If the relative displacement and / or relative rotation magnitude is large, the user is determined to have intended to view the displayed image; otherwise, the user is determined not to have intended to view the displayed image.
[0112] Based on the method described in steps S1021 to S1022 above, one or more of relative displacement and relative rotation amplitude can be used to accurately determine whether the user wants to view the display screen, which is conducive to improving the accuracy of switching between zero-degree-of-freedom control mode and multi-degree-of-freedom control mode of the augmented reality display device and improving the user's viewing experience of viewing the screen.
[0113] The above step S1022 is further explained below.
[0114] In some embodiments of step S1022, an amplitude threshold and / or a displacement threshold may be pre-set, and the magnitude of the relative displacement and relative rotation amplitude may be determined by comparing the thresholds. If the relative displacement is greater than the displacement threshold, the relative displacement is large; otherwise, the relative displacement is small. If the relative rotation amplitude is greater than the amplitude threshold, the relative rotation amplitude is large; otherwise, the relative rotation amplitude is small.
[0115] Based on this, in this embodiment, it is possible to quickly and accurately determine whether the user intends to view the display screen through the following steps 11 to 12.
[0116] Step 11: Obtain preset amplitude thresholds and / or displacement thresholds corresponding to each spatial coordinate axis. In this step, the amplitude threshold and displacement threshold can be fixed thresholds that apply to different users, i.e., the same amplitude threshold and displacement threshold are used for different users. Alternatively, the amplitude threshold and displacement threshold can be variable thresholds that apply to different users, i.e., the thresholds are adaptively switched to the thresholds that apply to the current user.
[0117] Step 12: Determine whether the relative rotation amplitude of at least one spatial coordinate axis is greater than the corresponding amplitude threshold and / or whether the relative displacement of at least one spatial coordinate axis is greater than the corresponding displacement threshold; if so, determine that the user intends to view the display screen; if not, determine that the user does not intend to view the display screen.
[0118] Based on the method described in steps 11 to 12 above, it is possible to accurately determine whether the user intends to view the display image based on the thresholds and relative motion information of different spatial coordinate axes.
[0119] In the above step 11, when the amplitude threshold or displacement threshold adopts a fixed threshold that is applicable to different users, those skilled in the art can flexibly set the numerical value of this fixed threshold according to actual needs, and the embodiment of the present application does not make specific limitations.
[0120] In the above step 11, when the amplitude threshold or the displacement threshold adopts a variable threshold that is respectively applicable to different users, the threshold size applicable to each user can be set according to the habit of each user in viewing the display screen. In some preferred embodiments, a big data training model can be used to obtain the threshold value using the trained model. Specifically, for each user, the first historical information of the current user using the augmented reality display device can be obtained, and the first threshold model can be trained based on the first historical information. The trained first threshold model is used to obtain the amplitude threshold and displacement threshold corresponding to each spatial coordinate axis matching the current user. The first historical information may include information on the relative movement between the wearing part and the movable carrier and the corresponding real viewing result. The real viewing result may include whether the current user has viewed the display screen and whether the current user has not viewed the display screen.
[0121] Based on the above implementation, the first historical information can be used to train a first threshold model specifically suitable for the current user, and the threshold value specifically suitable for the current user can be obtained through this model. It should be noted that in the embodiment of the present application, a conventional model training method in the field of machine learning technology can be used to train the first historical information to obtain the first threshold model. The embodiment of the present application does not specifically limit the model training method, as long as a model capable of obtaining an amplitude threshold and a displacement threshold can be obtained.
[0122] 3. Explain step S103.
[0123] In some implementations of the above-mentioned step S103, the movable carrier may shake during the movement. When the movable carrier shakes, the user may also shake along with the shaking of the movable carrier, which causes the augmented reality display device worn by the user to also shake, and ultimately causes the display screen to also shake. When the display screen shakes, it is likely that the user cannot see the content of the screen clearly, and may even cause the user to feel dizzy. In this regard, in the process of controlling the display screen to move along with the movement of the wearing part, the display screen can also be anti-shake processed. Specifically, the position of the display screen in the 3D mobile space can be adjusted according to the motion state of the augmented reality display device, so that the display screen remains basically unchanged relative to the user-set area, wherein the user-set area can be the user's eyes.
[0124] In some embodiments of the above-mentioned step S103, in response to the received display exception handling instruction, the display screen can be forcibly controlled to be displayed within the set area, and then the display screen can be controlled to move in the 3D mobile space following the movement of the wearing part. The display exception handling instruction can be automatically generated when an abnormality is detected in the display screen, or it can be received through the human-computer interaction component. The human-computer interaction component includes but is not limited to physical buttons set on the virtual reality display device or virtual buttons displayed on the touch screen of the virtual reality display device, and the user can trigger the above-mentioned physical buttons or virtual buttons to issue a display exception handling instruction. For example, the user can double-click the virtual button for turning off the screen on the touch screen. It should be noted that after controlling the display screen to move in the 3D mobile space following the movement of the wearing part, the steps in the above-mentioned steps S101 to S104 can still be performed normally.
[0125] In some embodiments of step S103 above, the display screen can be controlled to follow the movement of the wearing part within the 3D moving space based on the sensor data of the augmented reality display device and / or the sensor data of the removable carrier, so that the display screen is always located within the user's field of view determined by the position of the wearing part. Similar to the related method in step S101 above, when it is impossible to control the display screen based on the sensor data of the augmented reality display device or the removable carrier alone, control can be performed based on the sensor data of both. Or, to avoid reducing the accuracy of display screen control due to errors in the sensor data, control can also be performed based on the sensor data of both.
[0126] 4. Explain step S104.
[0127] In some embodiments of the above step S104, when controlling the display image to be displayed within the set area of the 3D mobile space, a portion of the display image may still exist within the user's field of view determined by the wearing position. If this portion of the display image shakes, it will also cause interference to the user. In addition, the images displayed by other devices in the user's field of view in the 3D mobile space may also shake, thereby causing interference to the user. To this end, in the process of controlling the display image to be displayed within the set area of the 3D mobile space, the field of view image located in the user's field of view can be anti-shake processed. The field of view image can include at least a portion of the display image and / or images displayed in the 3D mobile space by other devices other than the current augmented reality display device. For the display image included in the field of view image, the same method as the above step S103 can be used for anti-shake processing. For the images displayed by other devices included in the field of view image, the position of the displayed images in the 3D mobile space can be adjusted according to the motion state of these other devices, so that the image remains substantially unchanged relative to the user-set area. The user-set area can be the user's eyes.
[0128] In some implementations of the above step S104, when the display image is controlled to be displayed within the set area of the 3D motion space, the user may also want to review the display image. To this end, review control can be performed through the following steps S1041 to S1044 shown in FIG3 .
[0129] Step S1041: Obtain the position according to the wearing part.
[0130] Step S1042: Based on the position, determine whether the user intends to review the displayed image; if so, go to step S1043; if not, go to step S1044.
[0131] Step S1043: The display image is re-controlled to follow the movement of the wearing part in the 3D moving space so that the display image is always within the user's field of view determined by the position of the wearing part. In other words, the augmented reality display device is switched from the multi-degree-of-freedom control mode to the zero-degree-of-freedom control mode.
[0132] Step S1044: Continue controlling the display image to be displayed within the set area of the 3D motion space. In other words, control the augmented reality display device to continue to be in the multi-degree-of-freedom control mode.
[0133] Based on the method described in steps S1041 to S1044 above, it is possible to promptly determine whether the user wants to watch the display screen again, and selectively switch the control mode of the display screen based on the determination result, thereby further improving the user's viewing experience of the display screen.
[0134] The above step S1042 is further explained below.
[0135] In some implementations of step S1042 above, the current user field of view can be determined based on the position of the wearing part, or information about the relative movement between the wearing part and the removable carrier can be obtained, and then based on the user field of view or information, whether the user intends to review the displayed image can be determined. The following describes these two determination methods.
[0136] (1) Determine whether the user intends to look back at the displayed image based on the user's field of view.
[0137] In the embodiment of the present application, after determining the current user's field of view based on the location of the wearable part, the user's intention to view the displayed image can be determined based on the corresponding field of view area in the 3D moving space and the set area. If the field of view area is the same as or similar to the set area, it indicates that the user's field of view has returned to the set area, and it can be determined that the user intends to view the displayed image again; otherwise, it is determined that the user does not want to view the displayed image again.
[0138] In some preferred embodiments, it is possible to determine whether the viewing area overlaps with the set area; if so, it is determined that the user intends to review the displayed image; if not, it is determined that the user does not intend to review the displayed image. Furthermore, when determining whether the two areas overlap, either of the following two methods can be used.
[0139] 1. The first judgment method
[0140] The degree of regional overlap between the field of view area and the set area can be obtained to determine whether the degree of regional overlap is greater than a preset overlap threshold; if so, there is overlap; otherwise, there is no overlap.
[0141] 2. The second judgment method
[0142] The degree of overlap between the field of view area and the set area can be obtained to determine whether the degree of overlap is greater than a preset overlap threshold and whether the duration of the overlap reaches a preset first duration threshold; if so, overlap occurs; otherwise, no overlap occurs. The duration refers to the duration of time that the degree of overlap exceeds the overlap threshold.
[0143] In the above two judgment methods, the overlap threshold and the first duration threshold can be a fixed threshold that applies to different users at the same time. In this regard, those skilled in the art can flexibly set the numerical value of this fixed threshold according to actual needs, and the embodiments of this application do not specifically limit this.
[0144] The overlap threshold and the first duration threshold can also be variable thresholds that are applicable to different users, that is, the thresholds are adaptively switched to the thresholds applicable to the current user for each user. In this regard, the overlap threshold and the first duration threshold can be set for each user based on their viewing habits. In some preferred embodiments, a big data training model can be used to obtain the thresholds using the trained model.
[0145] Specifically, for each user, second historical information about the current user's use of the augmented reality display device can be obtained, a second threshold model can be trained based on the second historical information, and the trained second threshold model can be used to respectively obtain a coincidence threshold and a first duration threshold that match the current user. The second historical information can include the position of the wearing part during the process of controlling the display screen to be displayed within a set area of the 3D mobile space, as well as the corresponding real-time playback result. The real-time playback result can include whether the current user has reviewed the display screen and whether the current user has not reviewed the display screen.
[0146] Based on the above implementation, the second historical information can be used to train a second threshold model specifically suitable for the current user, and the threshold value specifically suitable for the current user can be obtained through this model. It should be noted that in the embodiment of the present application, a conventional model training method in the field of machine learning technology can be used to train the second historical information to obtain the second threshold model. The embodiment of the present application does not specifically limit the model training method, as long as a model capable of obtaining the overlap threshold and the first duration threshold can be obtained.
[0147] (2) Determine whether the user intends to review the displayed image based on the relative motion information.
[0148] In the embodiments of the present application, after obtaining information about the relative movement between the wearing part and the removable carrier based on the position of the wearing part, it is possible to determine whether the user intends to view the displayed image based on this information. Based on this relative movement information, it is possible to determine whether the user's field of view has returned to the vicinity of the displayed image or to the vicinity of a set area. If so, it indicates that the user intends to view the displayed image again; otherwise, it indicates that the user does not intend to view the displayed image again.
[0149] Furthermore, when determining whether the user intends to review the displayed image based on the relative motion information, the determination may be made by any one of the following two methods.
[0150] 1. The first judgment method
[0151] Based on the above relative motion information, it can be determined whether the wearing part and the movable carrier are in a static state without relative motion; if so, it is determined that the user intends to review the display screen; if not, it is determined that the user does not intend to review the display screen.
[0152] Specifically, when there is no relative motion between the wearable portion and the removable carrier, the relative motion information is essentially zero, indicating a stationary state. However, when relative motion occurs, the information is significantly greater than zero, indicating a non-stationary state. When step S104 is executed, the two are in a non-stationary state. If the relative motion information indicates that the two have transitioned to a stationary state, this indicates that the user's field of view has returned to the vicinity of the display screen or the designated area, and it can be determined that the user intends to view the display screen again.
[0153] 2. The second judgment method
[0154] Based on the relative motion information, it can be determined whether the wearing part and the movable carrier are in a static state without relative motion and whether the duration of the static state has reached a preset second duration threshold; if so, it is determined that the user intends to review the displayed image; if not, it is determined that the user does not intend to review the displayed image. The duration refers to the duration of the static state.
[0155] In the second judgment method described above, the second duration threshold may be a fixed threshold that applies to different users. Those skilled in the art may flexibly set the value of this fixed threshold based on actual needs, and this embodiment of the present application does not impose any specific limitation thereto.
[0156] The second duration threshold can also be a variable threshold that applies to different users, that is, the threshold is adaptively switched to a threshold that is applicable to the current user for each user. In this regard, the second duration threshold can be set for each user based on their viewing habits. In some preferred embodiments, a big data training model can be used to obtain the threshold using the trained model.
[0157] Specifically, for each user, third historical information about the current user's use of the augmented reality display device can be obtained, a third threshold model can be trained based on the third historical information, and the trained third threshold model can be used to obtain a second duration threshold that matches the current user. The third historical information can include the position of the wearing part during the process of controlling the display screen within a set area of the 3D mobile space and the corresponding real-time playback result. The real-time playback result can include whether the current user has reviewed the display screen and whether the current user has not reviewed the display screen.
[0158] Based on the above implementation, the third historical information can be used to train a third threshold model specifically applicable to the current user, and the threshold value specifically applicable to the current user can be obtained through this model. It should be noted that in the embodiment of the present application, conventional model training methods in the field of machine learning technology can be used to train the third threshold model using the third historical information. The embodiment of the present application does not specifically limit the model training method, as long as a model capable of obtaining the second duration threshold can be obtained.
[0159] In some embodiments of step S104, in response to the received display exception handling instruction, the display image can be forced to be displayed within a set area, and then the display image can be controlled to move in the 3D moving space in accordance with the movement of the wearing part. The control method of this step is the same as the related method in step S103 above and will not be repeated here.
[0160] In some embodiments of step S104 above, the display image can be controlled to be displayed within a set area of the 3D mobile space based on the sensor data of the augmented reality display device and / or the sensor data of the movable carrier. Similar to the related method in step S103 above, when it is impossible to control the display image based solely on the sensor data of the augmented reality display device or the movable carrier, control can be performed based on the sensor data of both. Or, to avoid reducing the accuracy of display image control due to errors in the sensor data, control can also be performed based on the sensor data of both.
[0161] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effect of the present application, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These adjusted solutions are equivalent to the technical solutions described in this application, and therefore will also fall within the scope of protection of this application.
[0162] It will be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment of the present application can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium can include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.
[0163] Furthermore, the present application also provides an augmented reality display device.
[0164] Referring to FIG4 , in an embodiment of an augmented reality display device according to the present application, the augmented reality display device includes a storage device and a processor. The storage device can be configured to store a program for executing the display method of the augmented reality display device of the above-mentioned method embodiment, and the processor can be configured to execute the program in the storage device, which includes but is not limited to a program for executing the display method of the augmented reality display device of the above-mentioned method embodiment. For ease of explanation, only the parts related to the embodiment of the present application are shown, and the other parts can be referred to the aforementioned method embodiment. In some embodiments, the augmented reality display device is augmented reality glasses, eye masks, or a helmet.
[0165] In some possible implementations, the augmented reality display device may include multiple storage devices and multiple processors. The program for executing the display method of the augmented reality display device of the above-mentioned method embodiment can be divided into multiple subroutines, each of which can be loaded and run by the processor to execute different steps of the above-mentioned method. Specifically, each subroutine can be stored in different storage devices respectively, and each processor can be configured to execute the programs in one or more storage devices to jointly implement the display method of the augmented reality display device of the above-mentioned method embodiment, that is, each processor executes different steps of the above-mentioned method respectively to jointly implement the above-mentioned method.
[0166] Furthermore, the present application also provides a vehicle.
[0167] In an embodiment of a vehicle according to the present application, the vehicle may include a storage device and a processor, the storage device may be configured to store a program for executing the display method of the augmented reality display device of the above-mentioned method embodiment, and the processor may be configured to execute the program in the storage device, which includes but is not limited to a program for executing the display method of the augmented reality display device of the above-mentioned method embodiment. For ease of explanation, only the parts related to the embodiment of the present application are shown, and the other parts can be referred to the aforementioned method embodiment. In some embodiments, the vehicle may be an autonomous driving vehicle, an unmanned vehicle, or other vehicles. In addition, according to the type of power source, the vehicle in this embodiment may be a fuel vehicle, an electric vehicle, a hybrid vehicle that mixes electricity and fuel, or a vehicle that uses other new energy sources, etc.
[0168] In some possible implementations, a vehicle may include multiple storage devices and multiple processors. The program for executing the display method of the augmented reality display device of the above-described method embodiment may be divided into multiple subroutines, each of which may be loaded and executed by a processor to execute different steps of the above-described method. Specifically, each subroutine may be stored in a different storage device, and each processor may be configured to execute the programs in one or more storage devices to jointly implement the display method of the augmented reality display device of the above-described method embodiment. That is, each processor executes different steps of the above-described method to jointly implement the above-described method.
[0169] Furthermore, the present application also provides a mobile terminal.
[0170] In an embodiment of a mobile terminal according to the present application, the mobile terminal may include a storage device and a processor, the storage device may be configured to store a program for executing the display method of the augmented reality display device of the above-mentioned method embodiment, and the processor may be configured to execute the program in the storage device, which includes but is not limited to a program for executing the display method of the augmented reality display device of the above-mentioned method embodiment. For ease of explanation, only the parts related to the embodiment of the present application are shown, and the other parts can be referred to the aforementioned method embodiment. In some embodiments, the mobile terminal may be a mobile phone, a tablet computer, a smart watch, etc.
[0171] In some possible implementations, the mobile terminal may include multiple storage devices and multiple processors. The program for executing the display method of the augmented reality display device of the above-mentioned method embodiment can be divided into multiple subroutines, each of which can be loaded and run by the processor to execute different steps of the above-mentioned method. Specifically, each subroutine can be stored in different storage devices respectively, and each processor can be configured to execute the programs in one or more storage devices to jointly implement the display method of the augmented reality display device of the above-mentioned method embodiment, that is, each processor executes different steps of the above-mentioned method respectively to jointly implement the above-mentioned method.
[0172] Furthermore, the present application also provides a computer-readable storage medium.
[0173] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium can be configured to store a program for executing the display method of the augmented reality display device of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the display method of the above-mentioned augmented reality display device. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-transitory computer-readable storage medium.
[0174] Thus far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present application.
Claims
1. A display method of an augmented reality display device, characterized in that: The method comprises: When the user wears the augmented reality display device and is in the 3D moving space corresponding to the movable carrier, judging whether the user intends to watch the display screen of the augmented reality display device in the 3D moving space according to the information of the relative movement between the wearing part and the movable carrier; If yes, controlling the display image to move in the 3D moving space following the movement of the wearing part, so that the display image is always located within the user's field of view determined by the position of the wearing part; If not, the display image is controlled to be displayed in a set area of the 3D mobile space, where the set area is an area corresponding to the user's visual field determined by the position of the wearing part when no relative movement occurs between the wearing part and the movable carrier.
2. The method according to claim 1, characterized in that The step of “determining whether the user intends to view the display screen of the augmented reality display device in the 3D mobile space” specifically includes: According to the relative motion information, the relative displacement generated during the relative translation motion and / or the relative rotation amplitude generated during the relative rotation motion between the wearing part and the movable carrier along the spatial coordinate axis are obtained; Whether the user intends to view the display image is determined according to the relative displacement and / or the relative rotation amplitude.
3. The method according to claim 2, characterized in that The step of “determining whether the user intends to view the display image according to the relative displacement and / or the relative rotation amplitude” specifically includes: Obtaining preset amplitude thresholds and / or displacement thresholds corresponding to each spatial coordinate axis; Determining whether the relative rotation amplitude of at least one spatial coordinate axis is greater than a corresponding amplitude threshold and / or whether the relative displacement of at least one spatial coordinate axis is greater than a corresponding displacement threshold; If so, determining that the user intends to view the display screen; If not, it is determined that the user does not intend to view the display image.
4. The method according to claim 3, characterized in that The preset amplitude threshold and displacement threshold corresponding to each spatial coordinate axis are obtained by the following methods: Acquire first historical information of the current user using the augmented reality display device; Training a first threshold model according to the first historical information, and using the trained first threshold model to obtain an amplitude threshold and a displacement threshold corresponding to each spatial coordinate axis matched with the current user; The first historical information includes information about the relative movement between the wearing part and the movable carrier and corresponding real viewing results, wherein the real viewing results include whether the current user has viewed the display screen and whether the current user has not viewed the display screen.
5. The method according to claim 1, characterized in that In the process of controlling the display image to move in the 3D moving space following the movement of the wearing part, the method further includes: performing anti-shake processing on the display image; and / or, In the process of controlling the display image to be displayed in the set area of the 3D mobile space, the method further includes: The visual field image within the user's visual field is anti-shake processed, and the user's visual field is determined by the position of the wearing part. The visual field image includes at least a portion of the display image and / or an image displayed in the 3D mobile space by other devices other than the current augmented reality display device.
6. The method according to claim 1, characterized in that In the process of controlling the display image to be displayed in the set area of the 3D mobile space, the method further includes: Determining whether the user intends to review the display screen according to the position of the wearing part; If yes, re-control the display image to move in the 3D moving space following the movement of the wearing part, so that the display image is always located within the user's field of view determined by the position of the wearing part; If not, continue to control the display image to be displayed within the set area of the 3D mobile space.
7. The method according to claim 6, characterized in that The step of "determining whether the user intends to look back at the displayed image based on the position of the wearing part" specifically includes: Determine the current user field of view based on the location of the wearing part; According to the field of view area corresponding to the current user field of view in the 3D mobile space and the set area, it is determined whether the user intends to look back at the displayed image.
8. The method according to claim 7, characterized in that The step of "determining whether the user intends to look back at the displayed image based on the field of view area corresponding to the current user field of view in the 3D mobile space and the set area" specifically includes: Determining whether the visual field area overlaps with the set area; If so, determining that the user intends to review the displayed image; If not, it is determined that the user does not intend to review the displayed image.
9. The method according to claim 8, characterized in that The step of "determining whether the visual field area overlaps with the set area" includes: obtaining the area overlap between the visual field area and the set area, and determining whether the area overlap is greater than a preset overlap threshold; if so, determining that they overlap; if not, determining that they do not overlap; or, The step of "determining whether the field of view area overlaps with the set area" includes: determining whether the degree of overlap of the areas is greater than a preset overlap threshold and whether the duration reaches a preset first duration threshold; if so, determining that there is overlap; if not, determining that there is no overlap.
10. The method according to claim 9, characterized in that The preset overlap threshold and the preset first duration threshold are obtained respectively in the following ways: Acquire second historical information of the current user using the augmented reality display device; Training a second threshold model according to the second historical information, and using the trained second threshold model to respectively obtain a coincidence threshold and a first duration threshold that match the current user; Among them, the second historical information includes the position of the wearing part and the corresponding real review result in the process of controlling the display screen to be displayed in the set area of the 3D mobile space, and the real review result includes the current user reviewing the display screen and the current user not reviewing the display screen.
11. The method according to claim 6, characterized in that The step of "determining whether the user intends to look back at the displayed image based on the position of the wearing part" also includes: According to the position of the wearing part, information on the relative movement between the wearing part and the movable carrier is obtained; Based on the information, it is determined whether the user intends to review the displayed image.
12. The method according to claim 11, characterized in that The step of "determining whether the user intends to review the displayed image based on the information" specifically includes: determining whether the wearing part and the movable carrier are in a static state without relative motion based on the information; if so, determining that the user intends to review the displayed image; if not, determining that the user does not intend to review the displayed image; or, The step of "determining whether the user intends to review the display screen based on the information" specifically includes: determining whether the wearing part and the movable carrier are in a static state without relative movement and whether the duration reaches a preset second duration threshold based on the information; if so, determining that the user intends to review the display screen; if not, determining that the user does not intend to review the display screen.
13. The method according to claim 12, characterized in that The preset second duration threshold is obtained in the following manner: Obtain third historical information of the current user using the augmented reality display device; Training a third threshold model according to the third historical information, and using the trained third threshold model to acquire a second duration threshold matching the current user; Among them, the third historical information includes the position of the wearing part and the corresponding real review result in the process of controlling the display screen to be displayed in the set area of the 3D mobile space, and the real review result includes the current user reviewing the display screen and the current user not reviewing the display screen.
14. The method according to claim 1, characterized in that In the process of controlling the display image to move in the 3D moving space following the movement of the wearing part, and / or in the process of controlling the display image to be displayed in a set area of the 3D moving space, the method further includes: In response to the received display exception processing instruction, the display image is forcibly controlled to be displayed in the set area, The display image is then controlled to move in the 3D moving space following the movement of the wearing part.
15. The method according to claim 1, characterized in that The method further includes determining whether the user is wearing the augmented reality display device by: Determine whether first information fed back by a screen information output device used in conjunction with the augmented reality display device is received, and whether second information fed back by the augmented reality display device after detecting that the augmented reality display device is worn on the wearing part is received, wherein the first information is that the screen information output device detects that the augmented reality display device is worn on the wearing part. Generated after the display device establishes communication; If yes, it is determined that the user is wearing an augmented reality display device; If not, it is determined that the user is not wearing the augmented reality display device.
16. The method according to any one of claims 1 to 15, characterized in that The method further comprises obtaining information about the relative movement between the wearing part and the movable carrier by: obtaining the information about the relative movement according to sensor data of the augmented reality display device and / or sensor data of the movable carrier; and / or, The step of "controlling the display image to move in the 3D moving space following the movement of the wearing part" comprises: controlling the display image to move in the 3D moving space following the movement of the wearing part according to the sensor data of the augmented reality display device and / or the sensor data of the movable carrier, so that the display image is always located in the user's field of view determined by the position of the wearing part; and / or, The step of "controlling the display screen to be displayed within a set area of the 3D mobile space" specifically includes: controlling the display screen to be displayed within a set area of the 3D mobile space based on sensor data of the augmented reality display device and / or sensor data of the movable carrier.
17. An augmented reality display device, comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by the processor to execute the display method of the augmented reality display device according to any one of claims 1 to 16.
18. The augmented reality display device according to claim 17, characterized in that: The augmented reality display device is augmented reality glasses, eye mask or helmet.
19. A vehicle comprising a processor and a storage device, wherein the storage device is adapted to store a plurality of program codes, wherein: The program code is suitable for being loaded and run by the processor to execute the display method of the augmented reality display device according to any one of claims 1 to 16.
20. A mobile terminal comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by the processor to execute the display method of the augmented reality display device according to any one of claims 1 to 16.
21. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the display method of the augmented reality display device according to any one of claims 1 to 16.
Citation Information
Patent Citations
Self-adaptive screen display method and system, and head-mounted device
CN106850967A
VR device and adaptive display method thereof
CN109426345A
Image rendering method, electronic equipment and system
CN113223129A
Augmented reality display device and display method thereof, vehicle, mobile terminal and medium
CN116204068A
Augmented Reality Display Method and Head-Mounted Display Device
US20190235622A1
