Control method, device and equipment of AR head-mounted device and storage medium

Abstract

This invention provides a control method, apparatus, device, and storage medium for an AR head-mounted device, belonging to the field of AR head-mounted devices. The method includes: acquiring a target physical plane in the environment where a target user wearing the AR head-mounted device is currently located; acquiring a reference motion trajectory; and, based on the reference motion trajectory, controlling the AR head-mounted device to anchor a preset virtual object onto the target physical plane, wherein the position of the preset virtual object on the target physical plane is adjusted based on the reference motion trajectory as the target user moves. The technical solution of this invention improves the user's immersion and competitive experience during movement.

Description

Technical Field

[0001] This invention relates to the field of AR head-mounted device technology, and in particular to a control method, apparatus, device, and storage medium for an AR head-mounted device. Background Technology

[0002] The integration of Augmented Reality (AR) technology with smart wearable devices is bringing entirely new interactive experiences to scenarios such as sports and fitness, and outdoor racing. Currently, AR headsets primarily display users' exercise data and virtual objects in a two-dimensional format. This allows users to know their pace, heart rate, distance, and other exercise data in real time during exercise, while also interacting with virtual objects. However, displaying exercise data and virtual objects in a two-dimensional format on the surface of the AR headset screen makes it difficult for users to intuitively perceive the distance and orientation between themselves and the virtual objects, thus compromising the user's immersion and competitive experience. Summary of the Invention

[0003] This invention provides a control method, device, equipment, and storage medium for an AR head-mounted device, aiming to enhance the user's immersion and competitive experience during exercise.

[0004] In a first aspect, embodiments of the present invention provide a control method for an AR head-mounted device, comprising: Obtain the target physical plane of the current environment of the target user wearing the AR headset; A reference motion trajectory is obtained, and based on the reference motion trajectory, the AR head-mounted device is controlled to anchor a preset virtual object on the target physical plane. The position of the preset virtual object on the target physical plane is adjusted based on the reference motion trajectory as the target user moves.

[0005] Secondly, embodiments of the present invention also provide a control device for an AR head-mounted device, the control device comprising: The acquisition module is used to acquire the target physical plane of the current environment of the target user wearing the AR headset and to acquire the reference motion trajectory; The control module is used to control the AR head-mounted device to anchor a preset virtual object on the target physical plane according to the reference motion trajectory. The pose of the preset virtual object on the target physical plane can be adjusted based on the reference motion trajectory as the target user moves.

[0006] Thirdly, embodiments of the present invention also provide an AR head-mounted device, the AR head-mounted device including a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for implementing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements any of the AR head-mounted device control methods provided in this specification.

[0007] Fourthly, embodiments of the present invention also provide a storage medium for computer-readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement any of the AR head-mounted device control methods provided in this specification.

[0008] This invention provides a control method, apparatus, device, and storage medium for an AR head-mounted device. The invention acquires the target physical plane and reference motion trajectory of the current environment of the target user wearing the AR head-mounted device. Based on the reference motion trajectory, a virtual object is anchored on the target physical plane. Furthermore, the pose of the virtual object dynamically adjusts based on the reference motion trajectory as the user moves, allowing for an intuitive comparison between its position and movement in space and the user's real-time position. Thus, the user does not need to rely on an abstract two-dimensional data window but can directly perceive the position, distance, and movement trend of the virtual object in real space through visual perception, achieving a deep fusion of virtual and real elements and enhancing the user's immersive and competitive experience. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating a control method for an AR head-mounted device provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating another control method for an AR head-mounted device provided in an embodiment of the present invention; Figure 3 This is a flowchart illustrating another control method for an AR head-mounted device provided in an embodiment of the present invention; Figure 4 This is a flowchart illustrating another control method for an AR head-mounted device provided in an embodiment of the present invention; Figure 5 This is a schematic block diagram of the structure of a control device for an AR head-mounted device provided in an embodiment of the present invention; Figure 6 This is a schematic block diagram of the structure of an AR head-mounted device provided in an embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the described order. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0013] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0014] The integration of Augmented Reality (AR) technology with smart wearable devices is bringing entirely new interactive experiences to scenarios such as sports and fitness, and outdoor racing. Currently, AR headsets primarily display users' exercise data and virtual objects in a two-dimensional format. This allows users to know their pace, heart rate, distance, and other exercise data in real time during exercise, while also interacting with virtual objects. However, displaying exercise data and virtual objects in a two-dimensional format on the surface of the AR headset screen makes it difficult for users to intuitively perceive the distance and orientation between themselves and the virtual objects, thus compromising the user's immersion and competitive experience.

[0015] To address the aforementioned issues, this invention provides a control method for an AR head-mounted device. First, the method acquires the target physical plane and reference motion trajectory of the current environment of the target user wearing the AR head-mounted device. Then, based on the reference motion trajectory, a virtual object is anchored onto the target physical plane. Furthermore, the pose of the virtual object dynamically adjusts based on the reference motion trajectory as the user moves, allowing for an intuitive comparison between its position and movement in space and the user's real-time position. Thus, the user does not need to rely on an abstract two-dimensional data window but can directly perceive the position, distance, and movement trend of the virtual object in real space through visual perception. This achieves a deep fusion of virtual and real elements, enhancing the user's immersive experience and competitive enjoyment.

[0016] In some embodiments, the control method for the AR headset can be applied to the AR headset itself, or to a terminal device communicatively connected to the AR headset. The AR headset includes AR glasses, AR helmets, Mixed Reality (MR) glasses, and MR helmets. Mobile terminals can include mobile phones, tablets, laptops, desktop computers, personal digital assistants, and wearable devices, etc.

[0017] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0018] Please see Figure 1 , Figure 1 This is a flowchart illustrating a control method for an AR head-mounted device provided in an embodiment of the present invention.

[0019] like Figure 1 As shown, the control method of the AR head-mounted device includes steps S101 to S102.

[0020] Step S101: Obtain the target physical plane of the environment where the target user wearing the AR headset is currently located.

[0021] In this embodiment, 3D point cloud data of the environment currently occupied by the target user of the AR headset can be acquired, and plane detection can be performed based on the 3D point cloud data to obtain the target physical plane of the environment currently occupied by the target user of the AR headset. This target physical plane includes the plane containing the ground, road surface, water surface, sea surface, underwater surface, or seabed, etc. For example, the 3D point cloud data of the environment currently occupied by the target user of the AR headset can be obtained by scanning the environment using the visual sensor of the AR headset.

[0022] Step S102: Obtain the reference motion trajectory of the target user. Based on the reference pose of the target user in the reference motion trajectory, control the AR head-mounted device to anchor the preset virtual object on the target physical plane. The pose of the preset virtual object on the target physical plane is adjusted based on the reference motion trajectory as the target user moves.

[0023] This embodiment acquires the target physical plane and reference motion trajectory of the current environment of the target user wearing the AR headset. Then, based on the reference motion trajectory, the virtual object is anchored on the target physical plane, and the pose of the virtual object is dynamically adjusted according to the user's movement based on the reference motion trajectory. This allows the virtual object's position and movement in space to be intuitively compared with the user's real-time position. As a result, the user does not need to rely on an abstract two-dimensional data floating window, but can directly perceive the position, distance and movement trend of the virtual object in the real space through visual perception. This achieves a deep integration of virtual and reality, enhancing the user's immersion and competitive experience.

[0024] In some embodiments, controlling the AR headset to anchor a preset virtual object onto a target physical plane based on the target user's reference pose in a reference motion trajectory includes: obtaining the mapped position of the target user's reference pose in the physical environment; creating a spatial anchor point at the mapped position; and attaching the preset virtual object to the spatial anchor point, such that the preset virtual object is positioned on the target physical plane with the orientation corresponding to the reference pose. The target user's reference pose in the reference motion trajectory includes the pose of the target user at the starting point of the reference motion trajectory or the pose corresponding to the target user at the current moment in the reference motion trajectory.

[0025] In some embodiments, obtaining the mapped position of the reference pose in the physical environment includes: converting the geographic coordinates of the reference pose into three-dimensional coordinates in the world coordinate system; projecting the three-dimensional coordinates vertically onto the target physical plane to obtain a projection point; and determining the projection point as the mapped position.

[0026] In some embodiments, the reference motion trajectory includes running trajectories, swimming trajectories, cycling trajectories, skiing trajectories, roller skating trajectories, etc. The type of reference motion trajectory determines the preset virtual object anchored to the target physical plane. For example, if the reference motion trajectory is a running trajectory, the preset virtual object anchored to the target physical plane is a silhouette of a person in a running posture; or, if the reference motion trajectory is a swimming trajectory, the preset virtual object anchored to the target physical plane is a virtual swimming fish or a glowing spot, etc.

[0027] In some embodiments, the reference motion trajectory includes the target user's historical motion trajectory or the historical motion trajectory of a reference person. The reference person is someone other than the target user. For example, the reference motion trajectory includes the historical motion trajectory of an athlete or a celebrity. When the reference motion trajectory is the target user's historical motion trajectory, the target user races against themselves, allowing them to better understand whether their current racing performance is better or worse than before. When the reference motion trajectory is the historical motion trajectory of a reference person, the target user races against other people, allowing them to better understand whether their racing performance is better or worse than others.

[0028] In some embodiments, obtaining a reference motion trajectory includes: acquiring historical motion data of a target user or reference person from a server, and using an interpolation algorithm to convert the historical motion data into a continuous motion trajectory to obtain the reference motion trajectory. The interpolation algorithm may include spline interpolation, Bézier curve interpolation, or linear interpolation, etc. The historical motion data includes timestamps, latitude and longitude, altitude, attitude, and speed, etc.

[0029] In some embodiments, obtaining historical motion data of a target user from a server includes: obtaining the target user's current location, and obtaining historical motion data of the target user or a reference person associated with the current location from the server. This embodiment can automatically obtain associated historical motion data based on the target user's current location, eliminating the need for manual acquisition by the target user, thus improving intelligence and reducing data transmission between the server and the terminal, thereby reducing network latency and terminal power consumption.

[0030] In some embodiments, obtaining a reference motion trajectory includes: obtaining the historical motion trajectory of a target user or reference person from a server. For example, obtaining the current location of the target user and obtaining the historical motion trajectory of the target user or reference person associated with the current location from the server.

[0031] In some embodiments, such as Figure 2 As shown, after step S102, the following steps are also included: Step S103: During the movement of the target user, in response to the detection that an obstacle is located between the target user and the preset virtual object, adjust the visual presentation of the preset virtual object to present the visual effect that the preset virtual object is partially or completely obscured by the obstacle.

[0032] This embodiment simulates a physical occlusion mechanism. When a real obstacle is located between the user and the virtual object, the visual presentation of the virtual object is automatically adjusted to present an occlusion effect. This not only greatly enhances the realism of the virtual-real fusion, allowing the virtual object to truly "integrate" into the environment, but also provides the user with strong depth cues, enabling them to instinctively perceive the relative relationship between the road conditions and the preset virtual objects. Thus, under the premise of ensuring safety during movement, it achieves the unity of information continuity and immersive experience.

[0033] In some embodiments, adjusting the visual presentation of a preset virtual object to present the visual effect of the preset virtual object being partially or completely obscured by an obstacle includes: hiding the preset virtual object or setting the transparency of the preset virtual object to a preset transparency. The preset transparency can be set based on actual conditions, and this embodiment of the invention does not impose specific limitations on it. For example, the preset transparency may be 85%, 90%, 95%, or 100%, etc.

[0034] In some embodiments, the control method provided in this application further includes: acquiring a depth image of the environment where the target user is currently located through a depth sensor in an AR head-mounted device; spatially aligning a preset virtual object with the depth image to determine the pixel region corresponding to the preset virtual object in the depth image; extracting the measured depth value of the pixel region from the depth image to obtain a depth value set; determining whether there is a depth value in the depth value set that is less than the distance from the preset virtual object to the target user; if so, determining that the real object corresponding to the depth value is an obstacle, and that the obstacle is located between the user and the preset virtual object.

[0035] In some embodiments, such as Figure 3 As shown, after step S102, the following steps are also included: Step S104: During the movement of the target user, the first position vector of the target user and the second position vector of the preset virtual object are acquired in real time. Based on the first position vector and the second position vector, the relative positional relationship between the target user and the preset virtual object is determined.

[0036] In this embodiment, the relative positional relationship between the target user and the preset virtual object includes the target user being in front of the preset virtual object or the target user being behind the preset virtual object. For example, based on the first position vector and the second position vector, a unit direction vector is determined from the target user's position to the preset virtual object's position. When the unit direction vector points behind the target user and the rate of change of the relative distance is positive, it is determined that the target user is in front of the preset virtual object; when the unit direction vector points in front of the target user and the rate of change of the relative distance is negative, it is determined that the target user is behind the preset virtual object.

[0037] Step S105: Based on the relative positional relationship, control the AR head-mounted device to output corresponding motion guidance information, which is used to guide the target user to perform movements.

[0038] This embodiment calculates the relative positional relationship between the user and the virtual object in real time and outputs corresponding motion guidance information, transforming discrete motion data into dynamic and personalized interactive guidance. This not only enhances the immersive competitive experience of "chasing and chasing" but also effectively solves the safety hazard of "interactive distraction" by freeing up visual attention.

[0039] In some embodiments, controlling the AR headset to output corresponding motion guidance information based on the relative positional relationship includes: in response to determining that the target user is behind a preset virtual object based on the relative positional relationship, controlling the AR headset to display the preset virtual object in a first visual presentation mode; in response to determining that the target user is ahead of the preset virtual object based on the relative positional relationship, controlling the AR headset to display the preset virtual object in a second visual presentation mode, wherein the first visual presentation mode and the second visual presentation mode are different. This embodiment transforms the abstract "leading / lagging" state into intuitive visual symbols, achieving zero-cognitive-load state perception, immersive emotional projection, and interference-free progress prompts, resulting in a better user experience.

[0040] In some embodiments, the visual appeal of the first visual presentation method is stronger than that of the second visual presentation method. This embodiment endows the preset virtual object with emotional expression, greatly enhancing the immersion of the racing experience. For example, when the target user lags behind the preset virtual object, the preset virtual object presents a "pressure" visual (such as red light, highlighting, and solidification), visually reinforcing its presence as a target to catch up with and stimulating the target user's desire to catch up. When the target user leads the preset virtual object, the preset virtual object presents a "weakened" visual (such as gray, transparent, and blurred), visually implying that it has been surpassed, giving the user a sense of accomplishment and positive motivation.

[0041] In some embodiments, the control method provided in this application further includes: while displaying a preset virtual object in a first visual presentation mode, playing a first motion guidance audio in front of the target user; while displaying the preset virtual object in a second visual presentation mode, playing a second motion guidance audio to the side or rear of the target user, wherein the first motion guidance audio and the second motion guidance audio are different. This embodiment creates a fully immersive virtual racing space for the user through the deep integration of the visual and auditory channels, which not only greatly enhances the immersion and emotional tension of the racing scene, but also improves the reliability of information transmission through the redundant design of the auditory channel, allowing the user to perceive the competitive situation "instinctively" in any environment.

[0042] For example, the differences between the two types of audio content (such as the rhythm of footsteps, the intensity of breathing sounds, and the content of encouraging voices), combined with their spatial orientation, jointly construct a dynamic emotional narrative. For instance, when the target user is behind the preset virtual object, the first motion guidance audio could be rapid footsteps and strong breathing sounds, conveying a sense of pressure that "the opponent is strong and needs to be caught up," thus inspiring fighting spirit. When the target user is ahead of the preset virtual object, the second motion guidance audio could be a soothing rhythm and encouraging voices, conveying positive motivation that "you are doing well, keep it up."

[0043] In some embodiments, such as Figure 4 As shown, after step S102, the following steps are also included: Step S106: During the target user's movement, obtain the target user's eye gaze area within the field of view of the AR head-mounted device.

[0044] In this embodiment, the eye-tracking sensor in the AR headset can determine the target user's eye gaze area within the AR headset's field of view. Specifically, the target user's eye gaze area within the AR headset's field of view changes with the movement of the target user's eyes or head.

[0045] In some embodiments, obtaining the eye gaze area of ​​a target user within the field of view of an AR headset includes: acquiring eye feature data of the target user in real time using an eye-tracking sensor; estimating the user's gaze direction based on the eye feature data; determining the user's gaze point position within the field of view based on the spatial correspondence between the gaze direction and the optical display system of the AR headset; and determining the eye gaze area based on the gaze point position.

[0046] In some embodiments, obtaining the eye gaze area of ​​a target user within the field of view of an AR headset includes: projecting infrared light onto the user's eyes through a near-infrared illuminator in the AR headset, and acquiring an eye image containing pupil and corneal reflection spots in real time through an infrared image sensor in the AR headset; extracting the pupil center position and at least one corneal reflection spot position from the eye image, and calculating a pupil-corneal reflection vector; estimating the user's gaze direction based on the pupil-corneal reflection vector and a preset eye optical model; spatially aligning the gaze direction with the optical display system of the AR headset to determine the gaze point coordinates on the field of view plane; and identifying the eye gaze area using a clustering algorithm based on multiple gaze point coordinates within a preset time period.

[0047] Step S107: In response to the overlap between the display area of ​​the preset virtual object and the eye's gaze area, adjust the display parameters of the preset virtual object so that the preset virtual object after adjusting the display parameters does not obstruct the eye's gaze area.

[0048] In this embodiment, adjusting the display parameters of the preset virtual object includes adjusting the transparency of the preset virtual object to a preset transparency, so that the preset virtual object after adjusting the transparency does not obstruct the eye's gaze area. Adjusting the display parameters of the preset virtual object also includes hiding the preset virtual object, so that the hidden preset virtual object does not obstruct the eye's gaze area.

[0049] Step S108: In response to the overlap between the target user's motion data displayed on the AR head-mounted device and the eye's gaze area, adjust the display parameters of the motion data so that the adjusted motion data does not obscure the eye's gaze area.

[0050] This embodiment fundamentally solves the safety hazard of information occlusion in the core field of vision in AR racing scenarios by using eye-gazing area perception and dynamic avoidance mechanism, realizing an intelligent interactive experience of "information following the eye", ensuring the core field of vision is absolutely unobstructed while ensuring information visibility.

[0051] In some embodiments, adjusting the display parameters of motion data includes: displaying the target user's motion data within a target area of ​​the AR headset, wherein the target area does not overlap with the eye's gaze area, so that the motion data of the target user displayed within the target area does not obscure the eye's gaze area. Adjusting the display parameters of motion data includes: adjusting the transparency of the target user's motion data to a preset transparency, so that the motion data of the target user after adjusting the transparency does not obscure the eye's gaze area. Adjusting the display parameters of motion data includes: hiding the target user's motion data, so that the hidden motion data does not obscure the eye's gaze area. Adjusting the display parameters of motion data includes: reducing the display size of the motion data, so that the reduced display size of the motion data does not obscure the eye's gaze area.

[0052] In some embodiments, the control method provided in this application further includes: during the movement of the target user, in response to the target user's speed being greater than or equal to a preset speed, setting the transparency of the preset virtual object and / or the target user's motion data to a first transparency; in response to the target user's speed being less than the preset speed, setting the transparency of the preset virtual object and / or the target user's motion data to a second transparency, wherein the first transparency is greater than the second transparency.

[0053] In some embodiments, after step S102, the method further includes: during the movement of the target user, in response to the target user surpassing a preset virtual object, determining the position where the target user surpasses the preset virtual object as the target position; anchoring a preset surpassing marker on the target physical plane at the target position, wherein the position of the preset surpassing marker on the target physical plane does not change with the movement of the target user. In this embodiment, the surpassing position is anchored as a permanently stored virtual marker the instant the user surpasses the preset virtual object, making each self-breakthrough a spatial imprint that can be repeatedly recalled, thus improving the user experience.

[0054] Taking an outdoor running scenario as an example: During the target user's run, the AR glasses use visual sensors to identify park paths, roadside trees, and steps ahead, locking the road surface as the target physical plane. The AR glasses retrieve the target user's running track in the park from the previous week, recording it as the running track. Based on this running track, a semi-transparent preset virtual object (which could be the target user's silhouette) is anchored on the target physical plane. When the target user goes uphill, the preset virtual object also tilts its body and reduces its stride with the slope, visually appearing to always be precisely "stepping" on the target physical plane. When a real pedestrian (or obstacle) appears in front of the target user, and the target user's gaze is detected focusing on the pedestrian, the preset virtual object, which was originally overlapping the pedestrian's position, instantly becomes transparent, ensuring that the target user will not bump into people while looking at data. As the target user's speed increases, the heart rate and pace text automatically slides from the center of the field of vision to the lower right corner of the AR glasses. When the target user falls behind the preset virtual object, the preset virtual object emits a faint red light, and its breathing sound is heard through the headphones directly in front of the target user; when the target user accelerates and overtakes, the preset virtual object changes color to grayish-white and quickly retreats. At the instant the target user overtakes, a horizontal "virtual finish line" will light up on the target physical plane, and a preset overtaking mark (which may include a gold mark) will fall under the target user's feet.

[0055] Taking swimming as an example, AR swimming goggles identify the black marker line (T-shaped line) at the bottom of the pool and the pool wall at the end of the pool to establish an underwater coordinate system, using the black marker line at the bottom of the pool as the target physical plane. Based on the target user's fastest 50-meter freestyle swim data, the AR swimming goggles anchor a preset virtual object (including a "glowing spot" or "virtual fish") to the black marker line at the bottom of the pool. Regardless of how the target user moves their head to breathe, the preset virtual object remains locked on the black marker line at the bottom of the pool and moves with the target user. Because the field of vision is limited while swimming and the timing of turning needs to be judged, when the target user's head approaches the pool wall (about 2 meters away), the motion data of the preset virtual object and the target user immediately disappears, leaving the entire field of vision to the pool wall to prevent head collisions during turns. If the target user is ahead of the preset virtual object, the side frame of the AR swimming goggles will light up with a faint green light, prompting the target user to maintain their pace. If the target user is behind the preset virtual object, the preset virtual object will flash in front of the target user's line of sight, like bait to entice the target user to speed up. When the target user touches the pool wall and turns around before the preset virtual object, a ripple effect will appear at the bottom of the pool, indicating that the target user has won the current round.

[0056] Please see Figure 5 , Figure 5 This is a schematic block diagram of the structure of a control device for an AR head-mounted device provided in an embodiment of the present invention.

[0057] like Figure 5 As shown, the control device 110 of the AR headset includes: The acquisition module 111 is used to acquire the target physical plane of the current environment of the target user wearing the AR headset and to acquire the reference motion trajectory; Control module 112 is used to control the AR head-mounted device to anchor a preset virtual object on the target physical plane according to the reference motion trajectory. The pose of the preset virtual object on the target physical plane is adjusted based on the reference motion trajectory as the target user moves.

[0058] In some embodiments, the control device 110 for the AR headset further includes: An adjustment module is used to adjust the visual presentation of the preset virtual object in response to the detection of an obstacle between the target user and the preset virtual object during the target user's movement, so as to present the visual effect that the preset virtual object is partially or completely obscured by the obstacle.

[0059] In some embodiments, the control device 110 for the AR headset further includes: The acquisition module 111 is also used to acquire, in real time, the first position vector of the target user and the second position vector of the preset virtual object during the movement of the target user; The determining module is used to determine the relative positional relationship between the target user and the preset virtual object based on the first position vector and the second position vector; The control module 112 is further configured to control the AR head-mounted device to output corresponding motion guidance information based on the relative positional relationship, the motion guidance information being used to guide the target user to perform exercise.

[0060] In some embodiments, the control module 112 is further configured to control the AR head-mounted device to display the preset virtual object in a first visual presentation mode in response to determining, based on the relative positional relationship, that the target user is behind the preset virtual object; The control module 112 is further configured to, in response to determining that the target user is ahead of the preset virtual object based on the relative positional relationship, control the AR head-mounted device to display the preset virtual object in a second visual presentation mode, wherein the first visual presentation mode is different from the second visual presentation mode.

[0061] In some embodiments, the control device 110 for the AR headset further includes: The playback module is used to play a first motion guidance audio in front of the target user while displaying the preset virtual object in the first visual presentation mode; The playback module is further configured to play a second motion-guided audio at the side or rear of the target user while displaying the preset virtual object in the second visual presentation mode, wherein the first motion-guided audio is different from the second motion-guided audio.

[0062] In some embodiments, the control device 110 for the AR headset further includes: The determination module is used to determine the position where the target user passes the preset virtual object as the target position during the movement of the target user, in response to the target user passing the preset virtual object. The control module 112 is further configured to anchor a preset overtaking marker at the target position on the target physical plane, wherein the position of the preset overtaking marker on the target physical plane does not change with the movement of the target user.

[0063] In some embodiments, the control device 110 for the AR headset further includes: The acquisition module 111 is also used to acquire the eye gaze area of ​​the target user within the field of view of the AR head-mounted device during the target user's movement. The adjustment module is further configured to, in response to the overlap between the display area of ​​the preset virtual object and the eye-gazing area, adjust the display parameters of the preset virtual object so that the preset virtual object, after adjusting the display parameters, does not obstruct the eye-gazing area; and / or The adjustment module is further configured to adjust the display parameters of the motion data in response to the overlap between the motion data of the target user displayed on the AR head-mounted device and the eye gaze area, so that the adjusted motion data does not obscure the eye gaze area.

[0064] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the control device of the AR head-mounted device described above can be referred to the corresponding process in the aforementioned embodiment of the control method for the AR head-mounted device, and will not be repeated here.

[0065] Please see Figure 6 , Figure 6 This is a schematic block diagram of the structure of an AR head-mounted device provided in an embodiment of the present invention.

[0066] like Figure 6 As shown, the AR head-mounted device 100 includes a processor 101 and a memory 102, which are connected via a bus 103, such as an I2C (Inter-integrated Circuit) bus.

[0067] Specifically, processor 101 provides computing and control capabilities to support the operation of the entire AR headset. Processor 101 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0068] Specifically, the memory 102 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0069] Those skilled in the art will understand that Figure 6The structure shown is merely a block diagram of a portion of the structure related to the embodiments of the present invention, and does not constitute a limitation on the AR head-mounted device to which the embodiments of the present invention are applied. A specific AR head-mounted device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0070] The processor 101 is used to run a computer program stored in the memory 102, and to implement any of the control methods for the AR head-mounted device provided in the embodiments of the present invention when executing the computer program.

[0071] In one embodiment, the processor 101 is configured to run a computer program stored in a memory, and when executing the computer program, to perform the following steps: Obtain the target physical plane of the current environment of the target user wearing the AR headset; A reference motion trajectory is obtained, and based on the reference motion trajectory, the AR head-mounted device is controlled to anchor a preset virtual object on the target physical plane. The position of the preset virtual object on the target physical plane is adjusted based on the reference motion trajectory as the target user moves.

[0072] In some embodiments, after implementing the step of controlling the AR headset to anchor a preset virtual object onto the target physical plane according to the reference motion trajectory, the processor 101 is further configured to implement: During the movement of the target user, in response to the detection of an obstacle between the target user and the preset virtual object, the visual presentation of the preset virtual object is adjusted to present the visual effect that the preset virtual object is partially or completely obscured by the obstacle.

[0073] In some embodiments, after implementing the step of controlling the AR headset to anchor a preset virtual object onto the target physical plane according to the reference motion trajectory, the processor 101 is further configured to implement: During the movement of the target user, the first position vector of the target user and the second position vector of the preset virtual object are acquired in real time, and the relative positional relationship between the target user and the preset virtual object is determined based on the first position vector and the second position vector. Based on the relative positional relationship, the AR head-mounted device is controlled to output corresponding motion guidance information, which is used to guide the target user to perform exercise.

[0074] In some embodiments, when the processor 101 controls the AR head-mounted device to output corresponding motion guidance information based on the relative positional relationship, it is configured to: In response to determining, based on the relative positional relationship, that the target user is behind the preset virtual object, the AR headset is controlled to display the preset virtual object in a first visual presentation mode; In response to determining that the target user is ahead of the preset virtual object based on the relative positional relationship, the AR head-mounted device is controlled to display the preset virtual object in a second visual presentation mode, wherein the first visual presentation mode is different from the second visual presentation mode.

[0075] In some embodiments, the processor 101 is further configured to implement: While displaying the preset virtual object in the first visual presentation mode, a first motion guidance audio is played in front of the target user; While displaying the preset virtual object in the second visual presentation mode, a second motion guidance audio is played to the side or behind the target user. The first motion guidance audio is different from the second motion guidance audio.

[0076] In some embodiments, after the processor 101 controls the AR headset to anchor a preset virtual object onto the target physical plane according to the reference motion trajectory, it is further configured to: During the movement of the target user, in response to the target user passing the preset virtual object, the position where the target user passes the preset virtual object is determined as the target position; A preset overtaking marker is anchored at the target position on the target physical plane, and the position of the preset overtaking marker on the target physical plane does not change with the movement of the target user.

[0077] In some embodiments, after the processor 101 controls the AR headset to anchor a preset virtual object onto the target physical plane according to the reference motion trajectory, it is further configured to: During the movement of the target user, the eye gaze area of ​​the target user within the field of view of the AR headset is acquired; In response to the overlap between the display area of ​​the preset virtual object and the eye's gaze area, the display parameters of the preset virtual object are adjusted so that the preset virtual object, after adjusting the display parameters, does not obstruct the eye's gaze area; and / or In response to the overlap between the motion data of the target user displayed on the AR headset and the eye gaze area, the display parameters of the motion data are adjusted so that the adjusted motion data does not obscure the eye gaze area.

[0078] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the AR head-mounted device described above can be referred to the corresponding process in the aforementioned control method embodiment of the AR head-mounted device, and will not be repeated here.

[0079] This invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement any of the AR head-mounted device control methods provided in the specification of this invention.

[0080] The storage medium can be volatile or non-volatile. It can be an internal storage unit of the AR headset described in the foregoing embodiments, such as the hard drive or memory of the AR headset. Alternatively, it can be an external storage device of the AR headset, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the AR headset.

[0081] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware embodiments, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0082] It should be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0083] The sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The above descriptions are only specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A control method for an AR head-mounted device, characterized in that, include: Obtain the target physical plane of the current environment of the target user wearing the AR headset; A reference motion trajectory is obtained, and based on the reference motion trajectory, the AR head-mounted device is controlled to anchor a preset virtual object on the target physical plane. The position of the preset virtual object on the target physical plane is adjusted based on the reference motion trajectory as the target user moves.

2. The control method according to claim 1, characterized in that, After controlling the AR headset to anchor the preset virtual object onto the target physical plane according to the reference motion trajectory, the method further includes: During the movement of the target user, in response to the detection of an obstacle between the target user and the preset virtual object, the visual presentation of the preset virtual object is adjusted to present the visual effect that the preset virtual object is partially or completely obscured by the obstacle.

3. The control method according to claim 1, characterized in that, After controlling the AR headset to anchor the preset virtual object onto the target physical plane according to the reference motion trajectory, the method further includes: During the movement of the target user, the first position vector of the target user and the second position vector of the preset virtual object are acquired in real time, and the relative positional relationship between the target user and the preset virtual object is determined based on the first position vector and the second position vector. Based on the relative positional relationship, the AR head-mounted device is controlled to output corresponding motion guidance information, which is used to guide the target user to perform exercise.

4. The control method according to claim 3, characterized in that, The step of controlling the AR head-mounted device to output corresponding motion guidance information based on the relative positional relationship includes: In response to determining, based on the relative positional relationship, that the target user is behind the preset virtual object, the AR headset is controlled to display the preset virtual object in a first visual presentation mode; In response to determining that the target user is ahead of the preset virtual object based on the relative positional relationship, the AR head-mounted device is controlled to display the preset virtual object in a second visual presentation mode, wherein the first visual presentation mode is different from the second visual presentation mode.

5. The control method according to claim 4, characterized in that, The method further includes: While displaying the preset virtual object in the first visual presentation mode, a first motion guidance audio is played in front of the target user; While displaying the preset virtual object in the second visual presentation mode, a second motion guidance audio is played to the side or behind the target user. The first motion guidance audio is different from the second motion guidance audio.

6. The control method according to claim 1, characterized in that, After controlling the AR headset to anchor the preset virtual object onto the target physical plane according to the reference motion trajectory, the method further includes: During the movement of the target user, in response to the target user passing the preset virtual object, the position where the target user passes the preset virtual object is determined as the target position; A preset overtaking marker is anchored at the target position on the target physical plane, and the position of the preset overtaking marker on the target physical plane does not change with the movement of the target user.

7. The control method according to claim 1, characterized in that, After controlling the AR headset to anchor the preset virtual object onto the target physical plane according to the reference motion trajectory, the method further includes: During the movement of the target user, the eye gaze area of ​​the target user within the field of view of the AR headset is acquired; In response to the overlap between the display area of ​​the preset virtual object and the eye's gaze area, the display parameters of the preset virtual object are adjusted so that the preset virtual object, after adjusting the display parameters, does not obstruct the eye's gaze area; and / or In response to the overlap between the motion data of the target user displayed on the AR headset and the eye gaze area, the display parameters of the motion data are adjusted so that the adjusted motion data does not obscure the eye gaze area.

8. A control device for an AR head-mounted device, characterized in that, The control device includes: The acquisition module is used to acquire the target physical plane of the current environment of the target user wearing the AR headset and to acquire the reference motion trajectory; The control module is used to control the AR head-mounted device to anchor a preset virtual object on the target physical plane according to the reference motion trajectory. The pose of the preset virtual object on the target physical plane is adjusted based on the reference motion trajectory as the target user moves.

9. An AR head-mounted device, characterized in that, The AR head-mounted device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for enabling communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the steps of the control method as described in any one of claims 1 to 7.

10. A storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the control method according to any one of claims 1 to 7.

Images