Movement control method and device based on augmented reality space, equipment and medium

By identifying the user's hand posture and responding to gesture changes, the movement control in extended real-life space is simplified, the complexity of gesture operation is solved, and the control practicality of moving objects is improved.

CN119024952APending Publication Date: 2024-11-26BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310587095.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-23
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In the extended real-life scenario, due to the diversification of movement control instructions, there are too many gesture operations, which reduces the practicality of gesture operations.

Method used

By identifying the user's hand posture, the moving object is determined and the movement mode is determined in response to obtaining the first preset gesture, and the gesture is then changed to the second preset gesture to determine the target moving position, flexible control of the moving object is achieved.

Benefits of technology

Simplifies gesture operations and improves the practicality of control over moving objects. Users can achieve multiple movement controls without memorizing a large number of gesture actions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119024952A_ABST
    Figure CN119024952A_ABST
Patent Text Reader

Abstract

The embodiment of the invention relates to a movement control method and device based on an augmented reality space, equipment and a medium, and the method comprises the steps: determining a moving object corresponding to a current control scene in response to an obtained hand posture of a user hand as a first preset gesture, and determining a moving mode of the moving object; in response to the change of the hand posture from the first preset gesture to a second preset gesture, determining a target moving position of the moving object; and controlling the moving object to move to the target moving position. According to the embodiment of the invention, flexible movement control of the moving object can be realized through simple gesture operation, and the practicability of movement control of the moving object based on the gesture operation is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of extended reality technology, and in particular to a mobile control method, device, equipment and medium based on an extended reality space. Background Art

[0002] Extended reality (XR) refers to the combination of reality and virtuality through computers to create a virtual environment for human-computer interaction. It is also a general term for various technologies such as augmented reality (AR), virtual reality (VR), and mixed reality (MR). By integrating the visual interaction technologies of the three, it brings the experiencer an "immersive feeling" of seamless transition between the virtual world and the real world. Among them, enhancing the sense of intelligence in the extended reality scene has become a mainstream.

[0003] In related technologies, considering that gesture operations have greater operational freedom, hand tracking can obtain the user's hand posture information within a certain range, and the movement status of the hand joints and key parts can reflect the user's rich operational intentions, thereby achieving interactive operations with higher degrees of freedom. Therefore, gesture-based operations have become a common mobile control method. In the extended reality scenario, a gesture operation action is set for each mobile control instruction.

[0004] However, due to the diversity of movement control instructions, there are many corresponding gesture operation actions, so the practicality of movement control based on gesture operation is not high. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a mobile control method, device, equipment and medium based on extended real space, which realizes flexible mobile control of mobile objects through simple gesture operations, and improves the practicality of mobile control of mobile objects based on gesture operations.

[0006] An embodiment of the present disclosure provides a mobile control method based on an extended real space, the method comprising: in response to obtaining a hand posture of a user's hand as a first preset gesture, determining a moving object corresponding to a current control scene, and determining a moving mode of the moving object; in response to the hand posture changing from the first preset gesture to a second preset gesture, determining a target moving position of the moving object according to the moving mode; and controlling the moving object to move to the target moving position.

[0007] The disclosed embodiment also provides a mobile control device based on an extended real space, the device comprising: a first determination module, for determining a moving object corresponding to a current control scene in response to obtaining a hand posture of a user's hand as a first preset gesture; a second determination module, for determining a moving mode of the moving object; a third determination module, for determining a target moving position of the moving object according to the moving mode in response to the hand posture changing from the first preset gesture to the second preset gesture; and a mobile control module, for controlling the moving object to move to the target moving position.

[0008] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement a mobile control method based on an extended real space as provided in an embodiment of the present disclosure.

[0009] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the mobile control method based on the extended real space as provided in the embodiment of the present disclosure.

[0010] The embodiments of the present disclosure also provide a computer program product. When the instructions in the computer program product are executed by a processor, the mobile control method based on the extended real space provided in the embodiments of the present disclosure is implemented.

[0011] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0012] The mobile control scheme based on the extended real space provided by the embodiment of the present disclosure, in response to obtaining that the hand posture of the user's hand is a first preset gesture, determines the mobile object corresponding to the current control scene, and determines the movement mode of the mobile object, in response to the hand posture changing from the first preset gesture to the second preset gesture, determines the target movement position of the mobile object according to the movement mode, and then controls the mobile object to move to the target movement position. In the embodiment of the present disclosure, flexible movement control of the mobile object can be achieved through simple gesture operation, which improves the practicality of mobile control of the mobile object based on gesture operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0014] Figure 1A schematic diagram of an application scenario of an extended reality device provided in an embodiment of the present disclosure;

[0015] Figure 2 A flow chart of a mobile control method based on an extended real space provided in an embodiment of the present disclosure;

[0016] Figure 3 A schematic diagram of a first preset gesture provided in an embodiment of the present disclosure;

[0017] Figure 4 A schematic diagram of a second preset gesture provided in an embodiment of the present disclosure;

[0018] Figure 5 A schematic diagram of a mobile control scenario based on an extended real space provided in an embodiment of the present disclosure;

[0019] Figure 6 A schematic diagram of another mobile control scenario based on an extended real space provided by an embodiment of the present disclosure;

[0020] Figure 7 A schematic diagram of another mobile control scenario based on an extended real space provided by an embodiment of the present disclosure;

[0021] Figure 8 A schematic diagram of another mobile control scenario based on an extended real space provided by an embodiment of the present disclosure;

[0022] Fig.9A A schematic diagram of an angle indication component provided in an embodiment of the present disclosure;

[0023] Fig. 9B A schematic diagram of another angle indication component provided in an embodiment of the present disclosure;

[0024] Fig. 9C A schematic diagram of another angle indication component provided in an embodiment of the present disclosure;

[0025] Fig. 10A A schematic diagram of another mobile control scenario based on an extended real space provided by an embodiment of the present disclosure;

[0026] Fig. 10B A schematic diagram of another mobile control scenario based on an extended real space provided by an embodiment of the present disclosure;

[0027] Fig.11 A schematic diagram of the structure of a mobile control device based on an extended real space provided in an embodiment of the present disclosure;

[0028] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0032] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0034] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0035] In order to solve the above problems, the embodiment of the present disclosure provides a mobile control method based on an extended real space. Before introducing the method, some technical concepts or noun concepts involved in this article are first explained:

[0036] AR: AR scenery refers to a simulated scenery in which at least one virtual object is superimposed on a physical scenery or its representation. For example, an electronic system may have an opaque display and at least one imaging sensor, which is used to capture images or videos of physical scenery, which are representations of physical scenery. The system combines the image or video with the virtual object and displays the combination on the opaque display. Individuals use the system to indirectly view the physical scenery via the image or video of the physical scenery and observe the virtual objects superimposed on the physical scenery. When the system uses one or more image sensors to capture images of the physical scenery and uses those images to present the AR scenery on the opaque display, the displayed image is called video transmission. Alternatively, the electronic system for displaying AR scenery may have a transparent or translucent display through which individuals can directly view the physical scenery. The system can display virtual objects on a transparent or translucent display so that individuals use the system to observe virtual objects superimposed on the physical scenery. For example, the system may include a projection system that projects virtual objects into the physical scenery. Virtual objects can be projected, for example, on a physical surface or as a hologram, so that individuals using the system observe virtual objects superimposed on a physical setting. Specifically, a technology that calculates the camera's attitude parameters in the real world (or three-dimensional world, real world) in real time during the process of the camera capturing images, and adds virtual objects to the images captured by the camera based on the camera attitude parameters. Virtual objects include but are not limited to three-dimensional models. The goal of AR technology is to integrate the virtual world into the real world on the screen for interaction.

[0037] MR: MR establishes an interactive feedback information loop between the real world, the virtual world, and the user by presenting extended reality scene information in the real scene to enhance the realism of the user experience. For example, computer-created sensory input (e.g., virtual objects) is integrated with sensory input from a physical scene or its representation in a simulated scene. In some MR scenes, computer-created sensory input can adapt to changes in sensory input from the physical scene. In addition, some electronic systems used to present MR scenes can monitor the orientation and / or position relative to the physical scene so that virtual objects can interact with real objects (i.e., physical elements from the physical scene or their representations). For example, the system can monitor movement so that virtual plants appear stationary relative to physical buildings.

[0038] VR: VR is a technology for creating and experiencing a virtual world. It generates a virtual environment by calculation. It is a kind of multi-source information (the virtual reality mentioned in this article includes at least visual perception, and can also include auditory perception, tactile perception, motion perception, and even taste perception, olfactory perception, etc.), realizing the fusion of virtual environment, interactive three-dimensional dynamic vision and simulation of entity behavior, so that users can immerse themselves in a simulated virtual reality environment, and realize applications in various virtual environments such as maps, games, videos, education, medical treatment, simulation, collaborative training, sales, assisted manufacturing, maintenance and repair. Virtual reality equipment, the terminal that realizes the virtual reality effect, can usually be provided in the form of glasses, helmet-mounted displays (Head Mount Display, HMD), and contact lenses to realize visual perception and other forms of perception. Of course, the form of virtual reality equipment is not limited to this, and it can be further miniaturized or enlarged as needed.

[0039] The virtual reality devices described in the embodiments of the present disclosure may include but are not limited to the following types:

[0040] Computer-based virtual reality (PCVR) devices use the PC to perform relevant calculations and data output for virtual reality functions. External computer-based virtual reality devices use the data output by the PC to achieve virtual reality effects.

[0041] Mobile virtual reality devices support the setting of mobile terminals (such as smart phones) in various ways (such as head-mounted displays with dedicated card slots). Through wired or wireless connection with the mobile terminal, the mobile terminal performs relevant calculations of virtual reality functions and outputs data to the mobile virtual reality device, such as watching virtual reality videos through the mobile terminal's APP.

[0042] The all-in-one virtual reality device has a processor for performing relevant calculations of virtual functions, and thus has independent virtual reality input and output functions. It does not need to be connected to a PC or mobile terminal and has a high degree of freedom in use.

[0043] A virtual reality object is an object that interacts in a virtual scene, is controlled by a user or a robot program (for example, a robot program based on artificial intelligence), and can be still, move, and perform various behaviors in a virtual scene. Among them, the mobile object in this embodiment can belong to a virtual reality object.

[0044] like Figure 1As shown, the HMD is relatively light, ergonomically comfortable, and provides high-resolution content with low latency. A posture detection sensor (such as a nine-axis sensor) is provided in the virtual reality device to detect the posture changes of the virtual reality device in real time. If the user wears the virtual reality device, when the posture of the user's head changes, the real-time posture of the head will be transmitted to the processor to calculate the user's gaze point in the virtual environment, and the image in the user's gaze range (i.e., virtual field of view) in the three-dimensional model of the virtual environment is calculated based on the gaze point, and displayed on the display screen, so that people can have an immersive experience as if they were watching in the real environment.

[0045] In this embodiment, when the user wears the HMD device and opens a predetermined application, such as a live video application, the HMD device will run a corresponding virtual scene, which can be a simulated environment of the real world, a semi-simulated and semi-fictitious virtual scene, or a purely fictitious virtual scene. The virtual scene can be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The embodiment of the present disclosure does not limit the dimension of the virtual scene.

[0046] In order to solve the above technical problems, in the embodiments of the present disclosure, multiple movement controls of moving objects can be achieved through only two gesture operations. Users do not need to memorize a large number of gesture operation actions, and multiple movement controls can be achieved through simple gesture operations, which improves the practicality of controlling moving objects in the extended reality space and is of great significance to the application of gesture operations to the field of extended reality.

[0047] The method is introduced below in conjunction with specific embodiments.

[0048] Figure 2 The present invention provides a flow chart of a mobile control method based on an extended reality space, which can be performed by a mobile control device based on an extended reality space, wherein the device can be implemented by software and / or hardware, and can generally be integrated into an electronic device. Figure 2 As shown, the method includes:

[0049] Step 201, in response to obtaining that the hand posture of the user's hand is a first preset gesture, determining a moving object corresponding to the current control scene, and determining a moving mode of the moving object.

[0050] The first preset gesture can be understood as an "activation gesture" for entering a mode of controlling the movement of a moving object through gesture operation. The first preset gesture is different in different application scenarios. As a possible implementation method, the first preset gesture is as follows: Figure 3 The thumb and index finger are shown open, and the middle finger, ring finger and little finger are in a relaxed and slightly curled gesture.

[0051] In one embodiment of the present disclosure, in order to determine whether a scene in which a moving object is controlled by gesture operation is currently entered, the user's hand posture is obtained, for example, by capturing an image of the user's hand in an extended reality space, and identifying the user's hand posture based on the user's hand image; for example, the positional relationship between the user's inter-finger joints in the user's hand image is identified, and the user's hand posture is determined based on the positional relationship between the inter-finger joints.

[0052] When the hand posture is the first preset gesture, in response to obtaining the hand posture as the first preset gesture, the moving object corresponding to the current control scene is determined, wherein the moving object can be understood as any virtual object in the extended reality space. In some possible embodiments, the current control scene in the extended reality space can be identified, wherein, when the current control scene is a preset first type of control scene, the virtual character model corresponding to the user is determined as the mobile object, wherein the virtual character model is the image embodiment of the user in the extended reality space, and the first type of control scene can be a free movement scene of the virtual character model, etc. In the first type of control scene, the movement of the virtual character model can be controlled.

[0053] When the current control scene is a preset second-class control scene, the control object corresponding to the current control scene is determined to be a mobile object, wherein the control object can be any control object in the current control scene, for example, it can be a "vase" object in the current control scene, it can be a "book" object in the current control scene, etc. When there is only one preset movable object in the current control scene, the corresponding movable object is determined as the control object, and when there are multiple movable objects in the current control scene, the control object can be determined according to the user's selection, etc. Among them, the way in which the user selects the control object can be set according to the scene requirements, which are not listed here one by one.

[0054] In addition, in one embodiment of the present disclosure, taking into account that there are many forms of movement modes for controlling moving objects in an extended reality scene, such as walking, climbing, teleporting, jumping, etc., among which teleporting and continuous movement are two of the more commonly used movement modes. Therefore, in order to meet the corresponding movement needs, in one embodiment of the present disclosure, the movement mode of the moving object is also determined to facilitate the subsequent control of the movement of the moving object according to the movement mode.

[0055] It should be noted that in different application scenarios, the methods for determining the movement mode of a moving object are different, as shown in the following examples:

[0056] In some possible examples, it is determined that a historical movement mode of the mobile object in the extended real space when the control of the movement of the mobile object by gesture operation was exited last time, and the historical movement mode is determined as the current movement mode of the mobile object.

[0057] In some possible examples, a movement mode of a user-defined moving object may also be received. For example, a setting event of the movement mode may be predefined. When the user executes the setting event, multiple candidate movement mode models are displayed in the extended reality space, and the user can select a movement mode model from the multiple candidate movement mode models. The setting event of the predefined movement mode may be any binding event related to the movement mode setting. For example, multiple candidate movement mode models may be displayed for a preset control that triggers a virtual control device (such as an operating handle). By operating the virtual control device to select a movement mode model from the multiple candidate movement mode models, etc., it is determined that the movement mode corresponding to the selected movement mode model is the movement mode of the mobile object.

[0058] Step 202 : in response to the hand posture changing from the first preset gesture to the second preset gesture, determining a target moving position of the moving object according to the moving mode.

[0059] Step 203, controlling the moving object to move to the target moving position.

[0060] The second preset gesture may be a predefined hand gesture different from the first preset gesture, for example, Figure 4 As shown, the second preset gesture may be a gesture of pinching the thumb and index finger together, and curling and relaxing the other three fingers.

[0061] In one embodiment of the present disclosure, the hand gesture of the user's hand is continuously detected. When the hand gesture of the user changes from the first preset gesture to the second preset gesture, the target moving position of the moving object is determined according to the moving mode, and the moving object is further controlled to move to the target moving position. Thus, the movement control of the moving object in the corresponding moving mode can be realized through two simple gesture operations, which is simple to operate and highly practical.

[0062] It should be noted that in different application scenarios, the corresponding movement modes are different, and thus, the methods for determining the target movement position of the moving object are different according to the movement modes. Examples are as follows:

[0063] In some possible examples, the movement mode is a first preset movement mode, wherein the first preset movement mode can be understood as a fast movement mode such as “teleportation”.

[0064] In this example, in order to achieve rapid movement control of a moving object, the first hand displacement of the user's hand under a first preset gesture is obtained, and the candidate moving position of the moving object is determined based on the first hand displacement, that is, the user can select the candidate moving position through the first hand displacement of the user's hand under the first preset gesture.

[0065] In order to intuitively display the candidate moving positions corresponding to the first hand displacement, in some possible examples, a moving position indication model can also be displayed in the current control scene, and the moving position indication model is used to indicate the candidate moving positions corresponding to the first hand displacement. The style of the moving position indication model can be set according to the needs of the scene. For example, when the moving position indication model is a ray trajectory model, the candidate moving position corresponding to the current first hand displacement can be indicated by the end position of the ray trajectory model, and the trajectory angle and end point of the ray trajectory model can be adjusted by the first hand displacement, so as to realize the selection of the candidate moving position; for example, when the moving position indication model is a preset pattern model, the candidate moving position corresponding to the current first hand displacement can be indicated by displaying the display position of the preset pattern model in the extended reality space.

[0066] Furthermore, when it is determined that the hand position changes from the first preset gesture to the second preset gesture, it is considered that the movement confirmation gesture of the moving object is obtained, so that the corresponding candidate moving position is locked as the target moving position, and the moving object can be directly controlled to move to the target moving position to achieve "teleportation" control of the moving object. Of course, in order to avoid false triggering, before controlling the moving object to move to the target moving position, a release operation of the second preset gesture can also be obtained. For example, when the second preset gesture is Figure 4 As shown, the release operation of the second preset gesture can be a gesture of the thumb quickly touching the middle joint of the index finger after the thumb and index finger are released to perform a click operation, etc.; for another example, after the corresponding pinched fingers of the second preset gesture are detected to be separated, it is considered that the release operation of the second preset gesture is obtained, etc. In this embodiment, if the user performs a cancel operation of the second preset gesture operation (for example, performs other operations other than the release operation of the second preset gesture, etc.), it can be considered that the movement control is abandoned.

[0067] For example, Figure 5 As shown, in the first preset movement mode, the first preset gesture and the second preset gesture are respectively Figure 3 And Figure 4 As shown, and when the moving position indication model is a ray trajectory model, when it is detected that the hand posture of the user's hand is the first preset gesture, the ray trajectory model can be displayed with the user's hand as the starting point, wherein the user drives the starting direction and starting angle of the ray trajectory model through the first hand displacement of the user's hand under the first preset gesture to control the end point of the ray trajectory, and the end point of the ray trajectory is used to indicate the candidate moving position. When it is detected that the first preset gesture is switched to the second preset gesture, the current candidate moving position P1 is determined as the target moving position, and the moving object is moved to P1, thereby realizing the teleportation control of the moving object.

[0068] In some possible examples, the movement mode is a second preset movement mode, wherein the second preset movement mode can be understood as a movement mode such as "continuous movement", and the movement effect of the moving object "continuously moving step by step" can be achieved in the second preset movement mode.

[0069] In this example, in response to obtaining the movement operation of the user's hand in the first preset direction under the second preset gesture, wherein the first preset direction may be any preset direction in the extended real space, for example, when pointing to the outside of the extended real space is the positive direction of the Z axis, the first preset direction may be the negative direction of the Z axis, after detecting the movement operation of the user's hand in the first preset direction under the second preset gesture, the second hand displacement of the user's hand is detected, wherein the second hand displacement may be in the first preset direction or in other directions outside the first preset direction, that is, in this embodiment, the movement operation in the first preset direction is used as the entry gesture of the "continuous movement" mode, and after the entry gesture is executed, the second hand displacement may be executed in any direction, and the object movement displacement of the moving object is determined according to the second hand displacement (for example, a corresponding relationship between the second hand displacement and the object movement displacement is pre-constructed, and the object movement displacement is determined according to the object relationship), and the target moving position of the moving object is determined according to the object movement displacement.

[0070] In this embodiment, the moving object is controlled to move to the target moving position according to the moving displacement of the object, so that the moving process can intuitively reflect the moving effect of continuous movement.

[0071] For example, Figure 6 As shown, when the second preset movement mode is the continuous movement mode, the first preset gesture and the second preset gesture are respectively Figure 3 And Figure 4 As shown, in response to obtaining the second preset gesture, the user's hand moves in the first preset direction (for example, the coordinate system of the extended real space is as shown above Figure 1 As shown, when pointing to the outside of the extended space is the positive direction of the Z axis, it can be determined that the negative direction of the Z axis in the extended real space is the first preset direction, and the first preset direction is a visual forward movement operation, and the second hand displacement of the user's hand is detected. If the second hand displacement is to the positive left (i.e., the negative direction of the X axis) a, the object movement displacement corresponding to a is determined to be b, where the moving direction of b is consistent with a, and the target moving position P2 of the moving object is determined according to the object movement displacement, and the moving object is controlled to move to the corresponding P2 position according to b, thereby achieving a visual effect of continuous movement. When moving to the corresponding P2 position according to b, the picture information corresponding to each real-time position moved to is rendered to visually achieve the moving effect of following the hand movement.

[0072] Among them, when moving according to the object movement displacement, the movement parameter information of the second hand displacement can also be obtained. The movement parameter information includes movement speed, movement acceleration information, etc. When controlling the movement of the moving object, it moves according to the movement parameter information.

[0073] In addition, during the actual execution process, the user can switch the movement mode at any time. In one embodiment of the present disclosure, the user can switch the movement mode based on the movement of the user's hand in a second preset direction under a second preset gesture (wherein the second preset direction can be any operation direction different from the first preset direction. For example, when the coordinate system is as follows Figure 1 When shown, if the first preset direction is the negative direction of the Z axis pointing to the inside of the extended real space, then the vertical downward direction in the extended real space, that is, the negative direction of the Y axis can be determined as the second preset direction), the mobile mode to be switched corresponding to the mobile mode is determined, and the mobile mode is switched to the mobile mode to be switched, wherein, if there are two switching mobile modes A and B in the current scene, if A is the current mobile mode, the mobile mode to be switched is B, or multiple switching mobile modes can be sorted according to a preset sorting rule, and the next mobile mode of the current mobile mode is determined as the mobile mode to be switched according to the sorting result. Therefore, in this embodiment, under the same second preset gesture operation, operations in different directions can realize multiple adjustments to the moving object, and the user does not need to master more gesture actions.

[0074] In some possible embodiments, when the second preset switching direction is the vertical downward direction in the extended real space (negative direction of the Y axis), the user can perform the second preset gesture, such as Figure 7 As shown, moving downward realizes switching of the mobile mode, simulating the "pulling the light" scenario, in which pulling the "light rope" downward can switch the on / off state of the "light", making it easy for users to master the switching method of this mobile mode, further improving the practicality of gesture operation.

[0075] In summary, the mobile control method based on the extended real space in the embodiment of the present disclosure, in response to obtaining the hand posture of the user's hand as the first preset gesture, determines the moving object corresponding to the current control scene, and determines the moving mode of the moving object. Furthermore, in response to the hand posture changing from the first preset gesture to the second preset gesture, the target moving position of the moving object is determined according to the moving mode, and the moving object is controlled to move to the target moving position. As a result, flexible movement control of the moving object is achieved through simple gesture operation, which improves the practicality of movement control of the moving object based on gesture operation.

[0076] Based on the above embodiments, considering that when controlling the movement of a moving object, the more common movement control, in addition to the different movement modes mentioned above, also includes the display direction movement and display turning movement of the moving object, among which the display direction can be understood as the display orientation of the moving object in the extended reality space at the target moving position after the moving object moves to the target moving position; the display turning can be understood as the facing orientation of the moving object in the display space at the current display position before it moves to the target moving position, etc. When the moving object is a virtual character model, the display turning can be understood as the visual viewing orientation of the user in the extended reality space, etc.

[0077] In one embodiment of the present disclosure, when a movement operation of the user's hand in a third preset direction under the second preset gesture is obtained, the display direction of the moving object is rotated, wherein the third preset direction can be an arbitrarily defined direction different from the first preset direction and the second preset direction. For example, when the coordinate system direction in the extended real space is as above Figure 1 As shown, the third preset direction may be any direction on the ZX plane that is not parallel to the Z axis, for example, the third preset direction may be the positive direction of the X axis or the negative direction of the X axis, etc. That is, in this embodiment, when the third preset direction is the positive direction of the X axis or the negative direction of the X axis, the display direction of the moving object can be adjusted by the "pull left and right" operation under the second preset gesture.

[0078] In this example, in response to obtaining the movement operation of the user's hand in the third preset direction under the second preset gesture, the operation direction of the movement operation in the third preset direction is determined, wherein the third preset direction may include multiple preset movement directions, and the preset direction corresponding to the current movement operation is determined to be the operation direction of the movement operation in the third preset direction. The display direction of the moving object is determined according to the operation direction, and the moving object is controlled to rotate according to the display direction at the current position.

[0079] It should be noted that in different application scenarios, the way of controlling the rotation of the moving object at the current position according to the display steering is different. In some possible embodiments, the control of the moving object directly switches to the display steering, for example, displaying the corresponding screen information under the display steering, etc., that is, in this embodiment, the "quick steering" movement effect of the moving object is realized, which can meet the demand for fast control of the display steering of the moving object in the extended reality space.

[0080] In some possible embodiments, the hand position movement parameters corresponding to the movement operation of a preset third preset direction can be determined, and the hand position movement parameters include the movement distance, movement speed, etc. when the hand moves. Then, the preset correspondence relationship can be queried to obtain the angle movement parameters corresponding to the hand position parameters. The angle movement parameters can include the angle movement amount (that is, the specific angle value moved), the angle movement speed, etc., and the moving object is controlled according to the angle movement parameters to rotate from the initial turning direction corresponding to the current position to the operation direction of the movement operation of the third preset direction, thereby visually achieving the "continuous rotation" effect of the movement operation from the current position to the third preset direction.

[0081] For example, Figure 8 As shown, when a moving operation in the third preset direction is obtained and the moving object is a virtual character model, if the operation direction corresponding to the third preset direction is the negative direction of the X-axis, the moving object can be visually controlled to turn left, thereby realizing the need to adjust the displayed direction of the moving object, wherein the displayed screen information is synchronously switched to the screen information corresponding to the negative direction of the X-axis (not shown in the figure).

[0082] In one embodiment of the present disclosure, when the display direction of the moving object is moved, the user's hand is moved in a fourth preset direction under the second preset gesture, wherein the fourth preset direction may be an operation direction different from the first preset direction, the second preset direction, and the third preset direction. For example, when the first preset direction is the negative direction of the Z axis in the extended reality space, the second preset direction is the negative direction of the Y axis in the extended reality space, and the third preset direction is any direction on the ZX plane that is not parallel to the Z axis, the fourth preset direction may be the positive direction of the Y axis, etc.

[0083] In this embodiment, after obtaining the user's hand movement operation in the fourth preset direction, the target angle adjustment information corresponding to the movement operation in the fourth preset direction is obtained. Then, the target display orientation of the moving object is determined according to the target angle adjustment information, and when the moving object is controlled to move to the target moving position, the moving object can also be displayed according to the target display orientation, thereby realizing the control of the display orientation of the moving object.

[0084] In different application scenarios, the method of obtaining the target angle adjustment information corresponding to the movement operation in the fourth preset direction is different:

[0085] In some possible embodiments, the moving distance or moving duration corresponding to the moving operation in the fourth preset direction is determined, the correspondence between the moving distance and the moving angle is predefined, or the correspondence between the moving duration and the moving angle is predefined, the moving angle corresponding to the preset moving distance or moving duration is determined by querying the preset correspondence, and the angle corresponding to the moving angle rotated in the preset rotation direction by the current display angle of the moving object is determined as the target angle information.

[0086] Based on the above embodiments, considering that in many application scenarios, the movement of mobile objects is controlled based on a joystick device, and in the related art, it is necessary to memorize the operation instructions corresponding to each joystick action, and the operation is complicated. In order to solve this problem, in one embodiment of the present disclosure, the gesture movement operation in this embodiment to control the movement of the mobile object can also be mapped to the virtual joystick model corresponding to the joystick device to achieve it. Thus, there is no need to memorize the operation instructions corresponding to each joystick action, but only through simple gesture actions to operate the virtual joystick model, the movement control of the relevant mobile objects can be achieved. Compared with the joystick operation, the gesture action has a higher degree of freedom, and hand tracking can obtain the user's hand posture within a certain range. The user's hand posture can visually and intuitively present the corresponding operation instruction, and the gesture action in this embodiment is simple and the memory cost is not high.

[0087] Therefore, in one embodiment of the present disclosure, the second preset gesture can be used as a single control gesture for the virtual joystick model. In this embodiment, a call request for the virtual joystick model is obtained, wherein an image of the user's hand in the virtual reality space can be captured, and the user's hand posture can be identified based on the hand image. When the hand posture is a preset call hand posture, a call request for the virtual joystick model is obtained, or when it is detected that the above-mentioned first preset gesture changes to the second preset gesture, a call request for the virtual joystick model is obtained. Of course, in other possible implementations, other operating methods can also be used as the call operation of the virtual joystick model, which are not listed here one by one.

[0088] Furthermore, in response to obtaining a call request for a preset virtual joystick model, a virtual joystick model can be displayed in the extended reality space, wherein the virtual joystick model is used to indicate the joystick control angle, that is, in the actual execution process, the virtual joystick model can be of any style and any combination of components, as long as it can indicate the control angle of the corresponding joystick.

[0089] For example, the virtual joystick model may include an angle indicating component for indicating the control angle of the virtual joystick model, wherein the angle indicating component may be any component style that can indicate the control angle, for example, Fig.9AAs shown, the angle indicating component may be in the form of a disk, and the disk may include an indicating angle scale, and the corresponding indicating angle scale is used to indicate the corresponding indicating angle; for example, Fig. 9B As shown, the angle indication component can be in the form of a display screen, and indicates the corresponding indication angle by displaying a specific indication angle in the display screen; for example, Fig. 9C As shown, the angle indication component may be in the form of a progress bar, and the corresponding indication angle may be indicated by the progress of the progress bar.

[0090] In one embodiment of the present disclosure, the virtual joystick model may further include an angle indication component and an angle control component, wherein the angle control component is used to adjust the indicated angle of the angle indication component, and the gesture operation information under the second preset gesture operation can visually adjust the control angle indicated by the angle indication component by operating the corresponding angle control component. It can also be understood that in order to simulate the "joystick" operation experience in the real world, in this embodiment, the angle control component is used as the "joystick", and the control angle of the angle indication component is adjusted by controlling the angle control component.

[0091] In an embodiment of the present disclosure, when determining the target moving position of the moving object according to the moving mode, the preset gesture operation information of the user on the virtual joystick model under the second preset gesture is obtained, and the control angle indicated by the virtual joystick model is adjusted according to the preset gesture operation information, wherein the preset gesture operation information includes but is not limited to the specific hand displacement under the corresponding hand displacement under the second preset gesture mentioned in the above embodiment. That is, in this embodiment, the operation of the moving object under the second preset gesture operation is first mapped to the control angle of the virtual joystick model, so as to facilitate the movement control of the moving object based on the control angle.

[0092] When controlling the moving object to move to the target moving position, the angle movement parameter of the control angle indicated by the virtual joystick model is determined, wherein the angle movement parameter can be set in advance according to the scene, or can be determined according to the hand movement parameter corresponding to the preset gesture operation information, wherein the hand movement parameter includes the movement speed, etc., and the angle movement parameter can include the angle movement amount (that is, the specific angle value moved), the angle movement speed, etc., and then, the motion parameters corresponding to the moving mode are determined according to the angle movement parameters, for example, the preset database is queried to determine the motion parameters corresponding to the angle movement parameters, etc., wherein the motion parameter can be the motion amount corresponding to the moving mode (for example, the moving mode is a jumping motion type, and the corresponding motion amount is a jumping distance, etc.), motion speed, etc., and then, according to the motion parameters and the moving mode, the moving object is controlled to move to the target moving position.

[0093] For example, if the movement mode is a jumping motion type, the jumping speed and single jumping distance during the jumping motion can be determined according to the corresponding angle movement parameters, and the jumping process of the moving object can be displayed according to the jumping speed and single jumping distance until it jumps to the corresponding target moving position.

[0094] For example, if the movement mode is the "teleport" movement type and the moving object is the virtual interface information in the extended real space, the switching duration when the virtual interface information switches to the target moving position can be determined based on the corresponding angular movement parameters, and the virtual screen information can be switched to the target moving position display based on the switching duration.

[0095] It is easy to understand that the gesture operation information corresponding to the above-mentioned second preset gesture is mapped to the control angle in the virtual joystick model, which can be applied to any moving scenario based on the second preset gesture to control the moving object, including but not limited to the several common moving modes mentioned above. During the mapping process, the operation information such as the operation displacement corresponding to the second gesture operation is mapped to the control angle of the virtual joystick model using the corresponding mapping method, so as to achieve the visual effect of controlling the moving object through the virtual joystick model in the extended real space.

[0096] In addition, during the mapping process, the way in which the angle control component adjusts the control angle indicated by the angle indication component can also be set according to different scenarios. Any change method that can achieve the corresponding purpose belongs to the method protected in this embodiment. It should be emphasized that the gesture operation information corresponding to the second preset gesture operation can be mapped to the change of the angle control component, and the change of the angle control component can ultimately affect the change of the control angle indicated by the angle indication component, etc.

[0097] The following is an example of a target display scene in which a moving object is controlled to move to a target moving position under a second preset gesture, and the explanation is as follows: In some possible embodiments, in order to improve the intuitiveness of the adjustment, when a hand movement operation in a fourth preset direction under a second preset gesture is detected, the third hand displacement of the angle control component caused by the movement operation in the fourth preset direction is obtained, and the third hand displacement is used as the preset gesture operation information, and the target adjustment position of the angle control component is determined according to the third hand displacement, and the control angle indicated by the angle indication component is adjusted according to the target adjustment position. The control angle has a corresponding relationship with the target adjustment angle information corresponding to the movement operation in the fourth preset direction. Different target adjustment angle information can be mapped to different control angles.

[0098] As mentioned above, the angle indication component can be any component style that can indicate the control angle. For example, the angle indication component can be a disc style, and the disc style can include an indicating angle scale, and the corresponding indicating angle scale is used to indicate the corresponding control angle. The angle control component is used to adjust the control angle of the angle indication component. In order to simulate the "joystick" operation experience in the real world, in this embodiment, the angle control component is used as a "joystick" to adjust the control angle of the angle indication component by controlling the angle control component.

[0099] In this embodiment, if the angle indication component includes multiple candidate angle scales (in the actual execution process, each candidate angle scale may also include a specific scale value identifier, etc.), and the angle control component is a first indication trajectory component, then when adjusting the target adjustment position of the angle control component according to the third hand displacement, the real-time trajectory end point position of the first indication trajectory component can be adjusted according to the third hand displacement, and the real-time trajectory end point position is used as the target adjustment position corresponding to the third hand displacement, and the real-time trajectory end point position moves between multiple candidate angle scales.

[0100] That is, the change of the real-time trajectory end point position is controlled by the third hand displacement, that is, the first indication trajectory component is displayed according to the real-time trajectory end point position. Therefore, the control angle indicated by the angle indication component corresponding to the third hand displacement can be visually known directly according to the real-time trajectory end point position of the first indication trajectory component.

[0101] In some possible embodiments, the scale corresponding to the control angle corresponding to the third hand displacement may be further highlighted to further enhance the intuitiveness of the control angle corresponding to the target adjustment angle information. In different application scenarios, the scale corresponding to the control angle is highlighted in different ways, such as displaying a preset pattern on the scale corresponding to the control angle, or highlighting the scale corresponding to the control angle.

[0102] In different application scenarios, the styles of the first indication track component and the angle indication component may be different. For example, Fig. 10AAs shown, the angle indication component can be a "disc" component including multiple candidate angle scales, and the first indication trajectory component is a "pointer" component. The "pointer" component rotates between the multiple candidate angle scales of the "disc" component to visually and intuitively indicate the control angle corresponding to the current third hand displacement. When determining the target adjustment position of the angle control component according to the third hand displacement, the real-time trajectory end point position of the "pointer" component can be determined according to the real-time hand position of the third hand displacement. The real-time trajectory end point position indicated by the "pointer" component moves among the multiple candidate angle scales in the "disc" component. The corresponding control angle can be obtained by adjusting the display of the "pointer" according to the real-time trajectory end point position. In the figure, a "circular" pattern can also be displayed on the scale corresponding to the control angle to further intuitively indicate the control angle.

[0103] In some optional embodiments, if the angle indication component includes a real-time angle display area, the real-time angle display area always displays the real-time indication angle of the current angle indication component, and the angle control component is a second indication trajectory component, that is, the "joystick" is simulated by the second indication trajectory component, and when the target adjustment position of the angle control component is adjusted according to the real-time hand position of the third hand displacement, the real-time hand movement displacement can be determined according to the real-time hand position, and the real-time trajectory length of the second indication trajectory component is adjusted according to the real-time hand movement displacement. The real-time trajectory length can be used as the target adjustment position of the angle control component, that is, the second indication trajectory component can be visually lengthened as the third hand displacement increases, and the second indicator trajectory component can be visually lengthened. The indication trajectory component shortens as the third hand displacement decreases, etc., and further, when the indication angle of the control angle indication component is displayed as the target real-time indication angle, it can include displaying the corresponding control angle in the real-time angle display area, wherein, for example, when the second indication trajectory component lengthens as the third hand displacement increases, the control angle indicated by the angle indication component can be controlled to change in a direction of increasing the preset angle until the real-time angle display area displays the corresponding control angle; when the second indication trajectory component shortens as the third hand displacement decreases, the control angle indicated by the angle indication component can be controlled to change in a direction of decreasing the preset angle until the real-time angle display area displays the corresponding control angle.

[0104] That is, in this embodiment, the user does not need to pay attention to the real-time end point position of the indicated trajectory. Since the corresponding real-time control angle is always displayed in the real-time angle display area, the user can clearly know the current indicated real-time control angle of the virtual angle adjustment model in the real-time angle display area, which improves the convenience of knowing the real-time control angle.

[0105] The display styles of the real-time angle display area and the second indication track component are different in different application scenarios. For example, the real-time angle display area may be a display screen component displayed in a preset area of ​​the current extended reality screen.

[0106] For example, Fig. 10B As shown, the real-time angle display area can be a preset scale display area on the "disc" component, and the scale display area can be highlighted (highlighted by a larger circular pattern in the figure). The second indication track component is a straight line track model with the center of the "disc" component as the starting point. In this embodiment, the track length of the straight line track model is adjusted according to the real-time hand position of the third hand displacement. At the same time, in order to ensure that the scale display area displays the control angle, the "disc" can be rotated to display the scale value corresponding to the control angle in the corresponding scale display area. For example, when the third hand displacement increases, the length of the second indication track component becomes longer, and the "disc" is controlled to rotate clockwise to rotate to the position where the scale value corresponding to the control angle is displayed in the scale display area (refer to the above Fig. 10B ), of course, in other possible embodiments, when the third hand displacement decreases, the length of the second indication track component becomes shorter, and the "disc" is controlled to rotate counterclockwise to rotate to a position where the scale value corresponding to the control angle is displayed in the scale display area (not shown in the figure).

[0107] In this embodiment, after determining the control angle of the angle indication component according to the target adjustment position, the control angle is determined to be the target adjustment angle information, wherein when the control angle of the angle indication component is adjusted according to the target adjustment position, the moving object can also be displayed in real time in a preview form according to the target adjustment angle information corresponding to the control angle, or the moving object can be directly displayed and processed according to the target adjustment angle information after the fourth moving displacement is completed, etc. The specific display method can be calibrated by the system and can also be set by the user.

[0108] In summary, the mobile control method based on the extended real space of the embodiment of the present disclosure can realize the display steering or display orientation adjustment of the moving object based on the movement operations in different directions under the second preset gesture operation. On the basis of not increasing the complexity of the gesture action corresponding to the gesture operation, it further enriches the movement mode of the moving object and improves the practicality of the movement control of the moving object based on gesture operation.

[0109] In order to implement the above embodiments, the present disclosure also proposes a mobile control device based on an extended real space.

[0110] Fig.11This is a schematic diagram of the structure of a mobile control device based on an extended reality space provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated in an electronic device to perform mobile control based on an extended reality space. Fig.11 As shown, the device includes: a first determination module 1110, a second determination module 1120, a third determination module 1130 and a movement control module 1140, wherein:

[0111] A first determining module 1110, configured to determine a moving object corresponding to a current control scene in response to obtaining that the hand posture of the user's hand is a first preset gesture;

[0112] A second determining module 1120, configured to determine a moving mode of the moving object;

[0113] A third determining module 1130, configured to determine a target moving position of the moving object according to the moving mode in response to the hand posture changing from the first preset gesture to a second preset gesture;

[0114] The movement control module 1140 is used to control the moving object to move to the target moving position.

[0115] The mobile control device based on the extended reality space provided in the embodiments of the present disclosure can execute the mobile control method based on the extended reality space provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] In order to implement the above embodiments, the present disclosure also proposes a computer program product, including a computer program / instruction, which implements the mobile control method based on the extended real space in the above embodiments when executed by a processor.

[0117] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

[0118] The following specific reference Fig.12 , which shows a schematic diagram of the structure of an electronic device 1200 suitable for implementing the embodiment of the present disclosure. The electronic device 1200 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.12 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0119] like Fig.12As shown, the electronic device 1200 may include a processor (e.g., a central processing unit, a graphics processor, etc.) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a memory 1208 to a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the electronic device 1200 are also stored. The processor 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0120] Typically, the following devices may be connected to the I / O interface 1205: an input device 1206 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1207 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1208 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1209. The communication device 1209 may allow the electronic device 1200 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.12 The electronic device 1200 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0121] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1209, or installed from the memory 1208, or installed from the ROM 1202. When the computer program is executed by the processor 1201, the above-mentioned functions defined in the mobile control method based on the extended real space of the embodiment of the present disclosure are executed.

[0122] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0123] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0124] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0125] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0126] In response to obtaining that the hand posture of the user's hand is a first preset gesture, a moving object corresponding to the current control scene is determined, and the movement mode of the moving object is determined. Furthermore, in response to the hand posture changing from the first preset gesture to the second preset gesture, the target moving position of the moving object is determined according to the movement mode, and the moving object is controlled to move to the target moving position. Thus, flexible movement control of the moving object is achieved through simple gesture operation, thereby improving the practicality of movement control of the moving object based on gesture operation.

[0127] The electronic device may be written in one or more programming languages ​​or a combination thereof to write computer program code for performing the operations of the present disclosure, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0130] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0131] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0133] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0134] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A mobile control method based on extended real space, characterized in that: The following steps are involved: In response to obtaining that the hand posture of the user's hand is a first preset gesture, determining a moving object corresponding to the current control scene, and determining a moving mode of the moving object; In response to the hand gesture changing from the first preset gesture to a second preset gesture, determining a target moving position of the moving object according to the moving mode; The moving object is controlled to move to the target moving position.

2. The method according to claim 1, characterized in that The determining of the moving object corresponding to the current control scene includes: Identifying a current control scene in the extended reality space; When the current control scene is a preset first type of control scene, determining the virtual character model corresponding to the user as the moving object; When the current control scenario is a preset second type of control scenario, it is determined that the control object corresponding to the current control scenario is the mobile object.

3. The method according to claim 1, characterized in that Before determining the target moving position of the moving object according to the moving mode, the method further includes: In response to obtaining a call request for a preset virtual joystick model, displaying a virtual joystick model in the extended reality space, wherein the virtual joystick model is used to indicate a control angle of the joystick; When determining the target moving position of the moving object according to the moving mode, it includes: Acquire preset gesture operation information of the user on the virtual joystick model under the second preset gesture, and adjust the control angle indicated by the virtual joystick model according to the preset gesture operation information.

4. The method of claim 3, wherein: The virtual joystick model includes an angle indication component and an angle control component, wherein the angle control component is used to adjust the indication angle of the angle indication component, and the step of adjusting the indicated control angle of the virtual joystick model according to the preset gesture operation information includes: An operation displacement corresponding to the preset gesture operation information is determined, and a control angle indicated by the virtual joystick model is adjusted according to the operation displacement.

5. The method according to claim 3 or 4, characterized in that When controlling the moving object to move to the target moving position, the method further includes: Determining an angular movement parameter of a control angle indicated by the virtual joystick model; Determining a motion parameter corresponding to the movement mode according to the angular movement parameter; The moving object is controlled to move to the target moving position according to the moving mode based on the movement parameter.

6. The method according to claim 1, characterized in that When the movement mode is the first preset movement mode, determining the target movement position of the moving object according to the movement mode includes: Acquire a first hand displacement of the user's hand under the first preset gesture, and determine a candidate moving position of the moving object according to the first hand displacement; When it is determined that the hand posture changes from the first preset gesture to the second preset gesture, the candidate moving position corresponding to the first hand displacement is the target moving position.

7. The method according to claim 6, characterized in that A movement position indication model is displayed in the current control scene, wherein the movement position indication model is used to indicate a candidate movement position corresponding to the first hand displacement.

8. The method according to claim 7, characterized in that Before controlling the moving object to move to the target moving position, the method further includes: Acquire a release operation of the second preset gesture.

9. The method according to claim 1, characterized in that When the movement mode is the second preset movement mode, determining the target movement position of the moving object according to the movement mode includes: In response to acquiring the movement operation of the user's hand in the first preset direction under the second preset gesture, detecting a second hand displacement of the user's hand; determining an object movement displacement of the moving object according to the second hand displacement; The target moving position of the moving object is determined according to the object moving displacement.

10. The method according to any one of claims 6 to 9, characterized in that: Also includes: In response to obtaining the movement operation of the user's hand in a second preset direction under the second preset gesture, determining the movement mode to be switched corresponding to the movement mode; The moving mode is switched to the moving mode to be switched.

11. The method according to any one of claims 6 to 9, characterized in that: Also includes: In response to acquiring the movement operation of the user's hand in a third preset direction under the second preset gesture, determining the operation direction of the movement operation in the third preset direction; Determining the display direction of the moving object according to the operation direction; The moving object is controlled to rotate at the current position according to the displayed direction.

12. The method according to claim 4, characterized in that Also includes: In response to obtaining a movement operation of the user's hand in a fourth preset direction under the second preset gesture, obtaining target adjustment angle information corresponding to the movement operation in the fourth preset direction; determining a target display orientation of the moving object according to the target adjustment angle information; When controlling the moving object to move to the target moving position, the method further includes: The moving object is displayed according to the target display direction.

13. The method according to claim 12, characterized in that When the virtual joystick model is displayed in the extended real space, adjusting the control angle indicated by the virtual joystick model according to the preset gesture operation information includes: Acquire a third hand displacement of the angle control component caused by the movement operation in the fourth preset direction; determining a target adjustment position of the angle control component according to the third hand displacement; The control angle indicated by the angle indicating component is adjusted according to the target adjustment position.

14. The method according to claim 13, characterized in that The angle indication component includes a plurality of candidate angle scales. When the angle control component is a first indication track component, determining a target adjustment position of the angle control component according to the third hand displacement includes: Determining a real-time trajectory endpoint position of the first indication trajectory component according to the third hand displacement, wherein the real-time trajectory endpoint position moves between the multiple candidate angle scales; The step of adjusting the control angle indicated by the angle indicating component according to the target adjustment position comprises: The first indication track component is displayed according to the end point position of the real-time track.

15. The method according to claim 13, characterized in that The angle indication component includes a real-time angle display area, and the angle control component is a second indication track component. Then, determining the target adjustment position of the angle control component according to the third hand displacement includes: determining a real-time track length of the second indication track component according to the third hand displacement; The step of adjusting the control angle indicated by the angle indicating component according to the target adjustment position comprises: Determine a control angle corresponding to the real-time trajectory length; The real-time angle display area of ​​the angle indication component is controlled to display a control angle corresponding to the real-time track length.

16. A mobile control device based on an extended real space, characterized in that: include: A first determining module, configured to determine a moving object corresponding to a current control scene in response to obtaining that the hand posture of the user's hand is a first preset gesture; A second determining module, used to determine the movement mode of the moving object; a third determining module, configured to determine a target moving position of the moving object according to the moving mode in response to the hand posture changing from the first preset gesture to a second preset gesture; The movement control module is used to control the moving object to move to the target moving position.

17. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the mobile control method based on the extended real space as described in any one of claims 1-15 above.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to execute the mobile control method based on the extended real space as described in any one of claims 1-15 above.

19. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor, the mobile control method based on the extended real space as described in any one of claims 1 to 15 is implemented.