Virtual scene control method and apparatus

Abstract

The application provides a virtual scene control method and device. In the application, a real hand operation of a real user is detected, the real hand operation is mapped to a virtual hand operation of a virtual user in a virtual scene image, if a hand posture of the real hand operation belongs to a set holding posture for maintaining an object posture, voice information input by the real user is obtained, a target object described in the voice information is recognized, and a virtual target object is added to a virtual hand of the virtual user that makes the virtual hand operation.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a virtual scene control method and apparatus. Background Technology

[0002] In virtual scenes based on virtual technologies such as virtual reality or augmented reality, the input operations of real users in the physical scene can be converted into the input operations of virtual users in the virtual scene.

[0003] However, in the process of mapping real users to virtual users, only the actions and behaviors of real users can be converted into actions and behaviors related to virtual users in the virtual scene. However, it is impossible to present objects such as those held or expected by real users on the virtual user. As a result, the current virtual scene cannot accurately present the interactive scene expected by the user, and the display flexibility is poor. Summary of the Invention

[0004] On the one hand, this application provides a virtual scene control method, including:

[0005] Real hand gestures from real users are detected, and these real hand gestures are mapped to virtual hand gestures from virtual users in a virtual scene image.

[0006] If the hand gesture of the actual hand operation belongs to the set holding posture for maintaining the position of the object, obtain the voice information input by the actual user.

[0007] Identify the target object described in the voice information;

[0008] Add a virtual target object to the virtual hand of the virtual user performing the virtual hand operation.

[0009] In one possible implementation, obtaining the voice information input by the real user if the hand gesture of the actual hand operation belongs to a set holding gesture for maintaining the object's pose includes:

[0010] If the hand gesture of the actual hand operation belongs to the set holding posture for maintaining the position of the object, and the duration of the actual hand operation in the holding posture exceeds the set duration, the voice information input by the actual user is obtained.

[0011] In yet another possible implementation, identifying the target object described in the voice information includes:

[0012] Identify the object category and object characteristics of the target object described in the voice information;

[0013] Before adding the virtual target object to the virtual hand that performs the virtual hand operation in the virtual user, the method further includes:

[0014] Based on the object category and object characteristics of the target object, a virtual target object is constructed.

[0015] In another possible implementation, mapping the real hand gesture to a virtual hand gesture of a virtual user in a virtual scene image includes:

[0016] Determine the type of input gesture for the actual hand operation;

[0017] Based on the input gesture type, determine the input sensitivity of the virtual input operation mapped from the real hand operation to the virtual input operation in the virtual scene image;

[0018] Based on the input sensitivity, the real hand operation is mapped to the virtual hand operation of the virtual user in the virtual scene image.

[0019] In another possible implementation, mapping the real hand gesture to a virtual hand gesture of a virtual user in a virtual scene image according to the input sensitivity includes:

[0020] Based on the input sensitivity, determine the distance mapping relationship between the movement distance of the real hand operation and the virtual movement distance of the virtual hand in the virtual scene image;

[0021] Based on the distance mapping relationship and the movement trajectory of the real hand operation, the real hand operation is mapped to the virtual hand operation of the virtual user in the virtual scene image.

[0022] In yet another possible implementation, determining the input gesture type of the actual hand operation includes:

[0023] Determine the finger posture used to indicate the gesture type in the actual hand operation;

[0024] Based on the finger posture, the input gesture type of the actual hand operation is determined.

[0025] In yet another possible implementation, determining the finger posture used to indicate the gesture type in the actual hand operation includes:

[0026] If the actual hand operation is a hand movement operation, determine the finger posture used to indicate the gesture type in the actual hand operation.

[0027] In another possible implementation, before adding the virtual target object to the virtual hand performing the virtual hand operation in the virtual user, the method further includes:

[0028] Based on the actual hand movements of the real users, the object characteristics of the target object are determined;

[0029] Based on the object characteristics of the target object, a virtual target object is constructed.

[0030] In another aspect, this application also provides a virtual scene control device, including:

[0031] An operation mapping unit is used to detect real hand operations of a real user and map the real hand operations to virtual hand operations of a virtual user in a virtual scene image.

[0032] The voice acquisition unit is used to acquire the voice information input by the real user if the hand posture of the real hand operation belongs to a set holding posture for maintaining the position of the object.

[0033] A speech recognition unit is used to identify the target object described in the speech information;

[0034] An object adding unit is used to add a virtual target object to the virtual hand of the virtual user performing the virtual hand operation.

[0035] In one possible implementation, the operation mapping unit includes:

[0036] A type determination unit is used to detect real hand operations by real users and determine the input gesture type of the real hand operations;

[0037] A sensitivity determination unit is used to determine the input sensitivity of the virtual input operation mapped from the real hand operation to the virtual scene image based on the input gesture type.

[0038] The mapping processing unit is used to map the real hand operation to the virtual hand operation of the virtual user in the virtual scene image according to the input sensitivity. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0040] Figure 1 This paper illustrates a flowchart of a virtual scene control method provided in an embodiment of this application.

[0041] Figure 2This illustration shows another flowchart of the virtual scene control method provided in an embodiment of this application;

[0042] Figure 3 This illustration shows another flowchart of the virtual scene control method provided in an embodiment of this application;

[0043] Figure 4 This illustration shows another flowchart of the virtual scene control method provided in an embodiment of this application;

[0044] Figure 5 This illustration shows a schematic diagram of the composition structure of a virtual scene control device provided in an embodiment of this application;

[0045] Figure 6 A schematic diagram of the component architecture of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0046] The solution proposed in this application can be applied to any electronic device or system consisting of multiple electronic devices that needs to present virtual scenes using virtual reality, augmented display, or other types of virtual technologies, so as to more flexibly control the presentation of virtual scenes and improve the interactive flexibility of virtual scenes.

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0048] like Figure 1 This document illustrates a flowchart of a virtual scene control method provided in an embodiment of this application. The method of this embodiment can be applied to any electronic device or virtual scene control system for virtual scene control.

[0049] The method in this embodiment may include:

[0050] S101, Real hand gestures of a real user are detected, and the real hand gestures are mapped to virtual hand gestures of a virtual user in a virtual scene image.

[0051] In this context, "real user" refers to a user who actually exists in the physical scene. For ease of distinction, the hand gestures of a real user in the physical world are referred to as "real hand gestures," while the hand gestures of a virtual user in a virtual scene image are referred to as "virtual hand gestures."

[0052] It is understood that during the virtual scene display process, the display of virtual images in the virtual scene can be controlled or adjusted through the input operations of real users. In this application, the virtual scene images include at least virtual users associated with real users. Based on this, by capturing the actions and behaviors of real users, the actions and behaviors of real users can be converted into the response actions and behaviors of the virtual users in the virtual scene images.

[0053] For example, in digital communication scenarios or various scenarios such as virtual e-commerce live streamers, it may be necessary to construct virtual users in virtual scene images and drive virtual users through real users. Of course, there are other possible application scenarios for this case, and this application does not limit them.

[0054] There are several possible ways to detect real user hand gestures. For example, a camera can be used to capture images of the real user, and then the user's hand gestures can be determined from those images. Alternatively, sensors worn on at least one part of the user's body, such as their hand, can be used in conjunction with motion capture technology to obtain the user's real hand gestures.

[0055] Of course, this is just a simple illustration using two examples. In practical applications, there are other possibilities for detecting or capturing real user hand gestures. In practice, multiple different technologies may be combined to detect real user hand gestures, and there are no limitations on this.

[0056] It is understandable that mapping real user's real hand gestures to virtual user's virtual hand gestures means converting real user's hand gestures into virtual user's virtual hand gestures, so that the virtual user's virtual hand gestures are consistent with the real user's real hand gestures, or so that the virtual hand gestures and real hand gestures meet specific conversion rules, without any restrictions.

[0057] For example, if a real hand gesture is to spread out the fingers and move them upwards, then by continuously mapping the real hand gestures at different times to virtual hand gestures, the virtual hand can also perform the action of spreading out the virtual hand's fingers and moving them upwards.

[0058] It should be noted that a real hand operation is a continuous action. Therefore, in practical applications, real hand operations can be obtained at irregular intervals or according to a sampling period, and the real hand operations obtained at different times can be continuously mapped to virtual hand operations, so that step S101 can be executed multiple times.

[0059] S102, if the hand gesture of the actual hand operation belongs to the set holding gesture used to maintain the position of the object, obtain the voice information input by the actual user.

[0060] Among them, the holding posture refers to the hand posture that can maintain an object in a certain position or posture or at least one state.

[0061] For example, a holding posture can be a gripping posture of holding an object or other objects; a holding posture can also be a pushing posture of the hand supporting an object or other objects; a holding posture can also be a lifting posture of the hand pulling an object or other objects; of course, a holding posture can also be a posture of the hand hugging or pinching to maintain an object or other objects, and there are no restrictions on this.

[0062] It is understandable that when a real user's hand gesture at a certain moment is a holding gesture, it indicates that the real user's hand is currently holding an object, or that the real user wants to express the effect of holding an object through hand gestures. Based on this, in order to accurately determine the object the real user expects the virtual user's hand to hold, this application can further obtain the real user's voice information to subsequently analyze and determine the object the real user wants to present when the real hand is in a holding gesture.

[0063] S103, identify the target object described in the voice information.

[0064] The target object can be an object or a small animal that can be displayed through the hand.

[0065] It is understandable that, given that a real user's actual hand gestures involve holding an object, the object being held by the user can be determined based on the user's voice information.

[0066] It is understandable that there are various ways to recognize voice information, and this application does not impose any restrictions on the specific implementation of voice information recognition.

[0067] S104, add a virtual target object to the virtual hand of the virtual user that performs the virtual hand operation.

[0068] It is understandable that by adding virtual target objects to the virtual user's virtual hand, the virtual user can appear to be holding the target object, thus enabling the virtual user in the virtual scene to more realistically or flexibly present the holding effect that the real user would like to achieve.

[0069] To facilitate understanding, let's take an application scenario as an example:

[0070] Take, for example, a scenario where motion capture technology captures the actions of a real user and drives a virtual user for live streaming. In this scenario, the expressions and actions of the real user can be vividly presented on the virtual user. However, when the real user is holding an object, because the background other than the real user is removed in order to accurately capture the real user's actions, the object held in the real user's hand cannot be captured.

[0071] In this application, when it is recognized that the real user's real hand operation is in a holding posture, the target object described in the real user's voice input can be identified to determine the item that the user wants to be presented in the virtual user's virtual hand. By adding the corresponding virtual item to the virtual user's virtual hand, it can be shown that the virtual user is also holding the item.

[0072] For example, if a real user is holding an apple with their real hand and says "Look at this apple," then it can be determined that the real user's hand is holding an apple. Correspondingly, an apple can be added to the virtual hand of the virtual user making the holding gesture.

[0073] Of course, this example is based on a real user actually holding an item. If a real user makes a holding gesture with their hand, but the real user does not actually hold the item, but wants the virtual user to appear to be holding the item, then as long as the real user describes the object they want to hold with their hand, the virtual user's hand can also appear to be holding the item.

[0074] There are several possible ways to add virtual target objects to the virtual hand of a virtual user.

[0075] In one alternative approach, to more realistically represent the virtual user's holding action, a target object can be added to the virtual hand based on the category of the holding posture corresponding to the virtual hand's operation.

[0076] For example, if the virtual hand gesture corresponds to the posture of holding an object, then a virtual target object can be added to the center of the virtual hand to present the effect of the virtual hand holding the target object.

[0077] For example, the virtual hand gesture corresponding to the pinching and lifting of an object can be represented by adding a virtual target object to the finger part of the virtual hand that makes the pinching and lifting gesture, so as to present the effect of the virtual user pinching and lifting the virtual target object with the virtual hand.

[0078] As can be seen from the above, in this embodiment of the application, while mapping the real user's real hand operation to the virtual user's virtual hand operation in the virtual scene image, if the real user's real hand operation's hand posture belongs to a set holding posture, a virtual target object can be added to the corresponding virtual hand of the virtual user based on the target object described in the real user's voice information. This allows the virtual user in the virtual scene image to not only reflect the real user's actions and behaviors, but also to present the object described by the real user, enabling the virtual scene image to more accurately present the scene expected by the real user and improving the display flexibility of the virtual scene.

[0079] Understandably, in order to add a virtual target object to a virtual user's virtual hand, a virtual target object can first be constructed. The specific implementation of this virtual target object can have various possibilities; for example, it can be constructed using AI-generated content (AIGC) technology. Of course, there are other ways to construct virtual target objects, and there are no restrictions on this.

[0080] It is understandable that objects are diverse, and even the same type of object can be divided into different subcategories. Objects in different subcategories also have their own characteristics, and even the same type of object can have different physical characteristics. For example, apples can be divided into green apples and red apples, and different green apples can vary in size and color.

[0081] Therefore, in order to more realistically present the target object actually held or described by a real user, this application can also determine the object characteristics of the target object before constructing the virtual object.

[0082] The following description uses a method for determining the object characteristics of a target object to illustrate the solution of this application. For example... Figure 2 As shown, it illustrates a flowchart of another embodiment of the virtual scene control method provided in this application. The method of this embodiment may include:

[0083] S201, Real hand gestures of a real user are detected, and the real hand gestures are mapped to virtual hand gestures of a virtual user in a virtual scene image.

[0084] S202, if the hand gesture of the actual hand operation belongs to the set holding gesture for maintaining the position of the object, obtain the voice information input by the actual user.

[0085] The above steps S201 and S202 can be found in the relevant descriptions of the previous embodiments, and will not be repeated here.

[0086] S203, identify the object category and object characteristics of the target object described in the speech information.

[0087] The object category of the target object can be the type to which the target object belongs. The object category of the target object can represent what kind of object the target object is, and may also represent the name of the object.

[0088] For example, if the object category of the target object is apples among fruits, then the target object can be identified as apples. Of course, if the object category also includes further subcategories of apples, then the type or name of the apple can be further determined. For instance, if the object category includes the apple category "Golden Delicious," then the apple can be identified as Golden Delicious.

[0089] Object characteristics can be attributes or features inherent to the object itself. For example, if the object is an apple, the characteristics of an apple can include the color of its peel, the texture of its peel, and the size of the apple.

[0090] There can be various specific implementations for identifying object categories and features from voice information, and this application does not impose any restrictions on them.

[0091] S204: Construct a virtual target object based on the object category and object characteristics of the target object.

[0092] Similar to the previous examples, there are multiple ways to construct the virtual target object in this embodiment, and no restrictions are imposed on this.

[0093] For example, if the target object is an apple, and a real user is holding a large red apple in their hand, and the real user describes it in words as "Look at this apple, it's really big, round and red", then the target object can be identified as a large, red and round apple. Thus, a large and round red apple can be constructed, so that the virtual user can also present the effect of holding a large red apple in their hand.

[0094] Alternatively, a real user might make a gesture of lifting an object, but their hand isn't actually holding an apple. However, the real user wants the displayed virtual scene image to include a virtual user holding a large, red apple, so that other users can more intuitively and realistically understand the apple described by the real user. In this case, if the real user, while making the gesture of lifting an object, verbally describes, "Look at this apple, it's really big, round, and red," then the solution in this application will also construct a virtual large, red, round apple. Placing this apple in the corresponding hand of the virtual user will present the virtual scene image that the user desires.

[0095] S205, add the proposed target object to the virtual hand of the virtual user that performs the virtual hand operation.

[0096] It is understood that in constructing the virtual target object, this embodiment takes into account the object category and object characteristics of the target object, so that the constructed virtual target object can better match the target object described by the real user, thereby making the object actually held or desired by the real user accurately presented in the virtual user's hand.

[0097] Understandable, Figure 2 This example illustrates a method for constructing a virtual target object. In practical applications, while or after recognizing the target object based on real user voice input, the object's characteristics can also be determined based on the real user's actual hand gestures. Based on these characteristics, a virtual target object is then constructed.

[0098] It is understandable that when the actual hand operation is a holding posture, the type of holding posture, as well as the posture of the hand and fingers in the actual hand operation, can reflect the characteristics of the object being held, such as weight, size, volume, and softness. Based on this, the object characteristics of the target object can also be determined by combining the hand posture characteristics of the actual hand operation.

[0099] For example, if the target object is an apple, and the real user's actual hand gesture is a gripping posture, then the size of the apple can be analyzed based on the actual gripping posture, thus obtaining the apple's size as a feature.

[0100] For example, if the actual hand operation is a hugging posture, then the general posture and size of the object supported by the hand can be analyzed based on the hugging posture.

[0101] Of course, this is just a simple example of a few situations. In practical applications, in addition to combining the posture of the hand, arm and fingers of the real hand operation, information such as the body posture of the real user can also be considered to comprehensively determine the object characteristics of the target object.

[0102] It is understandable that, in this application, considering that the actual hand operations of a real user may vary at different times, it is possible that at a certain moment the real user does not intend to make a holding posture, but is identified as holding a posture due to a certain operation. Based on this, in order to reduce false recognition, this application can only obtain the voice information input by the real user and identify the target object described in the voice information if the actual hand operation of the user belongs to the set holding posture for maintaining the object's position, and the duration of the actual hand operation in the holding posture exceeds the set duration.

[0103] The duration of this setting can be adjusted as needed, such as 3 seconds or 5 seconds.

[0104] For example, if a real user holds an object and wants to present the same object to a virtual user, the real user will inevitably hold the object for a period of time. Based on this, by recognizing the object described by the real user's voice information, the object can be added to the hand of the virtual user making the holding action, so that the virtual user can present the effect of holding the object.

[0105] If a real user's hand makes a holding gesture while performing other actions, but the holding gesture is not maintained for the set duration, it means that the user is not actually holding the item. In this case, there is no need to recognize the item described by the real user's voice information, and naturally, there is no need to add the item to the virtual user's hand.

[0106] It is understandable that there may be many possibilities in the specific implementation of mapping real user operations to virtual user operations, and this application does not impose any restrictions on this.

[0107] However, research has found that, under the premise of ensuring that the virtual user's virtual operation behavior is consistent with the real user's operation behavior in terms of posture and action content, the real user's operation behavior is generally mapped to the corresponding virtual user's virtual operation behavior according to a set mapping ratio. For example, a proportional mapping method can be used to map the real user's real operation behavior to the virtual user's virtual operation behavior, so that the virtual user's virtual operation behavior is consistent with the real user's operation behavior in terms of action content, movement distance, and movement range.

[0108] However, when the operation range of a virtual scene is large or precise and continuous input is required, the system maps real actions to virtual actions based on a set mapping ratio. This results in poor controllability of the virtual scene and can easily lead to low input efficiency due to inconvenient input operations.

[0109] For example, consider a scenario where a real user needs to drag a slider in an interactive interface within a virtual scene image. Because the virtual space of the scene image is large, and the display area of ​​the interactive interface is also large, if a proportional mapping method is used, the real user needs to perform a relatively long sliding motion with their hand to drag the slider in order to be mapped to the virtual user in the virtual scene image completing the slider dragging operation. This results in a long sliding distance and significant time consumption for the real user, leading to low input efficiency.

[0110] Based on this, in the embodiments of this application, various input sensitivities can be set to map real hand operations to virtual hand operations in a virtual scene image. Different input sensitivities result in different mapping ratios from real hand operations to virtual hand operations. For example, under the same real hand operation, different input sensitivities will result in different amplitudes of motion changes in the mapped virtual hand operation.

[0111] The following example illustrates the solution proposed in this application by illustrating one method for determining input sensitivity. Figure 3 As shown, it illustrates a flowchart of yet another embodiment of the virtual scene control method provided in this application.

[0112] The method in this embodiment may include:

[0113] S301, a real user's real hand operation is detected, and the input gesture type of the real hand operation is determined.

[0114] The input gesture type refers to the gesture type corresponding to the gesture portion in a real hand operation that reflects input sensitivity. Accordingly, the input gesture type can reflect the input sensitivity required for that real hand operation.

[0115] In this application, the type of input gesture is different from the type of operation performed by the real hand to control the virtual scene. Therefore, depending on the different requirements for input sensitivity, when the real user performs the same type of control operation on the virtual scene, the type of input gesture corresponding to the real hand operation performed by the real user to perform that type of control operation may also be different.

[0116] There are several possible ways to determine the type of input gesture, which will be explained below in conjunction with several possible scenarios.

[0117] In one possible scenario, after obtaining a real hand gesture, the finger posture used to indicate the gesture type in that real hand gesture can be determined. Accordingly, based on that finger posture, the input gesture type of the real hand gesture can be determined.

[0118] The finger gesture for indicating a gesture type refers to the finger gesture indicating the input finger type. Finger gestures can be characterized through multiple dimensions, such as finger posture and the number of fingers, without limitation. In practical applications, a corresponding number of finger gestures can be set based on the number of input gesture types.

[0119] For example:

[0120] Let's take the number of finger pinches as an example to illustrate different input gesture types.

[0121] Assuming input sensitivity is divided into three levels: high, medium, and low, each level corresponds to one of three different input gesture types. Based on this, we can define a single finger as the first type of input gesture corresponding to the high sensitivity level, two fingers pinched together as the second type of input gesture corresponding to the medium sensitivity level, and three fingers pinched together as the third type of input gesture corresponding to the low sensitivity level.

[0122] Based on this, if a real user makes a real hand gesture and two fingers are pinched together, then the input gesture type can be determined to be the second type of input gesture.

[0123] In another implementation, the real user can pre-select the input gesture type corresponding to the real hand operation before performing the real hand operation. For example, the real user can pre-select the required input gesture type from a variety of gesture type options before performing the real hand operation.

[0124] In practical applications, in addition to obtaining the actual hand operation, body postures other than the hand can also be used to help determine the input gesture type corresponding to the actual hand operation. For example, the number of head movements or the number of consecutive blinks can be used to determine the input gesture type corresponding to the actual hand operation. Of course, there are other ways to determine the input gesture type, and there are no restrictions on this.

[0125] S302, based on the input gesture type, determine the input sensitivity of the virtual input operation mapped from the real hand operation to the virtual input operation in the virtual scene image.

[0126] Input sensitivity refers to the degree to which a real user's input triggers changes in virtual objects (virtual users or other virtual controlled objects) within a virtual scene image. Higher input sensitivity results in greater changes in the virtual objects' actions triggered by the real user's input.

[0127] Therefore, input sensitivity can affect the frequency and magnitude of virtual input operations mapped from real hand operations to virtual scene images.

[0128] For example, in one possible scenario, input sensitivity can reflect the mapping relationship between the frequency of real hand operations and the frequency of virtual hand operations. For instance, when the real hand operation is a click, different input sensitivities will result in different numbers of clicks in the virtual scene image compared to the actual number of clicks.

[0129] In another possible scenario, the input sensitivity is a distance mapping relationship between the movement distance of the real hand operation and the virtual movement distance of the virtual hand in the virtual scene image. This distance mapping relationship is actually a type of operation amplitude mapping relationship, reflecting the proportional relationship between the movement distance of the real hand operation and the movement distance of the virtual hand.

[0130] For example, if the distance mapping is 1:1, then the movement distance of the real hand operation is the same as the virtual movement distance of the virtual hand. If the distance mapping is 2:1, then the movement distance of the real hand operation is twice the virtual movement distance of the virtual hand.

[0131] The distance mapping relationship will vary depending on the input sensitivity. Generally, the higher the input sensitivity, the lower the ratio of the actual hand movement distance to the virtual hand movement distance, meaning that only a small movement of the real user's hand can trigger a large movement of the virtual hand.

[0132] S303, based on the input sensitivity, the real hand operation is mapped to the virtual hand operation of the virtual user in the virtual scene image.

[0133] As mentioned above, input sensitivity can affect the degree to which real hand operations trigger virtual hand operations from the virtual user. Therefore, with different input sensitivities, one or more of the frequency and magnitude of virtual hand operations triggered by real hand operations will vary.

[0134] For example, if the input sensitivity is a distance mapping relationship between the movement distance of a real hand operation and the virtual movement distance of a virtual hand in the virtual scene image, the real hand operation can be mapped to the virtual hand operation of the virtual user in the virtual scene image according to the distance mapping relationship and the movement trajectory of the real hand operation.

[0135] Of course, input sensitivity can also be reflected in multiple dimensions such as operation frequency and operation range. In this case, this application can also consider the mapping of the operation frequency of real hand operation such as click to the operation frequency of virtual hand operation to comprehensively determine the mapped virtual hand operation, which will not be elaborated further.

[0136] It should be noted that in the above steps S301 to S303, it does not matter whether the hand posture of the real hand operation is a holding posture. As long as the real hand operation is obtained, the real hand operation can be mapped to the virtual hand operation of the virtual user by following the above steps.

[0137] S304, if the hand gesture of the actual hand operation belongs to the set holding gesture for maintaining the position of the object, obtain the voice information input by the actual user.

[0138] S305, identify the target object described in the voice information.

[0139] S306, add a virtual target object to the virtual hand of the virtual user that performs the virtual hand operation.

[0140] The steps S304 to S306 above can be found in the relevant descriptions of the previous embodiments, and will not be repeated here.

[0141] In this embodiment, after obtaining real hand operations, the input sensitivity can be determined based on the input operation type of the real hand operations. This allows the real user to reasonably control the input sensitivity as needed, thereby achieving reasonable control over the mapping relationship between real hand operations and virtual hand operations, improving the flexibility of virtual scene control. Furthermore, by reasonably adjusting the input sensitivity through the input operation type of the real hand operations, the mapping relationship from real hand operations to virtual hand operations can be adjusted appropriately. This helps reduce input errors during fine-grained control of the virtual scene and reduces situations where real users need to move their hands a considerable distance to complete virtual input operations, thus improving input efficiency.

[0142] To facilitate understanding of the specific implementation of determining input sensitivity and mapping real hand operations to virtual hand operations based on input sensitivity in this application, one implementation method is described below as an example. Considering that real hand operations involve movement, combining input sensitivity allows for more flexible control of the amount of movement variation between the real hand and the virtual hand. Therefore, the explanation will focus on the case where real hand operations involve hand movement.

[0143] like Figure 4 As shown, it illustrates another flowchart of the virtual scene control method provided in this application embodiment. The method of this embodiment may include:

[0144] S401, a real user's real hand operation is detected. If the real hand operation includes a hand movement operation, the finger posture used to indicate the gesture type in the real hand operation is determined.

[0145] Among them, real hand operation includes hand movement operation, which refers to the movement of the real user's real hand during the process of a real user performing a real hand operation. For example, if a real user uses their hand to make a grasping or lifting action while the real user's hand moves upward, then the real hand operation is an operation of making a lifting gesture and moving upward.

[0146] Since real hand operations involve hand movement, the virtual hand will also move accordingly. Therefore, in order to determine the mapping relationship of movement distance between the real hand and the virtual hand, it is necessary to determine the desired input sensitivity.

[0147] The finger postures in actual hand operations can be referred to in the previous introduction. For example, in this application, three input sensitivities can be set, and correspondingly, three finger postures corresponding to the input finger types can be set. For example, the finger posture of the first input gesture type corresponding to high sensitivity is a single finger extended, while the finger posture of the second input gesture type corresponding to medium sensitivity is the thumb and index finger pinched together, and the finger posture of the third input gesture type corresponding to low sensitivity is the thumb, index finger, and middle finger pinched together.

[0148] S402, based on the finger posture, determine the input gesture type of the actual hand operation.

[0149] S403, Based on the input gesture type, determine the input sensitivity of the virtual input operation mapped from the real hand operation to the virtual input operation in the virtual scene image.

[0150] As explained above, when the finger posture corresponds to the set input finger type, the actual hand gesture type can be determined. Correspondingly, the input sensitivity can be determined based on the correspondence between the input gesture type and the input sensitivity.

[0151] S404, Based on the input sensitivity, determine the distance mapping relationship between the movement distance of the real hand operation and the virtual movement distance of the virtual hand in the virtual scene image.

[0152] S405, according to the distance mapping relationship and the movement trajectory of the real hand operation, map the real hand operation to the virtual hand operation of the virtual user in the virtual scene image.

[0153] For example, suppose the input sensitivity is divided into low sensitivity, medium sensitivity, and high sensitivity.

[0154] The distance mapping relationship corresponding to the medium sensitivity level can be a basic distance ratio, which can be set as needed. Taking a basic distance ratio of 1 as an example, when the input sensitivity is medium, the distance mapping relationship is that the ratio of the actual hand movement distance to the virtual hand movement distance is 1. Correspondingly, the movement distance of the virtual hand operation is the same as the movement distance of the actual hand operation.

[0155] In low-sensitivity mode, the distance ratio corresponding to the distance mapping relationship can be the base distance ratio multiplied by a set coefficient, which can be a value greater than 0 and less than or equal to 1. In high-sensitivity mode, the distance ratio corresponding to the distance mapping relationship can be the base distance ratio multiplied by (1 / set coefficient). After the distance ratios corresponding to low-sensitivity and high-sensitivity modes are determined, if the input sensitivity is low-sensitivity or high-sensitivity, the process of mapping the movement distance of the real hand operation to the movement distance of the virtual hand operation is similar and will not be elaborated further.

[0156] S406, if the hand gesture of the actual hand operation belongs to the set holding gesture for maintaining the position of the object, obtain the voice information input by the actual user.

[0157] S407, identify the target object described in the voice information.

[0158] S408, add the virtual target object to the virtual hand of the virtual user who performs the virtual hand operation.

[0159] The steps S406 to S408 above can be found in the relevant descriptions of the previous embodiments, and will not be repeated here.

[0160] Corresponding to the virtual scene control method of this application, this application also provides a virtual scene control device.

[0161] like Figure 5 The diagram illustrates a structural composition of a virtual scene control device provided in an embodiment of this application. The device in this embodiment may include:

[0162] The operation mapping unit 501 is used to detect the real hand operation of a real user and map the real hand operation to the virtual hand operation of a virtual user in a virtual scene image.

[0163] The voice acquisition unit 502 is used to acquire the voice information input by the real user if the hand posture of the real hand operation belongs to a set holding posture for maintaining the position of the object.

[0164] The speech recognition unit 503 is used to recognize the target object described in the speech information;

[0165] The object adding unit 504 is used to add a virtual target object to the virtual hand of the virtual user performing the virtual hand operation.

[0166] In one possible implementation, the speech acquisition unit includes:

[0167] The voice acquisition subunit is used to acquire the voice information input by the real user if the hand posture of the real hand operation belongs to a set holding posture for maintaining the position of an object, and the duration of the holding posture of the real hand operation exceeds a set duration.

[0168] In yet another possible implementation, the speech recognition unit includes:

[0169] An object recognition unit is used to identify the object category and object characteristics of the target object described in the voice information;

[0170] The device in this application also includes:

[0171] The first object construction unit is used to construct a virtual target object based on the object category and object characteristics of the target object before the object addition unit adds the virtual target object to the virtual hand of the virtual user performing the virtual hand operation.

[0172] In yet another possible implementation, the device further includes:

[0173] The feature determination unit is used to determine the object features of the target object based on the real hand operation of the real user before the object addition unit adds the virtual target object to the virtual hand of the virtual user who performs the virtual hand operation.

[0174] The second object construction unit is used to construct a virtual target object based on the object characteristics of the target object.

[0175] In yet another possible implementation, the operation mapping unit includes:

[0176] A type determination unit is used to detect real hand operations by real users and determine the input gesture type of the real hand operations;

[0177] A sensitivity determination unit is used to determine the input sensitivity of the virtual input operation mapped from the real hand operation to the virtual scene image based on the input gesture type.

[0178] The mapping processing unit is used to map the real hand operation to the virtual hand operation of the virtual user in the virtual scene image according to the input sensitivity.

[0179] In another possible implementation, the mapping processing unit includes:

[0180] The relationship determination subunit is used to determine the distance mapping relationship between the movement distance of the real hand operation and the virtual movement distance of the virtual hand in the virtual scene image, according to the input sensitivity.

[0181] The mapping processing subunit is used to map the real hand operation to the virtual hand operation of the virtual user in the virtual scene image according to the distance mapping relationship and the movement trajectory of the real hand operation.

[0182] In another possible implementation, the type determination unit includes:

[0183] The posture determination subunit is used to determine the finger posture used to indicate the gesture type in the actual hand operation;

[0184] The type determination subunit is used to determine the input gesture type of the actual hand operation based on the finger posture.

[0185] In another possible implementation, the gesture determination subunit is specifically used to determine the finger gesture used to indicate the gesture type in the real hand operation if the real hand operation is a hand movement operation.

[0186] Furthermore, this application also provides an electronic device, such as Figure 6 The diagram shows a structural composition of the electronic device, which can be any type of electronic device and includes at least a processor 601 and a memory 602.

[0187] The processor 601 is used to execute the virtual scene control method in any of the above embodiments.

[0188] The memory 602 is used to store programs required by the processor to perform operations.

[0189] It is understood that the electronic device may also include a display unit 603 and an input unit 604.

[0190] Of course, the electronic device can also have more than Figure 6 There are no restrictions on the number of components, whether more or fewer.

[0191] On the other hand, this application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the virtual scene control method as described in any of the above embodiments.

[0192] This application also proposes a computer program comprising computer instructions stored in a computer-readable storage medium. When executed on an electronic device, the computer program performs the virtual scene control method as described in any of the above embodiments.

[0193] It is understood that in this application, the terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar parts and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that illustrated herein.

[0194] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Furthermore, the features described in the various embodiments of this specification can be substituted or combined with each other, enabling those skilled in the art to implement or use this application. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0195] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0196] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0197] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A virtual scene control method, comprising: Real hand gestures from real users are detected, and these real hand gestures are mapped to virtual hand gestures from virtual users in a virtual scene image. If the hand gesture of the actual hand operation belongs to the set holding posture for maintaining the position of the object, obtain the voice information input by the actual user. Identify the target object described in the voice information; The virtual target object is added to the virtual hand of the virtual user performing the virtual hand operation based on the target object described in the voice information of the real user.

2. The method according to claim 1, wherein obtaining the voice information input by the real user if the hand posture of the actual hand operation belongs to a set holding posture for maintaining the object's position includes: If the hand gesture of the actual hand operation belongs to the set holding posture for maintaining the position of the object, and the duration of the actual hand operation in the holding posture exceeds the set duration, the voice information input by the actual user is obtained.

3. The method according to claim 1, wherein identifying the target object described in the voice information includes: Identify the object category and object characteristics of the target object described in the voice information; Before adding the virtual target object to the virtual hand that performs the virtual hand operation in the virtual user, the method further includes: Based on the object category and object characteristics of the target object, a virtual target object is constructed.

4. The method according to claim 1, wherein mapping the real hand operation to a virtual hand operation of a virtual user in a virtual scene image comprises: Determine the type of input gesture for the actual hand operation; Based on the input gesture type, determine the input sensitivity of the virtual input operation mapped from the real hand operation to the virtual input operation in the virtual scene image; Based on the input sensitivity, the real hand operation is mapped to the virtual hand operation of the virtual user in the virtual scene image.

5. The method according to claim 4, wherein mapping the real hand operation to a virtual hand operation of a virtual user in a virtual scene image according to the input sensitivity comprises: Based on the input sensitivity, determine the distance mapping relationship between the movement distance of the real hand operation and the virtual movement distance of the virtual hand in the virtual scene image; Based on the distance mapping relationship and the movement trajectory of the real hand operation, the real hand operation is mapped to the virtual hand operation of the virtual user in the virtual scene image.

6. The method according to claim 4 or 5, wherein determining the input gesture type of the actual hand operation includes: Determine the finger posture used to indicate the gesture type in the actual hand operation; Based on the finger posture, the input gesture type of the actual hand operation is determined.

7. The method according to claim 6, wherein determining the finger posture used to indicate the gesture type in the actual hand operation includes: If the actual hand operation includes hand movement, determine the finger posture used to indicate the gesture type in the actual hand operation.

8. The method according to claim 1, further comprising, before adding the virtual target object to the virtual hand performing the virtual hand operation in the virtual user: Based on the actual hand movements of the real users, the object characteristics of the target object are determined; Based on the object characteristics of the target object, a virtual target object is constructed.

9. A virtual scene control device, comprising: An operation mapping unit is used to detect real hand operations of a real user and map the real hand operations to virtual hand operations of a virtual user in a virtual scene image. The voice acquisition unit is used to acquire the voice information input by the real user if the hand posture of the real hand operation belongs to a set holding posture for maintaining the position of the object. A speech recognition unit is used to identify the target object described in the speech information; An object adding unit is used to add a virtual target object to the virtual hand of the virtual user performing the virtual hand operation, based on the target object described in the voice information of the real user.

10. The apparatus according to claim 9, wherein the operation mapping unit comprises: A type determination unit is used to detect real hand operations by real users and determine the input gesture type of the real hand operations; A sensitivity determination unit is used to determine the input sensitivity of the virtual input operation mapped from the real hand operation to the virtual scene image based on the input gesture type. The mapping processing unit is used to map the real hand operation to the virtual hand operation of the virtual user in the virtual scene image according to the input sensitivity.

Images