Entity prop interaction-oriented method, apparatus and device, and storage medium
Through a physical prop interaction method, the recognition and display units are used to respond to user interaction and automatically generate scene content, solving the problem of limited interaction in virtual space and enhancing children's immersive experience and creativity.
Patent Information
- Application Number
- CN202411014023.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-19
AI Technical Summary
Existing interactive games and learning are mainly based on virtual spaces and virtual props, resulting in user interaction being limited by preset scenes and content, reducing user interest in interaction, especially for children, limiting their imagination and exploration abilities.
Through the method of physical prop interaction, the recognition unit and display unit are used to select gestures and interaction types in response to the user's target, determine the behavioral actions and intentions of physical props, and automatically generate scene content based on the real environment and interactive information, providing multimodality Present.
It improves users' immersive interactive experience and exploration interests, especially children's creativity and perception, allowing users to interact and imagine independently without being restricted by preset scenarios and content.
Smart Images

Figure CN118976247A_ABST
Abstract
Description
Technical Field
[0001] The present application generally relates to the field of entity interaction technology. More specifically, the present application relates to a method, device, apparatus and computer-readable storage medium for entity prop interaction. Background Art
[0002] Existing interactive games, stories or learning are usually based on virtual space and virtual props, and the scenes or content (including storylines, game plots and role relationships) in the virtual space or virtual props are all pre-set, and the interactive content is often fixed. This limits users to preset scenes and preset content for interactive experience, reducing users' interest in interaction. Especially for children, only based on preset interactive content, it will limit children's imagination, exploration and other abilities.
[0003] In view of this, there is an urgent need to provide a solution for interaction with physical props, so that users can interact and imagine completely independently, thereby enhancing their immersive interactive experience and exploration interest. Summary of the invention
[0004] In order to at least solve one or more of the technical problems mentioned above, the present application proposes solutions for physical prop interaction in multiple aspects.
[0005] In a first aspect, the present application provides a method for physical prop interaction, comprising: within a recognition range, in response to an identified target selection gesture, determining a physical prop selected based on the target selection gesture as a target object; in response to an identified target interaction type, determining a target behavior action of the target interaction type acting on the target object; and presenting the behavior intention of the target object according to the target behavior action.
[0006] In some embodiments, the target selection gesture includes at least two fingers in a pinch state.
[0007] In some embodiments, the target interaction type includes at least one or more of the following: single-target interaction gesture, dual-target interaction gesture, target interaction voice or prop interaction posture; single-target interaction gesture and target interaction voice; dual-target interaction gesture and target interaction voice; single-target interaction gesture, target interaction voice and prop interaction posture; or dual-target interaction gesture, target interaction voice and prop interaction posture.
[0008] In some embodiments, the single-target interaction gesture and the dual-target interaction gesture are implemented by one or more basic gestures, and the basic gestures include at least one or more of pointing, grasping, pinching, making a fist, closing or crossing the fingers or palms; the prop interaction posture includes at least one or more of standing, lying flat, rolling, moving or lying prone; the target interaction voice includes the user's language recognized in real time.
[0009] In some embodiments, the method further includes: in response to recognizing that the target selection gesture selects the target object and stays at a target position for a target time, triggering a menu bar related to the target object.
[0010] In some embodiments, the method further includes: in response to the recognition range including a plurality of physical props, determining physical props other than the target object as interactive objects; and in response to moving the target object near the interactive object, causing the target object to interact with the interactive object.
[0011] In some embodiments, the interaction between the target object and the interactive object is achieved by the following operations: identifying target interaction information; and judging the interaction priority of the interaction between the interactive object and the target object based on the target interaction information, so as to achieve the interaction between the target object and the interactive object in order of interaction priority.
[0012] In some embodiments, the target interaction information includes at least roll call response, intention information, historical conversation data or distance information.
[0013] In some embodiments, the interaction priority of the interaction object with the target object is determined by the following operations: in response to the identified target interaction information including a roll call response, the interaction priority of the interactive object that is named and responds is determined to be the highest priority; in response to the identified target interaction information not including a roll call response but including intention information, the interaction priority of the interactive object with intention is determined to be the highest priority; in response to the identified target interaction information not including a roll call response and intention information but including historical conversation data, the interaction priority of the interactive object with a historical conversation with the target object is determined to be the highest priority; or in response to the identified target interaction information only including distance information, the interaction priority of the interactive object that is closest to the target object is determined to be the highest priority.
[0014] In some embodiments, the method further includes: in response to identifying that the target object is moved to the vicinity of the target interactive object and a conversation takes place, determining that the target interactive object directly responds; in response to identifying that the target object is moved to the vicinity of the target interactive object without a conversation, determining that the target interactive object starts a conversation; in response to identifying that the target interactive object has not moved or that the target interactive object has moved and a conversation takes place, determining that the target interactive object continues to respond; in response to identifying that the target interactive object has moved and the conversation takes place, determining that a nearby interactive object responds; or in response to identifying that any two target interactive objects move and have a conversation or do not have a conversation, determining that a target interactive object closest to the target object responds.
[0015] In some embodiments, the method also includes: identifying real environment information where the target object and the interactive object are located, interaction information between the target object and the interactive object, interaction postures corresponding to each of the target object and the interactive object, and language information of the user; and automatically generating specific scene content based on at least the real environment information, the interaction information, the interaction posture and the language information and presenting the specific scene content.
[0016] In some embodiments, the method of presenting the behavioral intention of the target object or presenting the specific scene content includes at least an image, a sound effect, or a light effect.
[0017] In some embodiments, the method is at least oriented to child users, and the method is at least applied in a children's game scenario, a children's story scenario, or a children's teaching scenario.
[0018] In some embodiments, the physical props include at least physical toys for children or physical teaching aids for children.
[0019] In a second aspect, the present application provides a device for physical prop interaction, comprising: an identification unit, which is used to determine, within a recognition range, in response to an identified target selection gesture, a physical prop selected based on the target selection gesture as a target object; a display unit, which is used to: determine, in response to an identified target interaction type, a target behavior action of the target interaction type acting on the target object; and present the behavior intention of the target object according to the target behavior action.
[0020] In some embodiments, the recognition unit includes at least a camera; the display unit includes at least a screen device, a sound device or a light device.
[0021] In a third aspect, the present application provides a device for physical prop interaction, comprising: a processor; and a memory, wherein program instructions for physical prop interaction are stored, and when the program instructions are executed by the processor, the device implements one or more embodiments of the aforementioned first aspect.
[0022] In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon computer-readable instructions for interaction with physical props, wherein when the computer-readable instructions are executed by one or more processors, one or more embodiments of the aforementioned first aspect are implemented.
[0023] Through the scheme for physical prop interaction provided above, the embodiment of the present application takes the physical prop as the target object, and directly acts on the target object according to the target behavior action determined by the identified target interaction type, so that the target object has a certain behavioral intention. Compared with the preset interactive content or interactive scenes of virtual props in traditional virtual spaces and fixed scenes, the embodiment of the present application can be not limited to preset scenes, preset entities or preset interactions, and the user can interact, imagine and set the behavioral intention of the target object completely independently, greatly enhancing the user's immersive interactive experience. Especially for children, it can enhance the exploration and perception ability in the physical space, and better cultivate children's ability and literacy in language, creativity or number sense. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:
[0025] Figure 1 is an exemplary schematic diagram showing the arrangement of a physical prop interaction scene according to an embodiment of the present application;
[0026] Figure 2 is an exemplary flowchart showing a method for physical prop interaction according to an embodiment of the present application;
[0027] Figure 3 is an exemplary schematic diagram showing a target selection gesture according to an embodiment of the present application;
[0028] Figure 4 is an exemplary schematic diagram showing a target behavior action and its behavior intention for a target object in a game scene or a story scene according to an embodiment of the present application;
[0029] Figure 5 is an exemplary schematic diagram showing the interaction priority of a physical prop according to an embodiment of the present application;
[0030] Figure 6 is an exemplary structural block diagram showing a device for physical prop interaction according to an embodiment of the present application;
[0031] Figure 7 It is an exemplary structural block diagram showing a device for physical prop interaction according to an embodiment of the present application. DETAILED DESCRIPTION
[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0033] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.
[0034] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.
[0035] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.
[0036] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.
[0037] It should be understood that in the implementation scenario, the scheme for physical prop interaction of the embodiment of the present application can be implemented by a device that is at least provided with a recognition unit and a display unit. In some embodiments, the aforementioned recognition unit may include but is not limited to a camera, a laser radar or other sensors, detectors, etc. with recognition (such as voice recognition) and perception functions. Based on the aforementioned recognition unit, for example, the user's interactive gestures, the user's language, the location information and posture information of the physical props, the user's real environment information, the interactive imagination between the user and the physical props, etc. can be recognized, and multimodal input data can be provided.
[0038] The aforementioned display unit may include but is not limited to a screen device (such as a display screen or a projection, etc.), an audio device (such as voice output), a light effect device (such as light output) or a special effect (such as vibration, blowing, bubbles, etc.) display device, etc. Based on the aforementioned display device, multimodal output data (such as images, videos, sounds, light effects or special effects, etc.) can be presented based on multimodal input data to enhance the user's interactive experience.
[0039] In some embodiments, the aforementioned device may be, for example, a robot, an electronic pet, etc., and the present application does not impose any limitation on this. In some embodiments, when performing physical prop interaction, the user places the physical prop in front of the device, such as a robot, an electronic pet, etc., to realize physical prop interaction within the recognition range of the device, for example Figure 1 shown.
[0040] Figure 1 is an exemplary schematic diagram showing the arrangement of a physical prop interaction scene according to an embodiment of the present application. Figure 1 As shown in FIG. 1 , the device 100 may include at least a recognition unit 101 and a display unit 102. In the implementation scenario, a user 103 places a physical prop 104 in front of the device 100 to interact with the physical prop. The specific interaction details will be described later in conjunction with FIG. Figure 2-Figure 5 Detailed description.
[0041] Preferably, the user in the context of the embodiment of the present application can be a child user oriented to the age group of 3 to 12 years old. In addition, the user of the embodiment of the present application can include one or more. In some embodiments, the scheme of the embodiment of the present application can be applied to but not limited to children's game scenes, children's story scenes or children's teaching scenes. Accordingly, the physical props of the embodiment of the present application may include but are not limited to children's physical toys or children's physical teaching aids, etc., and children's physical toys or children's physical teaching aids are also not limited to physical props such as cards, chess pieces, building blocks, cartoon dolls, cartoon animals, paintbrushes, etc.
[0042] It can be understood that children aged 3 to 12 are in the period of exploring in physical space and building up their perceptual abilities. Therefore, through the physical prop interaction of the embodiment of the present application, children can choose the physical props they are interested in without being limited by the preset scenes or preset entities, and they can imagine and interact with them independently. For example, they can independently set game scenes or game characters, independently set story scenes or story content, and independently learn, thereby greatly improving the cultivation of children's abilities and qualities in language, creativity, or number sense.
[0043] Figure 2 FIG. 2 is an exemplary flowchart showing a method 200 for interacting with physical props according to an embodiment of the present application. Figure 2 As shown in , at step S201, within the recognition range, in response to the recognized target selection gesture, the physical prop selected based on the target selection gesture is determined as the target object. As can be seen from the foregoing, the target object of the user interaction can be determined within its recognition range based on the recognized target selection gesture of the user, for example, by camera recognition. In some embodiments, the target selection gesture may include at least two fingers in a pinched state. Preferably, the thumb and index finger in a pinched state can be used as the target selection gesture (e.g. Figure 3 In one implementation scenario, the user can specifically determine whether the user is in a pinching state by, for example, recognizing the curvature of the key points of the user's hand and the distance between the fingers and the physical prop by a camera. When the user is in a pinching state, it is determined that the user has selected a physical prop, and the selected physical prop is used as the target object.
[0044] In some embodiments, the user can move by pinching the selected target object to achieve the movement or tracking of the physical prop. In other embodiments, in response to identifying that the target selection gesture selects the target object and stays at the target position for the target time, a menu bar related to the target object is triggered. That is, the embodiment of the present application can set a menu bar in the display unit, and when the user pinches the physical prop and stays for a period of time (e.g., 5 seconds), the menu bar can be triggered on the display unit to manually confirm the selection of the target object or perform relevant settings on the target object. For example, the attribute parameters (e.g., color, image, function, etc.) of the target object can be set through the menu bar to provide a variety of interactive methods for the physical prop. In other embodiments, the aforementioned menu bar can be set to an input box form, a selection box form, a drop-down menu form, or a combination of the aforementioned various forms, and the present application does not impose any restrictions in this regard.
[0045] Based on the selected target object, at step S202, in response to the recognized target interaction type, determine the target behavioral action of the target interaction type acting on the target object. In some embodiments, the target interaction type may include, but is not limited to, one or more of the following: single-target interaction gesture, double-target interaction gesture, target interaction voice, or prop interaction posture; single-target interaction gesture and target interaction voice; double-target interaction gesture and target interaction voice; single-target interaction gesture, target interaction voice, and prop interaction posture; or double-target interaction gesture, target interaction voice, and prop interaction posture.
[0046] In other embodiments, the single-target interaction gesture and the double-target interaction gesture are implemented by one or more basic gestures, and the basic gestures at least include one or more of pointing, grasping, pinching, making a fist, closing, or crossing of fingers or palms. In addition, the basic gesture may also be, for example, carrying, holding flat, a fist, etc. The aforementioned prop interaction posture may include, but is not limited to, one or more of standing, lying flat, rolling, moving, or prone, or may also include, for example, vertical / horizontal lifting, etc. The aforementioned target interaction voice may include, but is not limited to, the language of the user recognized in real time.
[0047] For example, taking a story or game scenario as an example, the user points at the target object with a single finger of one hand to represent the target behavioral action acting on the target object; or for example, the target behavioral action of closing the palm of one hand; the target behavioral action of the Guanyin gesture with both hands clasped; the target behavioral action of pointing at the target with a pistol gesture and matching it with the voice "da da da"; or the target behavioral action of pinching a prop upward with two fingers and matching it with the voice "xiu xiu". For more exemplary target behavioral actions, reference can be made to Figure 4 .
[0048] In still other embodiments, the aforementioned target interaction type may also include, for example, target interaction semantics. The target interaction semantics may be, for example, the text input by the user in real time via the display unit. Additionally, the aforementioned target interaction type may also be, for example, the user's expression, etc. Or, the aforementioned target interaction type may also include geographical points of interest ("POI"), such as cities, mountains, beaches, seaside, jungles, etc. in virtual or real scenes. The present application makes no limitation in this regard, and any action that can form a behavioral action on an entity prop belongs to the protection scope of the embodiments of the present application.
[0049] Further, at step S203, the behavioral intention of the target object is presented according to the target behavioral action. For example, taking the above-mentioned target behavioral actions such as pointing at the target object with one hand, closing the palm of one hand, the Guanyin gesture of putting both hands together, pointing at the target with the gesture of a pistol, and matching the voice "da da da", or pinching an item with two fingers and lifting it up, matching the voice "咻咻" (swish swish) as examples, corresponding behavioral intentions such as taking attendance, chopping with the hand, summoning a character, shooting, teleporting, etc. can be presented. In an implementation scenario, presenting the behavioral intention of the aforementioned target object can be achieved through one or more ways such as image screens, sound effects, light effects, or special effects.
[0050] In some embodiments, in response to multiple physical props being included within the recognition range, the physical props other than the target object are determined as interaction objects, and in response to moving the target object near the interaction object, the target object and the interaction object are enabled to interact. For example, as shown by Figure 1 in the figure, there are usually multiple physical props. In a children's story scenario or a children's game scenario, one is often selected as the protagonist, and the rest are used as supporting roles (or NPC characters), and the protagonist is usually played by a child user. Thus, in the embodiments of the present application, the physical prop selected by the recognized target selection gesture is used as the target object (i.e., the protagonist), and the physical props other than the target object are determined as interaction objects (i.e., NPC characters). In the implementation scenario, the target object can be moved near the interaction object by pinching it to enable interaction between the target object and the interaction object.
[0051] In some embodiments, by recognizing the target interaction information, the interaction priority of the interaction between the interaction object and the target object is judged based on the target interaction information, so as to implement the interaction between the target object and the interaction object in the order of the interaction priority. That is, the role assignment and the priority of role response can be implemented through the information recognized in the scenario. In some embodiments, the aforementioned target interaction information may include but is not limited to roll call response, intention information, historical dialogue data, or distance information, etc. It can be understood that the aforementioned roll call response means that the target object calls out the role name of the target interaction object through language, and the corresponding target interaction object responds. The intention information represents the scenario or plot set for the target object, and the corresponding target interaction object has intention expressions such as sound or action. The historical dialogue means that there has been a dialogue between the target object and the target interaction object before. The distance information represents the distance between the target object and the interaction object or between the interaction objects.
[0052] Specifically, in one implementation scenario, in response to the identified target interaction information including a roll call response, the interaction priority of the interactive object that is named and responds is determined to be the highest priority; in response to the identified target interaction information not including a roll call response but including intent information, the interaction priority of the interactive object with intent is determined to be the highest priority; in response to the identified target interaction information not including a roll call response and intent information but including historical conversation data, the interaction priority of the interactive object that has a historical conversation with the target object is determined to be the highest priority; or in response to the identified target interaction information only including distance information, the interaction priority of the interactive object that is closest to the target object is determined to be the highest priority.
[0053] That is, when the target object is named, the interactive object named will respond first; when the target object is not named but has an intended object, the intended object will respond first; when the target object is not named or has no intended object, but there is a historical conversation, the historical conversation object will respond first; when the target object is not named or has no intended object and has no historical conversation, the interactive object closest to the target will respond first. It should be understood that the response in the context of the embodiment of the present application can be achieved by, but not limited to, sound or action, and the sound is usually expressed in the form of a reply or conversation.
[0054] In some embodiments, the above-mentioned priority response may also include: in response to identifying that the target object is moved near the target interactive object and a conversation occurs, determining that the target interactive object directly responds; in response to identifying that the target object is moved near the target interactive object but no conversation occurs, determining that the target interactive object starts a conversation; in response to identifying that the target interactive object does not move or that the target interactive object moves and a conversation occurs, determining that the target interactive object continues to respond; in response to identifying that the target interactive object moves and a conversation occurs, determining that a nearby interactive object responds; or in response to identifying that any two target interactive objects move and have a conversation or do not have a conversation, determining that a target interactive object closest to the target object responds. That is, the object of priority response is determined by whether the face-to-face target object and / or interactive object moves and whether a conversation occurs. This will be discussed later in conjunction with Figure 5 Detailed description.
[0055] In some embodiments, the embodiments of the present application also automatically generate specific scene content and present specific scene content based on at least the real environment information, interaction information, interaction posture and language information by identifying the real environment information where the target object and the interactive object are located, the interaction information of the target object and the interactive object, the interaction posture corresponding to each of the target object and the interactive object, and the language information of the user. In some embodiments, the aforementioned specific scene content may include content such as storylines, game plots, or prompts and derivations in teaching. Among them, presenting specific scene content can also be through one or more methods such as image pictures, sound effects, light effects or special effects.
[0056] In other words, the embodiments of the present application can automatically generate, for example, storylines, game plots, etc., by identifying multiple inputs such as the real environment in the interaction of physical props, the language of the user, the posture of the physical props, and the interactive relationship, and present the corresponding plots in the form of images or videos. In other words, the user's speech or dialogue is simultaneously regarded as a chat, action, and scene change, and the scene description seen by the protagonist is automatically extracted from the perspective of the protagonist, and presented in the form of images or videos.
[0057] For example, in an exemplary scenario, a child controls a toy car to play in the living room. The car will avoid furniture obstacles or run according to the undulations of the terrain. The child will control the car to move forward and the real environment of the background rolling, etc., is presented through, for example, images or videos. Alternatively, if the scene described by the child through language is a jungle, a jungle picture will be displayed on the screen. Alternatively, for a series of jungle stories, the description or identification of characters in the jungle or the interactive relationship, posture and other information combined with physical props, real environment, etc., is also presented in real time by converting into images or videos. Alternatively, a complete storyline or game plot is automatically generated based only on the words, fragments, interactive relationships between characters, real environment or children's imagined environment described by the child, and presented in real time through images or videos.
[0058] Specifically, based on multimodal inputs such as the aforementioned interactive information, language information, and real environment, recognition and analysis can be performed using, for example, a large language model to output the aforementioned specific scene content and present it through, for example, images, sound effects, lighting effects, or special effects.
[0059] Combined with the above description, it can be seen that the embodiment of the present application focuses on the user's interaction with the physical props in the physical space. The target behavior actions determined by the identified multiple interaction types (such as single gestures, double gestures, voice or props, POI and their combination) can present the behavioral intentions given by the user to the physical props, so as to enhance the user's exploration interest and perception ability. Further, the embodiment of the present application sets the interaction priority of the physical props so that each interactive object can respond to the target object (i.e., the protagonist) in a specific scene imagined by the user's autonomous interaction. In addition, the embodiment of the present application also automatically generates and presents specific scene content by identifying multimodal inputs such as interactive information, language information, and real environment, thereby providing users with an immersive interactive experience, establishing the perception ability of users (especially children), and greatly enhancing the cultivation of children's ability and literacy in language, creativity, or sense of numbers.
[0060] Figure 3 FIG. 1 is an exemplary schematic diagram showing a target selection gesture according to an embodiment of the present application. Figure 3As shown in the figure, the thumb and index finger can be in a pinching state as the target selection gesture 301. In response to identifying the physical prop selected by the target selection gesture, it is determined as the target object, for example, the pinched physical prop shown in the figure is the target object 302. It can be understood that Figure 3 This is merely exemplary and not restrictive. For example, the thumb, index finger, and middle finger may be in a pinching state as a target selection gesture. Alternatively, the thumb and the remaining four fingers may be in a pinching state as a target selection gesture. As can be seen from the foregoing, by pinching the target object to move it, the movement or tracking of the physical prop can be achieved. Alternatively, by pinching the target object and stopping, the menu bar can be triggered.
[0061] Figure 4 is an exemplary schematic diagram showing a target behavior action and its behavior intention for a target object in a game scene or a story scene according to an embodiment of the present application. Figure 4 The target interaction types in the game scene or story scene are shown as examples, including single gesture, double gesture, voice, props, POI and their combination. Different basic gestures, voice or prop postures can form different behavioral actions for the target object and present different behavioral intentions. It can be understood that Figure 4 This is merely exemplary and not limiting. In some embodiments, the aforementioned behavior intention may be presented through, for example, image screens, sound effects, light effects, or special effects to enhance the user's interest in exploration.
[0062] Figure 5 is an exemplary schematic diagram showing the interaction priority of a physical prop according to an embodiment of the present application. Figure 5 The example shows how to judge the interaction priority based on the roll call response, intention information, historical conversation data or distance information, etc., or by whether the face-to-face target object and / or interactive object moves or whether a conversation occurs. Specifically, for the target object (protagonist), when the protagonist is recognized to roll the name, the NPC being called responds; when it is recognized that the protagonist has no roll call but has an intended object, the intended object NPC responds; when it is recognized that the protagonist has no roll call and no intended object, but has a previous round of conversation, the previous round of conversation NPC responds; when it is recognized that the protagonist has no roll call, no intended object and has a previous round of conversation, the nearest NPC responds. In addition, when the protagonist comes in front of an NPC and speaks or does not speak, the protagonist and the NPC move separately, and when the protagonist comes in front of an NPC and speaks or does not speak, the NPC opposite directly responds or starts a conversation.
[0063] Furthermore, for the interactive object (NPC), if you pinch the NPC to talk and do not move, the NPC will continue to answer. If the user pinches the NPC to move, but does not speak or does not speak, the NPC opposite will answer or the NPC will continue to answer. Or, if the user pinches two NPCs to move, but does not speak or does not speak, the NPC closest to the protagonist will answer. It can be understood that Figure 5 This is only exemplary and not limiting. For example, in some embodiments, the interaction priority may be determined in combination with content such as role relationships and plots.
[0064] Figure 6 1 is an exemplary structural block diagram showing an apparatus 100 for interacting with physical props according to an embodiment of the present application. Figure 6 As shown in , the device 100 may include at least a recognition unit 101 and a display unit 102. In some embodiments, the recognition unit 101 may be used to determine, within the recognition range, in response to the recognized target selection gesture, the physical prop selected based on the target selection gesture as the target object. The display unit 102 may be used to determine, in response to the recognized target interaction type, the target behavior action of the target interaction type acting on the target object and present the behavior intention of the target object according to the target behavior action.
[0065] As mentioned above, the aforementioned recognition unit may include but is not limited to a camera, a laser radar or other sensors with recognition and perception functions (such as voice recognition), detectors, etc. The aforementioned display unit may include but is not limited to a screen device (such as a display screen or projection, etc.), a sound effect device (such as voice output), a light effect device (such as light output) or a special effect (such as vibration, blowing, bubbles, etc.) display device, etc. Among them, for more details on the specific arrangement and interaction of the device and the physical props, please refer to the above Figure 1-Figure 5 The description of is not repeated here in this application.
[0066] Figure 7 is an exemplary structural block diagram showing a device 700 for interacting with physical props according to an embodiment of the present application. Figure 7 As shown in , the device 700 of the present application may include a processor 701 and a memory 702, wherein the processor 701 and the memory 702 communicate with each other via a bus. The memory 702 stores program instructions for physical prop interaction, and when the program instructions are executed by the processor 701, the method steps described in the above text in combination with the accompanying drawings are implemented: within the recognition range, in response to the recognized target selection gesture, the physical prop selected based on the target selection gesture is determined as the target object; in response to the recognized target interaction type, the target behavior action of the target interaction type acting on the target object is determined; and the behavior intention of the target object is presented according to the target behavior action.
[0067] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by software programs. Therefore, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores computer-readable instructions for physical prop interaction. When the computer-readable instructions are executed by one or more processors, the present application in combination with the accompanying drawings can be implemented. Figure 2 The described method for entity prop interaction.
[0068] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0069] It should be noted that although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.
[0070] It should be understood that when the terms "first", "second", "third" and "fourth" are used in the claims, the specification and the drawings of the present application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.
[0071] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.
[0072] Although the implementation methods of the present application are as above, the contents described are only examples adopted to facilitate the understanding of the present application, and are not intended to limit the scope and application scenarios of the present application. Any technician in the technical field described in the present application can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present application, but the scope of patent protection of the present application shall still be subject to the scope defined in the attached claims.
Claims
1. A method for physical prop interaction, comprising: In response to the recognized target selection gesture, within the recognition range, the physical prop selected based on the target selection gesture is determined as the target object; In response to the identified target interaction type, determining a target behavior action of the target interaction type acting on the target object; as well as The behavior intention of the target object is presented according to the target behavior action. 2 . The method of claim 1 , wherein the target selection gesture comprises at least two fingers in a pinch state.
3. The method according to claim 1, wherein the target interaction type includes at least one or more of the following: Single-target interaction gestures, dual-target interaction gestures, target interaction voice or prop interaction gestures; Single-target interactive gestures and target interactive voice; Dual-target interactive gestures and target interactive voice; Single-target interaction gestures, target interaction voice, and prop interaction postures; or Dual-target interaction gestures, target interaction voice, and prop interaction postures.
4. The method according to claim 3, wherein the single-target interaction gesture and the dual-target interaction gesture are implemented by one or more basic gestures, and the basic gestures include at least one or more of pointing, grasping, pinching, making a fist, closing or crossing of fingers or palms; the prop interaction posture includes at least one or more of standing, lying flat, rolling, moving or lying prone; and the target interaction voice includes the user's language recognized in real time.
5. The method according to claim 1, further comprising: In response to recognizing that the target object is selected by the target selection gesture and stays at a target position for a target time, a menu bar related to the target object is triggered.
6. The method according to claim 1, further comprising: In response to the recognition range including a plurality of physical props, determining the physical props other than the target object as interactive objects; as well as In response to moving the target object to the vicinity of the interactive object, the target object is caused to interact with the interactive object.
7. The method according to claim 6, wherein the target object interacts with the interactive object by the following operations: identifying target interaction information; and The interaction priority of the interaction between the interaction object and the target object is determined based on the target interaction information, so as to enable the target object to interact with the interaction object in an interaction priority order.
8. The method according to claim 7, wherein the target interaction information includes at least roll call response, intention information, historical conversation data or distance information.
9. The method according to claim 8, wherein the interaction priority of the interaction between the interactive object and the target object is determined by the following operations: In response to the identified target interaction information including a named response, determining the interaction priority of the interactive object that is named and responds as the highest priority; In response to the identified target interaction information not including the roll call response but including the intention information, determining the interaction priority of the interaction object with the intention as the highest priority; In response to the identified target interaction information not including the roll call response and the intention information but including the historical conversation data, determining the interaction priority of the interaction object having the historical conversation with the target object as the highest priority; or In response to the identified target interaction information only including distance information, the interaction priority of the interaction object closest to the target object is determined as the highest priority.
10. The method according to claim 6, further comprising: In response to recognizing that the target object is moved near the target interactive object and a conversation is taking place, determining that the target interactive object directly answers; In response to identifying that the target object is moved to the vicinity of the target interactive object without a conversation, determining that the target interactive object starts a conversation; In response to recognizing that the target interactive object has not moved or that the target interactive object has moved and is in conversation, determining that the target interactive object continues to answer; In response to recognizing that the target interactive object moves to be a conversation, determining that a nearby interactive object answers; or In response to recognizing that any two target interactive objects move and are in conversation or not in conversation, it is determined that a target interactive object closest to the target object answers.
11. The method according to claim 1 or 7, further comprising: Identify the real environment information where the target object and the interactive object are located, the interactive information between the target object and the interactive object, the interactive postures corresponding to the target object and the interactive object, and the language information of the user; as well as Automatically generate specific scene content and present the specific scene content based on at least the real environment information, the interaction information, the interaction gesture and the language information.
12. The method according to claim 11, wherein the manner of presenting the behavioral intention of the target object or presenting the specific scene content at least includes image pictures, sound effects or light effects.
13. The method according to any one of claims 1-10, wherein the method is at least oriented to child users, and the method is at least applied in a children's game scene, a children's story scene or a children's teaching scene.
14. The method according to claim 1, wherein the physical props at least include physical toys for children or physical teaching aids for children.
15. A device for interacting with physical props, comprising: A recognition unit, which is used to determine, within a recognition range, in response to a recognized target selection gesture, a physical prop selected based on the target selection gesture as a target object; A display unit for: In response to the identified target interaction type, determining a target behavior action of the target interaction type acting on the target object; and The behavior intention of the target object is presented according to the target behavior action.
16. The device according to claim 15, wherein the recognition unit at least includes a camera; and the display unit at least includes a screen device, a sound effect device or a light effect device.
17. A device for interacting with physical props, comprising: processor; as well as A memory storing program instructions for interaction with physical props, wherein when the program instructions are executed by the processor, the device implements the method according to any one of claims 1 to 14.
18. A computer-readable storage medium storing computer-readable instructions for interaction with physical props, wherein when the computer-readable instructions are executed by one or more processors, the method according to any one of claims 1 to 14 is implemented.
