Scene interaction method, terminal equipment and computer readable storage medium

Abstract

The invention discloses a scene interaction method, terminal equipment and a computer readable storage medium, and the scene interaction method responds to the situation that a shooting scene of the terminal equipment is a preset three-dimensional space, and at least one first virtual model is displayed in a picture of the three-dimensional space; in response to the trigger signal, selecting a target virtual model from the at least one first virtual model, and generating a second virtual model in the picture of the three-dimensional space based on the target virtual model; and obtaining a motion parameter of the terminal device, and adjusting a display state of the target virtual model and / or the second virtual model in the picture based on the motion parameter. According to the method, the target virtual model is selected from the multiple first virtual models according to the trigger signal, the display state of the target virtual model and / or the second virtual model in the picture is adjusted according to the motion parameters of the terminal equipment, the trigger signal and the motion parameters can be adjusted by a user, and the user experience is improved. The interaction between a user and a virtual model is improved, and the interestingness of scene interaction is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a scene interaction method, a terminal device, and a computer-readable storage medium. Background Art

[0002] Augmented Reality (AR) is a technology that cleverly integrates virtual information with the real world. It widely uses a variety of technical means such as multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing. AR simulates computer-generated text, images, 3D models, music, and videos, and applies them to the real world, so that the two types of information complement each other, thereby achieving "enhancement" of the real world.

[0003] With the development of augmented reality technology, we hope that one day it can be applied to scene interaction to provide convenience in life. The current scene interaction method is single and the user's interactivity needs to be improved. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present application provides a scene interaction method, a terminal device and a computer-readable storage medium.

[0005] To solve the above problems, the present application provides a scene interaction method based on augmented reality, which is applied to a terminal device, including: in response to the shooting scene of the terminal device being a preset three-dimensional space, displaying at least one first virtual model in the picture of the captured three-dimensional space; in response to a trigger signal, selecting a target virtual model from the at least one first virtual model, and displaying a second virtual model in the picture of the three-dimensional space based on the target virtual model; obtaining motion parameters of the terminal device, and adjusting the display status of the target virtual model and / or the second virtual model in the picture based on the motion parameters.

[0006] Therefore, a target virtual model can be selected from multiple first virtual models according to a trigger signal, and the display status of the target virtual model and / or the second virtual model in the screen can be adjusted according to the motion parameters of the terminal device. The trigger signal and the motion parameters can be adjusted by the user to improve the interactivity between the user and the virtual model and increase the fun of the scene interaction.

[0007] In one embodiment, the step of adjusting the display state of the target virtual model and / or the second virtual model in the screen based on the motion parameters includes: when the motion parameters meet the first preset state switching condition, adjusting the second virtual model to the first display state; the method also includes: displaying the third virtual model in the screen based on the first display state, wherein the third virtual model and the target virtual model correspond to different states of the same physical object.

[0008] Therefore, when the motion parameters meet the first preset state switching condition, the display state of the second virtual model is adjusted to enable the user to interact with the second virtual model, and the third virtual model is displayed based on the first display state, thereby improving the fun of the scene interaction process.

[0009] In one embodiment, the motion parameter includes at least one of an angle change parameter and a speed change parameter, and the first preset state switching condition includes: the angle change parameter is not greater than the first preset parameter, or the pitch angle of the terminal device represented by the angle change parameter in the three-dimensional space decreases from greater than the first preset parameter to less than the first preset parameter, or the pitch angle of the terminal device represented by the angle change parameter in the three-dimensional space decreases from greater than the first preset parameter to less than the first preset parameter and the speed change parameter is greater than the second preset parameter.

[0010] Therefore, the user determines whether the motion parameters meet the first preset state switching condition through the angle change parameters and / or speed change parameters to adjust the display state of the second virtual model, thereby realizing the interaction between the user and the second virtual model and improving the fun of the scene interaction process.

[0011] In one embodiment, the motion parameters include at least one of an angle change parameter and a speed change parameter. The step of adjusting the display state of the target virtual model and / or the second virtual model in the screen based on the motion parameters includes at least one of the angle change parameter and the speed change parameter. The step includes: when the motion parameters meet the second preset state switching condition, adjusting the second virtual model to the second display state; the second preset state switching condition includes: the pitch angle of the terminal device represented by the angle change parameter in the three-dimensional space increases from not more than the first preset parameter to greater than the first preset parameter, or the pitch angle of the terminal device represented by the angle change parameter in the three-dimensional space increases from not more than the first preset parameter to greater than the first preset parameter and the speed change parameter is greater than the second preset parameter; the display position and / or display orientation represented by the first display state and the second display state are different.

[0012] Therefore, the user determines whether the motion parameters meet the second preset state switching condition through the angle change parameters and / or speed change parameters to adjust the display state of the second virtual model, thereby realizing the interaction between the user and the second virtual model and improving the fun of the scene interaction process.

[0013] In one embodiment, the step of adjusting the display state of the target virtual model and / or the second virtual model in the picture based on the motion parameter further includes: hiding the target virtual model.

[0014] Therefore, the user adjusts the display state of the second virtual model by changing the angle change parameter and the speed change parameter, thereby improving the accuracy of the second virtual model and realizing the interaction between the user and the second virtual model.

[0015] In one embodiment, the method further includes: based on the built-in interactive content file of the target virtual model, displaying the interactive content used to represent the interactive operation result of the target virtual model in combination with the third virtual model.

[0016] Therefore, by hiding the target virtual model, the position of the original target display model is updated to the display of the third virtual model, so as to reduce the visual confusion of the target virtual model to the user when generating the third virtual model, thereby improving the user's visual perception.

[0017] In one embodiment, before the step of selecting a target virtual model from at least one first virtual model in response to a trigger signal, the method also includes: displaying a ray model in a three-dimensional space screen; determining a position of the ray model based on motion parameters of a terminal device; and selecting a target virtual model from at least one first virtual model in response to the trigger signal, including: in response to the trigger signal, obtaining a relative position relationship between the position of the ray model and at least one first virtual model, and determining the first virtual model to which the ray in the ray model points as the target virtual model based on the relative position relationship.

[0018] Therefore, the user can determine the first virtual model pointed to by the ray in the ray model as the target virtual model based on the relative position relationship between the current position of the ray model and at least one first virtual model, thereby realizing interaction between the user and the first virtual model and improving the fun of the scene interaction process.

[0019] In one embodiment, before the step of selecting a target virtual model from at least one first virtual model in response to a trigger signal, the method also includes: obtaining an angle change parameter of the terminal device and determining a motion trajectory corresponding to the angle change parameter in the three-dimensional space; selecting a target virtual model from at least one first virtual model in response to the trigger signal, including: selecting the target virtual model based on the motion trajectory and the distance parameters of each adjacent first virtual model in at least one first virtual model in response to the trigger signal.

[0020] Therefore, by adjusting the angle parameter, a first virtual model is selected from at least one first virtual model as a target virtual model, so that the interaction between the user and the first virtual model is achieved, thereby improving the fun of the scene interaction process.

[0021] In one embodiment, before the step of displaying at least one first virtual model in the captured three-dimensional space image, the scene interaction method also includes: obtaining environmental parameter information of the scene captured by the terminal device, and constructing a three-dimensional space model based on the environmental parameter information; setting multiple identification positions in the three-dimensional space model, and the identification positions are used to identify the positions of each first virtual model in the three-dimensional space model.

[0022] Therefore, a three-dimensional space model is generated by acquiring the environmental parameter information of the shooting scene, and multiple identification positions are set in the three-dimensional space model as the placement positions of the first virtual model, so that when shooting subsequently through the terminal device, at least one first virtual model can be displayed in the captured three-dimensional space picture.

[0023] In one embodiment, the step of displaying at least one first virtual model in the captured three-dimensional space image further includes: acquiring an environmental image of the shooting scene of the terminal device and identifying feature information of the environmental image; matching the feature information of the environmental image with the feature information of the identification position to obtain a feature area whose match with the identification position is greater than a preset threshold; and displaying at least one first virtual model in the feature area.

[0024] Therefore, the identification position is determined by identifying the characteristic information of the environmental image, thereby improving the accuracy of the scene display.

[0025] To solve the above problems, the present application provides a second technical solution: a terminal device is provided, the terminal device includes a processor and a memory connected to the processor, wherein program data is stored in the memory, and the processor calls the program data stored in the memory to execute the scene interaction method as described above.

[0026] To solve the above problem, the present application provides a third technical solution: providing a computer-readable storage medium storing program instructions, wherein the program instructions are executed to implement the scene interaction method as described above.

[0027] Compared with the prior art, the scene interaction method of the present application includes: in response to the shooting scene of the terminal device being a preset three-dimensional space, displaying at least one first virtual model in the screen of the three-dimensional space; in response to a trigger signal, selecting a target virtual model from at least one first virtual model, and generating a second virtual model in the screen of the three-dimensional space based on the target virtual model; obtaining the motion parameters of the terminal device, and adjusting the display state of the target virtual model and / or the second virtual model in the screen based on the motion parameters. Through the method of the present application, a target virtual model is selected from multiple first virtual models according to a trigger signal, and the display state of the target virtual model and / or the second virtual model in the screen is adjusted according to the motion parameters of the terminal device. The trigger signal and the motion parameters can be adjusted by the user to improve the interactivity between the user and the virtual model and improve the fun of the scene interaction.

[0028] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0030] Figure 1 This is a flowchart of the first embodiment of the scene interaction method provided by the present application;

[0031] Figure 2 It is an operation diagram of an embodiment of the scene interaction method provided by the present application;

[0032] Figure 3 This is a flow chart of a second embodiment of the scene interaction method provided by the present application;

[0033] Figure 4 It is a flowchart of the third embodiment of the scene interaction method provided by the present application;

[0034] Figure 5 It is a schematic diagram of a framework of an embodiment of a terminal device provided by the present application;

[0035] Figure 6 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0038] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0039] The present application relates to the field of augmented reality, which obtains image information of target objects in a real environment, and then uses various vision-related algorithms to detect or identify the relevant features, states and attributes of the target objects, so as to obtain an AR effect that combines virtuality and reality and matches the specific application. Exemplarily, the target object may involve face, limbs, gestures, movements, etc. related to the human body, or markers and landmarks related to objects, or target areas or target items related to venues or places. Vision-related algorithms may involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, key point extraction and tracking of objects, object posture or depth detection, etc. For example, specific applications may involve interactive scenes such as publicity, marketing, games, interactions, virtual effect overlay display, virtual model display, etc. related to real scenes or objects.

[0040] The convolutional neural network can be used to detect or identify the relevant features, states and attributes of the target object. The above convolutional neural network is a network model obtained by model training based on a deep learning framework.

[0041] Based on the above technical foundation, this application proposes a scene interaction method based on augmented reality, see Figure 1 , 2 , Figure 1 is a flowchart of the first embodiment of the scene interaction method provided by this application, Figure 2 1 is an operation diagram of an embodiment of the scene interaction method provided by the present application. Specifically, the scene interaction method of the embodiment of the present application may include the following steps S101 to S103:

[0042] Step S101: In response to a shooting scene of a terminal device being a preset three-dimensional space, at least one first virtual model is displayed in a picture of the shot three-dimensional space.

[0043] The first virtual model may include a model that needs to interact with the user. For example, the model can be selected according to the interactive scene. For example, in an interactive scene with marketing needs such as a shopping mall, the first virtual model can be selected based on the interactive form or marketing content. For example, when the interactive form is smashing a golden egg or a golden egg lottery, the first virtual model can be a golden egg model; when the interactive form is a balloon game, the first virtual model can be a balloon model; here, the first virtual model is not specifically limited. The three-dimensional space is generated when the terminal device shoots a preset real scene. The three-dimensional space can be related to the interactive scene. For example, when the interactive form is smashing a golden egg or a golden egg lottery, the three-dimensional space can be generated by the terminal device shooting a shopping mall, outdoor square, etc. with a relatively open space. There is no specific limitation here.

[0044] Among them, displaying at least one first virtual model in a three-dimensional space screen can be understood as displaying the first virtual model integrated into the real scene in the terminal device. It can be to directly render the presentation content of the first virtual model and merge it with the real scene, and its display effect is the first virtual model placed in the real scene. It can also be to merge the presentation content of the first virtual model with the real scene screen and display the fused display screen.

[0045] When the shooting scene of the terminal device is a preset three-dimensional space, at least one first virtual model is displayed at a designated position in the picture of the shot three-dimensional space.

[0046] Step S102: In response to a trigger signal, a target virtual model is selected from at least one first virtual model, and a second virtual model is displayed in a three-dimensional space screen based on the target virtual model.

[0047] In one embodiment, the trigger signal can be generated based on the motion parameters of the terminal device. Exemplarily, the motion parameters of the terminal device include parameters such as the rotation angle, movement trajectory, and speed of the terminal device. In another embodiment, the trigger signal can also be generated based on an operation instruction made to the first virtual model. Exemplarily, the operation instruction can be an instruction such as clicking or sliding on the first virtual model. In other embodiments, a trigger signal can also be generated when a preset voice message is recognized to select a target virtual model from at least one first virtual model. It can be understood that the trigger signal is generated based on the interaction behavior between the user and the first virtual model or the terminal device. Here, the trigger signal is not specifically limited.

[0048] The second virtual model can be selected based on the interactive scene. For example, in an interactive scene with marketing needs such as a shopping mall, the second virtual model can be selected based on the interactive form or marketing content, such as Figure 2As shown, when the interactive form is smashing a golden egg or drawing a golden egg, the first virtual model a can be a golden egg model, the second virtual model b can be a hammer model, and a target virtual model c is selected from at least one first virtual model a. It can be understood that the first virtual model a and the target virtual model c here are both golden egg models, but they are golden egg models in different states; when the interactive form is a card drawing game, the first virtual model can be a card box model, and the second virtual model can be a card; here, the second virtual model is not specifically limited.

[0049] In response to a trigger signal, a target virtual model is selected from at least one first virtual model according to a user's requirement, and a second virtual model is displayed in a three-dimensional space based on the target virtual model.

[0050] Step S103: acquiring motion parameters of the terminal device, and adjusting the display state of the target virtual model and / or the second virtual model in the picture based on the motion parameters.

[0051] When the user moves the terminal device, the motion parameters are obtained from the terminal device, and the display state of the target virtual model and / or the second virtual model in the screen is adjusted according to the motion parameters. Exemplarily, the motion parameters include speed parameters, speed change parameters, movement angle parameters, angle change parameters and other parameters. Among them, the display state of the target virtual model and / or the second virtual model may include the display position of the target virtual model and / or the second virtual model in the screen, the display angle, the distance between the target virtual model and the second virtual model, or the display image of the target virtual model and / or the second virtual model.

[0052] Optionally, in one embodiment, after selecting a target virtual model from at least one first virtual model, other first virtual models in the at least one first virtual model may be deleted, and only the target virtual model may be displayed in the three-dimensional space screen; in other embodiments, after selecting a target virtual model from at least one first virtual model, a second virtual model may be directly displayed on one side of the target virtual model in the three-dimensional space screen without changing other first virtual models.

[0053] It is understandable that the motion parameters can be used to simulate the user's interactive behavior, so that the display state of the target virtual model and / or the second virtual model is adjusted in three-dimensional space according to the user's interactive behavior. For example, in interactive scenarios for marketing needs such as shopping malls, when the interactive form is smashing golden eggs or golden egg draws, after selecting the target golden egg model from at least one golden egg model and generating a hammer model, the terminal device can be moved to simulate the action of lifting or falling the hammer model, so that the hammer model is smashed at the golden egg model when the hammer model falls, to achieve an interactive scene of smashing the golden egg. In other interactive scenarios, such as in scenes such as capsule draws, the capsule action can be simulated by a mobile terminal device. Here, there is no specific limitation on the scenario for adjusting the display state, and the corresponding adjustment method can be selected according to the interactive needs.

[0054] The motion parameters may be obtained through sensors built into the terminal device or electronic devices connected to the terminal device. For example, the terminal device includes but is not limited to AR glasses, tablet computers, smart phones, etc., the sensors include but are not limited to acceleration sensors, angular motion detection sensors, image sensors, etc., and the electronic devices include but are not limited to portable devices such as electronic bracelets.

[0055] In the above-mentioned embodiment, a target virtual model can be selected from multiple first virtual models according to a trigger signal, and the display status of the target virtual model and / or the second virtual model in the screen can be adjusted according to the motion parameters of the terminal device. The trigger signal and the motion parameters can be adjusted by the user to improve the interactivity between the user and the virtual model, and enhance the immersion and fun of the scene interaction.

[0056] See also Figure 3 , Figure 3 201 to 204:

[0057] Step S201: In response to the shooting scene of the terminal device being a preset three-dimensional space, at least one first virtual model is displayed in the picture of the shot three-dimensional space.

[0058] Step S202: In response to a trigger signal, a target virtual model is selected from at least one first virtual model, and a second virtual model is displayed in a three-dimensional space screen based on the target virtual model.

[0059] Steps S201-S202 are similar to steps S101-S102 and will not be described again herein.

[0060] Step S203: When the motion parameter satisfies the first preset state switching condition, the second virtual model is adjusted to the first display state.

[0061] Specifically, the motion parameters include but are not limited to speed parameters, speed change parameters, movement angle parameters, angle change parameters, etc. When the motion parameters meet the first preset state switching condition, the second virtual model can be adjusted to the first display state. The first display state of the second virtual model can be set according to the user's interactive purpose.

[0062] For example, in interactive scenes for marketing needs such as shopping malls, when the interactive form is smashing golden eggs or golden egg lottery, after selecting a target golden egg model from at least one golden egg model and generating a hammer model, the mobile terminal device, after the motion parameters of the terminal device meet the first preset state switching condition, at this time, the hammer model falls and presents a state of smashing the golden egg model, or the hammer model is displayed on the golden egg model, that is, the hammer model is in the first display state. In other interactive scenes, the second virtual model can be a model of other objects, and the first display state can be set according to the interactive needs of the second virtual model and the target virtual model, which is not specifically limited here.

[0063] Wherein, after step S202 or step S203, the scene interaction method may further include: displaying a third virtual model in the screen based on the first display state, wherein the third virtual model and the target virtual model correspond to different states of the same physical object.

[0064] When the second virtual model is in the first display state, a third virtual model is generated at the position of the target virtual model, wherein the third virtual model is a virtual object model or effect model generated based on the interaction between the second virtual model and the target virtual model.

[0065] For example, in interactive scenes for marketing needs such as shopping malls, when the interactive form is smashing a golden egg or a golden egg lottery, the hammer model is the second virtual model, the target virtual model is the selected golden egg model, and when the hammer model is in the first display state, the hammer model smashes the golden egg model. At this time, in order to enhance the fun, an egg smashing special effect or a golden egg model after the egg smashing, i.e., the third virtual model, can be generated at the position of the target golden egg model. In other interactive scenes, the third virtual model can select other virtual objects or effects according to the interactive needs, which are not specifically limited here.

[0066] In the above embodiment, when the motion parameters meet the first preset state switching condition, the display state of the second virtual model is adjusted to enable the user to interact with the second virtual model, and the third virtual model is displayed based on the first display state, thereby improving the fun of the scene interaction process.

[0067] Optionally, the motion parameter includes at least one of an angle change parameter and a speed change parameter, and the display state of the second virtual model is adjusted according to a size of at least one of the angle change parameter and the speed change parameter of the terminal device within a preset time.

[0068] Specifically, the angle change parameter is the change vector of the angle of the terminal device with the horizontal plane within the preset time. The angle change parameter includes the change value and the change direction. The speed change parameter is the change value of the speed of the terminal device within the preset time. The speed change parameter can be but is not limited to acceleration or variable acceleration.

[0069] Further, step S103 includes: when the angle change parameter and / or the speed change parameter meet the first preset state switching condition, adjusting the second virtual model to the first display state, and displaying the third model at the display position of the target virtual model. Wherein, the first preset state switching condition includes: the pitch angle of the terminal device represented by the angle change parameter in the three-dimensional space decreases from greater than the first preset parameter to not greater than the first preset parameter, or the pitch angle of the terminal device represented by the angle change parameter in the three-dimensional space decreases from greater than the first preset parameter to not greater than the first preset parameter and the speed change parameter is greater than the second preset parameter.

[0070] Through the above method, the terminal device can determine whether the motion parameters meet the first preset state switching condition through the angle change parameters and / or speed change parameters, so as to adjust the display state of the second virtual model, realize the interaction between the user and the second virtual model, and improve the fun of the scene interaction process.

[0071] Further, step S103 includes: when the motion parameter satisfies the second preset state switching condition, adjusting the second virtual model to the second display state. The second preset state switching condition includes: the pitch angle of the terminal device represented by the angle change parameter in the three-dimensional space increases from not more than the first preset parameter to more than the first preset parameter, or the pitch angle of the terminal device represented by the angle change parameter in the three-dimensional space increases from not more than the first preset parameter to more than the first preset parameter and the speed change parameter is greater than the second preset parameter. The display position and / or display orientation represented by the first display state and the second display state are different.

[0072] Through the above method, the terminal device can determine whether the motion parameters meet the second preset state switching conditions through the angle change parameters and / or speed change parameters, so as to adjust the display state of the second virtual model, realize the interaction between the user and the second virtual model, and improve the fun of the scene interaction process.

[0073] Specifically, the first display state and the second display state of the second virtual model at least include different display positions. For example, in interactive scenes for marketing needs such as shopping malls, when the interactive form is smashing golden eggs or golden egg lottery, after selecting the target golden egg model from at least one golden egg model and generating a hammer model, the mobile terminal device, when the angle change parameter and / or the speed change parameter meet the first preset state switching condition, the first display state of the second virtual model is the hammer model in the first display position, and the first display position is any position when the hammer model falls; when the angle change parameter and / or the speed change parameter do not meet the second preset state switching condition, the second display state of the second virtual model is the hammer model in the second display position, and the second display position is any position when the hammer model is lifted.

[0074] The first display state of the second virtual model at least includes displaying at a first position, and the second display state at least includes displaying at a second position, and the first position is different from the second position. In a possible implementation, the speed change parameter and the angle change parameter are both vector parameters, and the first preset parameter and the second preset parameter can be set to 0, or other preset values, which are not specifically limited here.

[0075] Optionally, different display effects may be included between the first display state and the second display state. For example, when the second virtual model is in the first display state, the hammer model falls and appears to be hitting the golden egg model; when the second virtual model is in the second display state, the hammer model is lifted and appears to be accumulating power.

[0076] Further, step S103 may also include the following steps: when the angle change parameter and / or the speed change parameter do not satisfy the first preset state switching condition and the second preset state switching condition, for example, when the speed change parameter is less than or equal to the second preset parameter, the target virtual model may be adjusted to a third display state. For example, at this time, the golden egg model may be shaken to remind the user that the golden egg has not been successfully smashed.

[0077] Optionally, at least one first virtual model includes a built-in file having a one-to-one correspondence therewith, and after the user selects a target virtual model from the at least one first virtual model, the user can view the corresponding project information by viewing the built-in file of the target virtual model.

[0078] Among them, the built-in file includes a built-in interactive content file. After step S204, the scene interaction method also includes: based on the built-in interactive content file of the target virtual model, the interactive content used to represent the interactive operation result of the target virtual model is displayed in combination with the third virtual model. Specifically, the interactive content may include the built-in file of the target virtual model and the interactive terms related to the generation of the third virtual model that can represent the interactive operation result. Exemplarily, in the interactive scene of smashing golden eggs or golden egg lottery, each golden egg model includes a corresponding prize label, and the prize label is written into the built-in file of the target golden egg model. After using the hammer model to smash the target golden egg model, the corresponding interactive content will be displayed in the three-dimensional space based on the built-in file of the target golden egg model and the third virtual model, and the interactive content includes the prize label in the built-in file and the relevant prompts of winning the prize, or, when the built-in file does not include the prize label, the interactive content includes the relevant prompts of not winning the prize.

[0079] In the above implementation, when the third virtual model is generated, the user can view the project information corresponding to the target virtual model by reading the interactive content, thereby improving the user's usage experience.

[0080] Optionally, step S103 further includes: hiding the target virtual model.

[0081] Specifically, after adjusting the display status of the target virtual model and / or the second virtual model, for example, based on the second virtual model in the first display state, the third virtual model is displayed at the display position of the target display model and the target virtual model is hidden, so that the position of the original target display model is updated to the display of the third virtual model, so as to reduce the visual confusion of the target virtual model to the user when generating the third virtual model, thereby improving the user's visual perception.

[0082] Further, in one embodiment, the target virtual model can be selected from at least one first virtual model by using a ray model. Figure 4 , Figure 4 3 is a flow chart of the third embodiment of the scene interaction method provided by the present application. Specifically, before step S102, the scene interaction method of the embodiment of the present application may include the following steps S301 to S302:

[0083] Step S301: Display the ray model in a three-dimensional space.

[0084] After at least one first virtual model is displayed in the three-dimensional space, a ray model is displayed in the three-dimensional space. The ray model is used to indicate the angle at which the user moves the terminal device. The ray model is located within the spatial range from the leftmost first virtual model to the rightmost first virtual model in the at least one first virtual model.

[0085] Step S302: Determine the position of the ray model based on the motion parameters of the terminal device.

[0086] After the motion parameter is obtained, the position of the ray model is set based on the change value of the motion parameter. Exemplarily, the setting position of the ray model is between the leftmost first virtual model and the rightmost first virtual model in at least one first virtual model. For example, when the motion parameter remains unchanged, the ray model is located at the leftmost first virtual model in at least one first virtual model. When the motion parameter gradually increases, the position of the ray model changes accordingly and shifts to the right.

[0087] Wherein, step S102 further includes: in response to a trigger signal, obtaining a relative position relationship between the position of the ray model and at least one first virtual model, and determining the first virtual model pointed to by the ray in the ray model as the target virtual model based on the relative position relationship.

[0088] In response to the trigger signal, the relative position relationship between the current position of the ray model and at least one first virtual model is obtained, so as to select the corresponding first virtual model from the at least one first virtual model as the target virtual model based on the relative position relationship. The trigger signal can be generated based on the motion parameters of the terminal device. For example, the user moves the terminal device with a preset action or a preset trajectory so that its motion parameters meet the preset trigger conditions, and then a trigger signal can be generated. Based on the relative position relationship, it can be determined that the ray in the ray model points to one of the at least one first virtual model, so as to use the pointed first virtual model as the target virtual model.

[0089] Optionally, in order to improve the user experience, multiple interval ranges of relative position relationships can be set, and the multiple intervals correspond to the position of the first virtual model in at least one first virtual model; and the pointing direction of the ray of the ray model can be the direction from the center position of the display module of the terminal device to at least one first virtual model.

[0090] In the above embodiment, the user can determine the first virtual model pointed to by the ray in the ray model as the target virtual model based on the relative position relationship between the position of the ray model and at least one first virtual model, thereby realizing interaction between the user and the first virtual model and improving the fun of the scene interaction process.

[0091] In another embodiment, the target virtual model may be selected from at least one first virtual model by motion parameters. Specifically, before step S102, the scene interaction method of this embodiment may further include: obtaining an angle change parameter of the terminal device, and calculating a motion trajectory corresponding to the angle change parameter in the three-dimensional space.

[0092] Specifically, the angle change parameter can be the angle change value of the terminal device relative to a preset axis. For example, when the interactive scene is smashing a golden egg or drawing a golden egg, the preset axis can be the central axis of the terminal device. When the user moves the terminal device, the target virtual model is selected according to the angle change value between the terminal device and the central axis.

[0093] When calculating the motion trajectory according to the angle change parameter, the leftmost first virtual model in at least one first virtual model is defined as the initial position, and the corresponding motion trajectory is calculated according to the initial position and the pointing position corresponding to the angle change parameter. The motion trajectory is used to indicate the moving trajectory within the screen of the three-dimensional space.

[0094] Wherein, step S102 of the scene interaction method further includes: in response to a trigger signal, selecting a target virtual model based on a motion trajectory and a distance parameter between adjacent first virtual models in at least one first virtual model.

[0095] In response to a trigger signal, based on the motion trajectory and the distance parameters of each adjacent first virtual model in at least one first virtual model, the moving position of at least one first virtual model in the three-dimensional space picture can be obtained, so as to determine the target virtual model from at least one first virtual model according to the moving position.

[0096] In the above implementation, by adjusting the angle change parameter, a first virtual model is selected from at least one first virtual model as a target virtual model, so that the interaction between the user and the first virtual model is achieved, thereby improving the fun of the scene interaction process.

[0097] Optionally, before displaying at least one first virtual model in the captured three-dimensional space image in step S101, the scene interaction method also includes: obtaining environmental parameter information of the scene captured by the terminal device, and constructing a three-dimensional space model based on the environmental parameter information; setting multiple identification positions in the three-dimensional space model, and the identification positions are used to identify the positions of each first virtual model in the three-dimensional space model.

[0098] Specifically, the three-dimensional space model can be obtained by reconstructing a spatial dense model through a visual recognition algorithm and a high-precision environment reconstruction-related algorithm. Before performing scene interaction design, environmental parameter information of the target environment can be collected to construct a three-dimensional space model based on the space of the environmental parameter information.

[0099] For example, before designing scene interactions, spatial images of target spaces (such as shopping malls, guide halls, exhibition halls, etc.) are taken from different shooting angles, and then the image features of the space are identified from the multiple spatial images obtained, and the virtual space is reconstructed using visual recognition algorithms and high-precision environment reconstruction and other related algorithms to obtain a three-dimensional spatial model of the interactive scene. Among them, the number of images taken is greater than 1, for example, it can be 10, 20, 50, etc. The more images there are, the more accurate the constructed three-dimensional spatial model is, and the better the user's augmented reality experience is, but relatively speaking, it takes more time to construct the three-dimensional spatial model.

[0100] After the three-dimensional space model is generated, a plurality of identification positions are set at preset positions in the three-dimensional space model, and the identification positions are used to identify the positions of the first virtual models in the three-dimensional space model. Specifically, the preset positions can be set according to the purpose of the first virtual model. For example, when the first virtual model is used for gaming, in order to facilitate observation of the first virtual model, the preset positions can be set at relatively open positions in the interactive scene, such as tables and chairs in the rest area of ​​a shopping mall. In addition, in order to facilitate identification and positioning, the preset positions can also be iconic locations, such as the entrance of a shopping mall or the entrance of a store. Here, the specific positions of the identification positions are not limited.

[0101] In the above embodiment, a three-dimensional space model is generated by acquiring environmental parameter information of the shooting scene, and multiple identification positions are set in the three-dimensional space model as placement positions of the first virtual model, so that when shooting is subsequently performed through a terminal device, at least one first virtual model is displayed in the captured three-dimensional space picture.

[0102] Optionally, the step of displaying at least one first virtual model in the captured three-dimensional space image further includes: acquiring an environmental image of the shooting scene of the terminal device and identifying feature information of the environmental image; matching the feature information of the environmental image with the feature information of the identification position to obtain a feature area whose matching rate with the identification position is greater than a preset threshold; and displaying at least one first virtual model in the feature area.

[0103] Before collecting environmental images, the spatial position of the user can be located by obtaining the location information of the terminal device. The terminal device is usually carried by the user, and the user's position can be determined by determining the position of the terminal device. The location data of the terminal device can be obtained by receiving the positioning data transmitted by the positioning module. For example, the positioning module can be equipped with a satellite positioning module, an inertial positioning module, or a combination of the two; the satellite positioning module can be a positioning function based on the global positioning system, the Galileo satellite positioning system, the GLONASS satellite positioning system or the Beidou positioning system.

[0104] After locating the spatial position of the user, an image of the environment where the user is located is collected, and the three-dimensional space corresponding to the spatial position of the user is determined from the environmental image. Specifically, feature points of the feature information of the environmental image and the feature information of the marked position are compared to determine whether the spatial position of the user corresponds to the marked position of the three-dimensional space.

[0105] When the matching rate between the feature information of the environment image and the identification position is greater than a preset threshold, the feature area of ​​the environment image can be determined to be the identification position, that is, the spatial position of the user corresponds to the identification position in the three-dimensional space, so as to generate at least one first virtual model at the feature area.

[0106] In the above implementation, the identification position is determined by identifying the characteristic information of the environment image, thereby improving the accuracy of the scene display.

[0107] See also Figure 5 , Figure 5 1 is a schematic diagram of a terminal device according to an embodiment of the present application. The terminal device 100 includes a processor 101 and a memory 102 connected to the processor 101, wherein the memory 102 stores program data, and the processor 101 retrieves the program data stored in the memory 102 to execute the above-mentioned scene interaction method.

[0108] Optionally, in one embodiment, the processor 101 is used to execute program data to implement the following method: in response to the shooting scene of the terminal device being a preset three-dimensional space, displaying at least one first virtual model in the picture of the captured three-dimensional space; in response to a trigger signal, selecting a target virtual model from the at least one first virtual model, and displaying a second virtual model in the picture of the three-dimensional space based on the target virtual model; obtaining motion parameters of the terminal device, and adjusting the display status of the target virtual model and / or the second virtual model in the picture based on the motion parameters.

[0109] The processor 101 may also be referred to as a CPU (Central Processing Unit). The processor 101 may be an electronic chip having the ability to process signals. The processor 101 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0110] The memory 102 can be a memory stick, a TF card, etc., which can store all the information in the terminal device 100, including the input raw data, computer programs, intermediate operation results and final operation results are all stored in the memory 102. It stores and retrieves information according to the location specified by the processor 101. With the memory 102, the terminal device 100 has a memory function and can ensure normal operation. The memory 102 of the terminal device 100 can be divided into main memory (internal memory) and auxiliary memory (external memory) according to its purpose, and there is also a classification method of dividing it into external memory and internal memory. External memory is usually a magnetic medium or an optical disk, etc., which can store information for a long time. Memory refers to the storage component on the motherboard, which is used to store the data and programs currently being executed, but is only used to temporarily store programs and data. If the power is turned off or the power is cut off, the data will be lost.

[0111] The present application also provides another terminal device, which includes a display module, a selection module and an adjustment module.

[0112] The display module is used to display at least one first virtual model in the picture of the captured three-dimensional space in response to the shooting scene of the terminal device being a preset three-dimensional space; the selection module is used to respond to a trigger signal and select a target virtual model from at least one first virtual model, and display a second virtual model in the picture of the three-dimensional space based on the target virtual model; the adjustment module is used to obtain the motion parameters of the terminal device, and adjust the display status of the target virtual model and / or the second virtual model in the picture based on the motion parameters.

[0113] In one embodiment of the present application, each module in the terminal device can be separately or completely combined into one or more units, or one (or more) of the units can be further divided into multiple functionally smaller sub-units to achieve the same operation without affecting the technical effect of the embodiment of the present application. The above modules are divided based on logical functions. In practical applications, the functions of a module can also be implemented by multiple units, or the functions of multiple modules can be implemented by one unit.

[0114] See also Figure 6 , Figure 6 1 is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 110 stores program instructions 111 capable of implementing all the above methods.

[0115] If the integrated units of the functional units in the various embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium 110. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer-readable storage medium 110 includes several instructions in a program instruction 111 to enable a computer device (which can be a personal computer, a system server, or a network device, etc.), an electronic device (such as MP3, MP4, etc., or a mobile terminal such as a mobile phone, a tablet computer, a wearable device, or a desktop computer, etc.) or a processor to execute all or part of the steps of the methods of various implementation methods of the present application.

[0116] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-readable storage media 110 (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by a computer-readable storage medium 110. These computer-readable storage media 110 can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the program instructions 111 executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer-readable storage media 110 may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the program instructions 111 stored in the computer-readable storage medium 110 produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer-readable storage media 110 can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing program instructions 111 executed on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0120] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A scene interaction method based on augmented reality, characterized in that: Applied to a terminal device, including: In response to the shooting scene of the terminal device being a preset three-dimensional space, displaying at least one first virtual model in a picture of the three-dimensional space shot; In response to a trigger signal, selecting a target virtual model from the at least one first virtual model, and displaying a second virtual model within the screen of the three-dimensional space based on the target virtual model; The motion parameters of the terminal device are acquired, and the display status of the target virtual model and / or the second virtual model in the screen is adjusted based on the motion parameters.

2. The scene interaction method according to claim 1, characterized in that: The step of adjusting the display state of the target virtual model and / or the second virtual model in the picture based on the motion parameter comprises: When the motion parameter satisfies a first preset state switching condition, adjusting the second virtual model to a first display state; The method further comprises: A third virtual model is displayed in the screen based on the first display state, wherein the third virtual model and the target virtual model correspond to different states of the same real object.

3. The scene interaction method according to claim 2, characterized in that: The motion parameter includes at least one of an angle change parameter and a speed change parameter, and the first preset state switching condition includes: the pitch angle of the terminal device in the three-dimensional space represented by the angle change parameter decreases from greater than the first preset parameter to no greater than the first preset parameter, or the pitch angle of the terminal device in the three-dimensional space represented by the angle change parameter decreases from greater than the first preset parameter to no greater than the first preset parameter and the speed change parameter is greater than the second preset parameter.

4. The scene interaction method according to claim 2 or 3, characterized in that: The motion parameter includes at least one of an angle change parameter and a speed change parameter, and the step of adjusting the display state of the target virtual model and / or the second virtual model in the picture based on the motion parameter includes: When the motion parameter satisfies a second preset state switching condition, adjusting the second virtual model to a second display state; The second preset state switching condition includes: the pitch angle of the terminal device in the three-dimensional space represented by the angle change parameter increases from not more than the first preset parameter to more than the first preset parameter, or the pitch angle of the terminal device in the three-dimensional space represented by the angle change parameter increases from not more than the first preset parameter to more than the first preset parameter and the speed change parameter is greater than the second preset parameter; The first display state and the second display state represent different display positions and / or display orientations.

5. The scene interaction method according to any one of claims 2 to 4, characterized in that: The step of adjusting the display state of the target virtual model and / or the second virtual model in the picture based on the motion parameter further includes: The target virtual model is hidden.

6. The scene interaction method according to any one of claims 2 to 4, characterized in that: The method further comprises: Based on the built-in interactive content file of the target virtual model, the interactive content used to represent the interactive operation result of the target virtual model is displayed in combination with the third virtual model.

7. The scene interaction method according to claim 1, characterized in that: Before the step of selecting a target virtual model from the at least one first virtual model in response to a trigger signal, the method further includes: Displaying the ray model in the picture of the three-dimensional space; Determining the position of the ray model based on the motion parameters of the terminal device; The step of selecting a target virtual model from the at least one first virtual model in response to a trigger signal comprises: In response to the trigger signal, the relative position relationship between the position of the ray model and the at least one first virtual model is acquired, and based on the relative position relationship, the first virtual model pointed to by the ray in the ray model is determined as the target virtual model.

8. The scene interaction method according to claim 1, characterized in that: Before the step of selecting a target virtual model from the at least one first virtual model in response to a trigger signal, the method further includes: Acquire an angle change parameter of the terminal device, and determine a motion trajectory corresponding to the angle change parameter in the three-dimensional space; The step of selecting a target virtual model from the at least one first virtual model in response to a trigger signal comprises: In response to the trigger signal, the target virtual model is selected based on the motion trajectory and a distance parameter between each adjacent first virtual model in the at least one first virtual model.

9. The scene interaction method according to claim 1, characterized in that: Before the step of displaying at least one first virtual model in the captured image of the three-dimensional space, the scene interaction method further includes: Acquire environmental parameter information of the scene photographed by the terminal device, and construct a three-dimensional space model based on the environmental parameter information; A plurality of identification positions are set in the three-dimensional space model, and the identification positions are used to identify the positions of the first virtual models in the three-dimensional space model.

10. The scene interaction method according to claim 9, characterized in that: The step of displaying at least one first virtual model in the captured image of the three-dimensional space further comprises: Collecting an environmental image of a shooting scene of the terminal device and identifying feature information of the environmental image; Matching the feature information of the environment image with the feature information of the identification position to obtain a feature area whose matching rate with the identification position is greater than a preset threshold; The at least one first virtual model is displayed in the feature area.

11. A terminal device, characterized in that: The terminal device includes a processor and a memory connected to the processor, wherein program data is stored in the memory, and the processor calls the program data stored in the memory to execute the scene interaction method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that: Program instructions are stored, and the program instructions are executed to implement the scene interaction method as described in any one of claims 1-10.

Images