Interaction method and device, storage medium and electronic equipment

Abstract

The invention discloses an interaction method and device, a storage medium and electronic equipment, and relates to the technical field of image processing, and the method comprises the steps: obtaining a target image of interaction between a user and an object in response to an interaction action between the user and the object; determining interaction information of the user and the object based on the target image; generating interaction special effect information of the user and the object and a first three-dimensional model corresponding to the object after interaction according to the target image and the interaction information; performing synthesis processing based on the interaction special effect information and the first three-dimensional model to obtain a target three-dimensional model, the target three-dimensional model being used for displaying an interaction result with the object to a user; compared with the prior art, the interaction between the user and the object can be completed without depending on the interaction event code set by the object, the code quantity of the interaction event set by the object in the application is reduced, meanwhile, the interaction flexibility of the object and the user in the application is improved, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more particularly to an interactive method, apparatus, storage medium, and electronic device. Background Technology

[0002] User interaction with objects in a game is the process by which players manipulate, influence, or respond to virtual objects in the game world through input devices. This interaction can directly affect the user experience and immersion.

[0003] Currently, user interaction with objects in the game is mainly achieved by setting up interactive pop-ups around the object models in the game interface. After the user interacts with the interactive pop-ups, the interactive events bound to the object models are triggered, and interactive animations are played through pre-made interactive animations.

[0004] However, when using this interaction method, if the game program does not bind trigger events to the object models, the user cannot interact with the objects in the game. This results in the user being unable to interact with some objects that are not bound to trigger events during the game, affecting the user experience. Summary of the Invention

[0005] In view of this, this application provides an interaction method, device, storage medium, and electronic device, the main purpose of which is to improve the technical problem in the prior art where, when the game program does not bind trigger events to the object model, the user cannot interact with the objects in the game, resulting in the user being unable to interact with some objects that are not bound to trigger events during the game, thus affecting the user experience.

[0006] Firstly, this application provides an interaction method, including: In response to the user's interaction with the object, acquire the target image of the interaction between the user and the object; The interaction information between the user and the object is determined based on the target image; Based on the target image and the interaction information, generate interaction effect information between the user and the object and a first three-dimensional model of the object after the interaction; Based on the interactive effects information and the first 3D model, a target 3D model is obtained through synthesis processing. The target 3D model is used to display the interaction results with the object to the user.

[0007] Secondly, this application provides an interactive device, comprising: The acquisition module is configured to acquire the target image of the interaction between the user and the object in response to the user's interaction with the object; The determination module is configured to determine the interaction information between the user and the object based on the target image; The generation module is configured to generate interaction effect information between the user and the object and a first three-dimensional model of the object after the interaction, based on the target image and the interaction information. The compositing module is configured to perform compositing processing based on the interactive effect information and the first 3D model to obtain a target 3D model, which is used to display the interaction results with the object to the user.

[0008] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the interaction method described in the first aspect.

[0009] Fourthly, this application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the interaction method described in the first aspect.

[0010] Fifthly, this application provides a computer program product, which includes a computer program that, when executed by a processor, implements the interaction method described in the first aspect.

[0011] By employing the above technical solutions, this application provides an interaction method, device, storage medium, and electronic device. Compared with existing technologies, this application obtains a target image of user-object interaction in response to user-object interaction actions; determines user-object interaction information based on the target image; generates user-object interaction effect information and a first three-dimensional model corresponding to the object after interaction based on the target image and interaction information; and performs synthesis processing based on the interaction effect information and the first three-dimensional model to obtain a target three-dimensional model, which is used to display the interaction result with the object to the user. This allows the application to enable user-object interaction without relying on object-set interaction event code through the synthesized target three-dimensional model, reducing the amount of code for setting object interaction events in the application, while improving the flexibility of object-user interaction in the application and enhancing the user experience. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating an interaction method provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating an interaction method provided in an embodiment of this application is shown; Figure 3 A schematic diagram illustrating an example provided in an embodiment of this application is shown; Figure 4 A schematic diagram illustrating an example provided in an embodiment of this application is shown; Figure 5 A schematic diagram illustrating an example provided in an embodiment of this application is shown; Figure 6 A schematic diagram illustrating an example provided in an embodiment of this application is shown; Figure 7 A schematic diagram illustrating an example provided in an embodiment of this application is shown; Figure 8 A schematic diagram illustrating an example provided in an embodiment of this application is shown; Figure 9 This illustration shows a structural schematic diagram of an interactive device provided in an embodiment of this application; Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0015] The embodiments of this application will now be described in more detail with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0016] To address the technical problem in existing technologies where users are unable to interact with objects in a game when the game program has not bound trigger events to object models, thus affecting the user experience, this embodiment provides an interaction method, such as... Figure 1 As shown, the method includes: Step 101: In response to the user's interaction with the object, acquire the target image of the user's interaction with the object.

[0017] In the embodiments of this application, the user's interaction with an object can be the user's interaction with an object in the application. For example, the interaction can include, but is not limited to, clicking, touching, swiping, etc.

[0018] In some examples, taking games as an example of applications, user interactions with objects can trigger different forms of actions on the objects, such as attack actions, viewing actions, and so on.

[0019] In this embodiment, the target image may include, but is not limited to, an image containing a user, an image containing an object interacting with the user, an image containing both the user and an object interacting with the user, and so on.

[0020] It should be noted that the application containing the object in which the user interacts can be an application within a cloud gaming platform. Based on this, the target image can be a screenshot of the video stream of the user interacting with the object, obtained through the video stream between the application and the user, when the user triggers an interaction with the object, and then the target image can be obtained based on the video stream screenshot.

[0021] Step 102: Determine the interaction information between the user and the object based on the target image.

[0022] In some examples, the user-object interaction information may include, but is not limited to, the user's basic information, the object's basic information, the user's interaction actions, the items used by the user to interact with the object, the environment in which the user interacts with the object, and so on.

[0023] Step 103: Based on the target image and interaction information, generate the interaction effects information between the user and the object and the first three-dimensional model of the object after the interaction.

[0024] In this embodiment of the application, the interactive effect information can be descriptive information of the interactive characteristics, specifically textual descriptions of the interactive effects, such as the color, size, shape, and other information of the interactive characteristics.

[0025] In some examples, the first 3D model of the object after interaction can be the 3D model that the object should be presented to the user after interaction. For example, if the object is a tree and the user's interaction with the object is shaking the tree, then the first 3D model of the object after interaction can be the tree with leaves falling after the user shakes the tree; if the object is a tree and the user's interaction with the object is cutting down the tree, then the first 3D model of the object after interaction can be the tree that falls to the ground after the user cuts down the tree, and so on. No further examples will be given here.

[0026] Step 104: Based on the interactive special effects information and the first 3D model, perform synthesis processing to obtain the target 3D model.

[0027] The target 3D model is used to show users the results of their interaction with the object.

[0028] In this embodiment of the application, a target 3D model containing special effects can be synthesized based on interactive special effects information and a first 3D model.

[0029] For example, if the object is a tree and the user's interaction with the object is shaking the tree, then the first 3D model of the object after the interaction can be the tree with leaves falling after the user shakes the tree, and the target 3D model can be the tree with leaves falling and the shaking effect included. If the object is a tree and the user's interaction with the object is cutting down the tree, then the first 3D model of the object after the interaction can be the tree lying on the ground after the user cuts down the tree, and the target 3D model can be the tree lying on the ground and the cutting down effect included. And so on, and will not be listed in detail here.

[0030] Compared with existing technologies, this embodiment obtains a target image of the user-object interaction in response to the user's interaction with the object; determines the interaction information between the user and the object based on the target image; generates interaction effect information between the user and the object and a first 3D model corresponding to the object after the interaction based on the target image and the interaction information; and performs synthesis processing based on the interaction effect information and the first 3D model to obtain a target 3D model, which is used to display the interaction result with the object to the user. This embodiment enables the user to interact with an object without setting an interaction event through the synthesized target 3D model, realizing the interaction between the user and the object without relying on the interaction event code set by the object. This reduces the amount of code for setting interaction events for objects in the application, while improving the flexibility of the interaction between the object and the user in the application and enhancing the user experience.

[0031] As a refinement and extension of the above embodiments, this embodiment provides an interaction method, such as... Figure 2 As shown, the method includes: Step 201: In response to the user's interaction with the object, acquire the target image of the user's interaction with the object.

[0032] In the embodiments of this application, such as Figure 3 As shown, in response to user interaction with an object, the system first checks the database to see if there is a data record of the object interacting with the user. If it exists, the interaction result can be directly replaced by the interactive video stream output. If it does not exist, an image containing the object can be obtained from a screenshot of the video stream that includes the object, and the main body of the object can be extracted to obtain the main image of the object.

[0033] In some examples, if the user's interaction with the object is an attack action against the object in a game, then... Figure 3As shown, in response to a user's attack action on an object in the game, the system can first check the database for data records of the attacked object being attacked by the user's weapon. If such records exist, the interactive video stream output will be directly replaced with the interactive result. If not, an image including the attacked object can be obtained from a screenshot of the video stream containing the attacked object, and the attacked object can be extracted to obtain the image of the attacked object.

[0034] For example, if the user's interaction with an object is an action of attacking a tree using character 1 in a game, then in response to character 1's attack on the tree in the game, the database can first be checked to see if there are any data records of trees being attacked by weapons used by other game characters. If they exist, the interaction result is directly replaced by the interactive video stream output. If they do not exist, an image including the tree body can be obtained from a screenshot of the video stream that includes the tree, and the tree body image can be obtained by cutting out the tree body image.

[0035] Step 202: Determine the interaction information between the user and the object based on the target image.

[0036] In this embodiment, an AI image recognition model can be called via an interface to input a target image into the AI ​​image recognition model for processing, converting the content contained in the target image into a text description. The training process of the AI ​​image recognition model may include: using object model images, prop images, and similar style object model and prop images in the application with pre-set interaction modes as input models as training set samples, and then inputting object models without pre-set interaction modes as a validation set for verification, ultimately generating image content that can recognize objects in the application and output text descriptions.

[0037] For example, if the user's interaction with an object is an attack on a tree by character 1 using item 1 in a game, the target image can be input into an AI image recognition model for processing. This model can then identify information about the attacked entity, the user's item information, the surrounding environment, and so on. Based on the information about the attacked entity, the user's item information, the surrounding environment, and so on, key features and elements can be extracted, and textual descriptions can be output.

[0038] For example, such as Figure 4 As shown, when a player's game character interacts with objects in the game, such as when a player wields a sword and attacks a tree in the scene, this embodiment of the application can read the attack range of the player's weapon, select objects within the player's attack range, and identify the interacting subject (i.e., such as...). Figure 5 The tree shown) identifies the player's weapon type (i.e. Figure 4 (The sword held by the player in the middle).

[0039] In some examples, the text description output by the AI ​​image recognition model can be further input into the trained AIGC large language model (i.e., the second model in the embodiments of this application). That is, input the text description of the attacked subject and the player's weapon, such as the depth, color, texture, size, and light effect information of the object, and output a detailed text description of the attack effect that will be generated after the interaction between the two, and a detailed text description of the state of the object after being attacked.

[0040] For example, the process of training the AIGC large language model (i.e. the second model in the embodiments of this application) may include: taking the results of the interaction mode already set in the game and the results of object interaction in similar style games, the generated special effects, screen style, etc. as input models as training set samples, then inputting the results of object interaction without setting interaction as a validation set for validation, and finally generating text descriptions of interaction with any object.

[0041] Step 203: Based on the target image, generate a second 3D model of the object before interaction.

[0042] In this embodiment of the application, a second three-dimensional model corresponding to the object before interaction can be generated based on the target image using an AI modeling model (i.e., the first model in this embodiment of the application), that is, a complete three-dimensional model of the object before interaction.

[0043] In some examples, if the user's interaction with the object is an attack on a tree by character 1 using item 1 in a game, the target image can be input into the AI ​​modeling model for processing, and a three-dimensional model of the tree before it is attacked can be generated (i.e., the second three-dimensional model in the embodiments of this application).

[0044] As an optional approach, such as Figure 6 As shown, the AI ​​modeling model can be called through the interface, and the image of the interactive subject (i.e. the object that interacts with the user in this embodiment) can be input. The spatial relationship between the interactive subject and the props used by the user can be analyzed, and matching 3D resource models that conform to the scene description can be retrieved from the material library. The size, position and other aspects of the model can be transformed according to the environment in which the interactive subject is located.

[0045] As an alternative approach, if no matching 3D model is found, an AIGC model can be used to generate a 3D resource model that conforms to the scene description. Specifically, the input image is first preprocessed, including resizing, cropping, and denoising, to improve the accuracy of subsequent processing. Then, a deep learning algorithm is applied to estimate the depth information of each pixel in the image, which may involve a trained network to predict the depth map. Based on the depth information and the 2D image, the geometry of the 3D model is reconstructed. The texture information of the original 2D image is applied to the 3D model to ensure that the 3D model is visually consistent with the original image. Finally, a 3D model (i.e., the second 3D model in this embodiment) is output. The generated 3D model is then optimized and adjusted in detail using descriptive prompts for the objects (i.e., interactive information in this embodiment) to improve the model's realism and accuracy. A unified style is achieved based on the surrounding environment information. If the 3D model needs to be viewed from different perspectives, multi-view images may need to be generated, and consistency between the viewpoints must be ensured. Post-processing steps such as smoothing, noise removal, and hole filling are performed on the 3D model (i.e., the second three-dimensional model in the embodiments of this application) to improve the model quality.

[0046] Step 204: Based on the target image, interaction information, and the second 3D model, generate the interaction effects between the user and the object, and the first 3D model of the object after the interaction.

[0047] Optionally, when performing the action of "generating user-object interaction effects and a first 3D model corresponding to the object after interaction based on the target image, interaction information, and the second 3D model", the following method can be used, but is not limited to: the method includes: using the first model to obtain a first 3D model based on the target image, interaction information, and the second 3D model; using the first model to generate state information corresponding to the object after interaction; and updating the second 3D model based on the state information to obtain the first 3D model; using the second model to generate interaction effect information based on the interaction information; and using the second model to generate effect information based on the interaction information to obtain the interaction effect information.

[0048] In this embodiment of the application, based on the target image, interaction information and the second three-dimensional model, an AI modeling model (i.e. the first model in this embodiment of the application) can be used to generate the first three-dimensional model corresponding to the object after interaction, that is, the three-dimensional model of the object after interaction.

[0049] In some examples, if the user's interaction with the object is an attack on a tree by character 1 using item 1 in a game, the target image, interaction information and the second three-dimensional model can be input into the AI ​​modeling model for processing, and a three-dimensional model of the tree after being attacked can be generated (i.e., the first three-dimensional model in the embodiments of this application).

[0050] In this embodiment, the AIGC large language model (i.e., the second model in this embodiment) can be used to generate textual description information of the interaction effects between the user and the object based on the interaction information, i.e., the interaction effect information in this embodiment.

[0051] In some examples, if the user's interaction with the object is an attack on a tree by character 1 using item 1 in a game, the interaction information can be input into the AIGC large language model (i.e., the second model in this application embodiment) for processing, and textual description information of the tree's attacked characteristics during the attack process (i.e., the interactive effect information in this application embodiment) can be generated.

[0052] Step 205: Based on the interactive special effects information and the first 3D model, perform synthesis processing to obtain the target 3D model.

[0053] The target 3D model is used to show users the results of their interaction with the object.

[0054] Optionally, when performing the "composite processing based on interactive special effects information and the first three-dimensional model to obtain the target three-dimensional model", the following method can be used, but is not limited to: generating an interactive special effects image based on the interactive special effects information; and compositing the interactive special effects image with the first three-dimensional model to obtain the target three-dimensional model.

[0055] For example, if the user's interaction with the object is an attack on a tree by character 1 using prop 1 in a game, a newly created model (i.e., the second three-dimensional model in this embodiment) can be used to replace the original model in the video stream transmitted by the game. When the player attacks the tree model, the model of the attacked subject (i.e., the second three-dimensional model in this embodiment) is highlighted to emphasize that the interaction is in progress.

[0056] As an alternative approach, an AI modeling model can be called via an interface, with an image of the object and a textual description of the object's state after being attacked as input. The model can then be modified based on the new model (i.e., the second three-dimensional model in this embodiment) to output a 3D model of the object's state after being attacked (i.e., the first three-dimensional model in this embodiment).

[0057] For example, such as Figure 7 and Figure 8 As shown, Figure 7 This is a diagram illustrating interactive effects. Figure 8The diagram shows the 3D model of the attacked object (i.e., the first three-dimensional model in this embodiment). It can also generate an image of the attack effect by calling an AI text-based image model, inputting the text description of the attack effect, and then compositing the attack effect and the 3D model of the attacked object (i.e., the first three-dimensional model in this embodiment). Finally, in the video stream transmitted by the game, the newly synthesized model with the effect (i.e., the target three-dimensional model in this embodiment) is used to replace the original model to display the interaction result.

[0058] Optionally, after performing the "synthesis processing based on interactive special effects information and the first 3D model to obtain the target 3D model", the following method can be used, but is not limited to: determining at least one interactive object in the target image other than the object; determining the target configuration information of the object based on the configuration information of the at least one interactive object; and sending the target configuration information and the target 3D model to the user through a data stream.

[0059] In some examples, after an object is attacked, other pre-defined interactive objects in the vicinity can be identified, and the post-interaction benefits of these pre-defined interactive objects (i.e., configuration information in the embodiments of this application) can be collected, such as the empirical value E. i , where i represents multiple surrounding objects (i.e., at least one interactive object in the embodiments of this application).

[0060] Optionally, when performing the "determine the target configuration information of an object based on the configuration information of at least one interactive object", the following method may be used, but is not limited thereto: the method includes: generating initial configuration information of the object using a second model based on the interaction information; and determining the target configuration information of the object based on the initial configuration information and the configuration information of at least one interactive object.

[0061] For example, based on the detailed textual description of the object (i.e., the interaction information in the embodiments of this application), the AIGC large language model (i.e., the second model in the embodiments of this application) is used to evaluate the possible experience value V of the object in the game (i.e., the initial configuration information in the embodiments of this application).

[0062] As an optional approach, the empirical value obtained from the final interaction (i.e., the target configuration information in this application embodiment) can be calculated using Formula 1, which is shown below: (Formula 1) In Formula 1, k1 represents the weight coefficient of the configuration information of at least one interactive object in the embodiment of this application, and k2 represents the weight coefficient of the initial configuration information in the embodiment of this application.

[0063] For example, in this application embodiment, interactive objects, prop information used by the user, AI-generated models (i.e., the first three-dimensional model and the second three-dimensional model in this application embodiment), special effects, and experience values ​​corresponding to the models (i.e., configuration information in this application embodiment) can all be saved to the database. When the object is interacted with a second time in the game, the above content can be directly used to replace and bring in.

[0064] In existing technical solutions, the success and outcome of player-object interactions are pre-set by the game program. When a player approaches an object, an interaction pop-up appears near the object's model in the game interface. After the player interacts with the pop-up, the interaction event bound to the object model is triggered, and a pre-made interaction animation is played.

[0065] In existing technologies, whether a player can interact with objects in a game scene depends on whether the game program has bound trigger events to the object models. If no interaction events are pre-set, the player cannot interact with some objects, affecting the player's gaming experience. If all objects in the game have pre-set interaction logic code, the workload would be enormous, it would lack flexibility, and the game package size would also be affected.

[0066] It should be noted that this embodiment uses AIGC technology to simulate the interaction result when the player interacts with objects that do not have pre-defined interaction events in the game, displays the result to the player, and distributes potential interaction benefits. This embodiment reduces the amount of game code, while increasing game flexibility and playability, thus improving the player's gaming experience. This embodiment can continuously monitor the player's game progress and the surrounding environment. When the player interacts with objects in the game scene, the system detects the interacting object, models the object using AIGC technology, generates a result model of the interaction, determines whether the interaction generates benefits based on the surrounding environment, and distributes potential benefits to the player. This solution will give almost all objects in the game interactive attributes, improving game playability.

[0067] Compared with existing technologies, this embodiment obtains a target image of the user-object interaction in response to the user's interaction with the object; determines the interaction information between the user and the object based on the target image; generates interaction effect information between the user and the object and a first 3D model corresponding to the object after the interaction based on the target image and the interaction information; and performs synthesis processing based on the interaction effect information and the first 3D model to obtain a target 3D model, which is used to display the interaction result with the object to the user. This embodiment enables the user to interact with an object without setting an interaction event through the synthesized target 3D model, realizing the interaction between the user and the object without relying on the interaction event code set by the object. This reduces the amount of code for setting interaction events for objects in the application, while improving the flexibility of the interaction between the object and the user in the application and enhancing the user experience.

[0068] Furthermore, as Figure 1 and Figure 2 To provide a specific implementation of the method shown, this embodiment offers an interactive device, such as... Figure 9 As shown, the device includes: an acquisition module 31, a determination module 32, a generation module 33, and a synthesis module 34.

[0069] The acquisition module 31 is configured to acquire the target image of the interaction between the user and the object in response to the user's interaction with the object; The determining module 32 is configured to determine the interaction information between the user and the object based on the target image; The generation module 33 is configured to generate interaction effect information between the user and the object and a first three-dimensional model of the object after the interaction, based on the target image and the interaction information. The compositing module 34 is configured to perform compositing processing based on the interactive effect information and the first three-dimensional model to obtain a target three-dimensional model, which is used to display the interaction results with the object to the user.

[0070] In some examples of this embodiment, the generation module 33 is further configured to generate a second three-dimensional model of the object before interaction based on the target image; correspondingly, the generation module 33 is specifically configured to generate the interaction effects between the user and the object and a first three-dimensional model of the object after interaction based on the target image, the interaction information and the second three-dimensional model.

[0071] In some examples of this embodiment, the generation module 33 is further configured to use a first model to obtain a first three-dimensional model based on the target image, interaction information, and a second three-dimensional model. The first model is used to generate state information corresponding to the object after interaction, and the second three-dimensional model is updated based on the state information to obtain the first three-dimensional model. Based on the interaction information, the second model is used to generate interaction effect information, and the second model is used to generate effect information based on the interaction information to obtain the interaction effect information.

[0072] In some examples of this embodiment, the synthesis module 34 is further configured to identify at least one interactive object in the target image; determine target configuration information of the object based on the configuration information of the at least one interactive object; and send the target configuration information and the target 3D model to the user via a data stream.

[0073] In some examples of this embodiment, the synthesis module 34 is further configured to generate initial configuration information of an object using a second model based on interactive information; and to determine the target configuration information of the object based on the initial configuration information and the configuration information of at least one interactive object.

[0074] In some examples of this embodiment, the compositing module 34 is further configured to generate an interactive effect image based on the interactive effect information; and to compose the interactive effect image with the first three-dimensional model to obtain the target three-dimensional model.

[0075] It should be noted that other corresponding descriptions of the functional units involved in the interactive device provided in this embodiment can be found in [reference needed]. Figure 1 and Figure 2 The corresponding descriptions in [the document] will not be repeated here.

[0076] Based on the above, Figure 1 and Figure 2 Accordingly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figure 1 and Figure 2 The method shown.

[0077] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0078] like Figure 10 The diagram shown is a hardware structure schematic of an electronic device according to the present invention, comprising: At least one processor 401; and, A memory 402 is communicatively connected to at least one of the processors 401; wherein, The memory 402 stores instructions that can be executed by at least one of the processors to enable the at least one of the processors to perform the interaction method as described above.

[0079] Figure 10 Take a processor 401 as an example.

[0080] The electronic device may also include an input device 403 and an output device 404.

[0081] The processor 401, memory 402, input device 403, and output device 404 can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0082] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the interaction method in the embodiments of this application, for example, Figure 1 and Figure 2 The method flow is shown. The processor 401 executes various functional applications and data processing by running non-volatile software programs, instructions, and modules stored in the memory 402, thereby implementing the interactive method in the above embodiments.

[0083] Memory 402 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the interactive method, etc. Furthermore, memory 402 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 402 may optionally include memory remotely located relative to processor 401, and these remote memories may be connected to the apparatus performing the interactive method via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0084] Input device 403 can receive user clicks and generate signal inputs related to user settings and function control of the interaction method. Output device 404 may include display devices such as a display screen.

[0085] When one or more modules are stored in the memory 402, and are run by one or more processors 401, the interaction methods in any of the above method embodiments are executed.

[0086] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0087] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0088] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms, or it can be implemented by hardware. By applying the solution of this embodiment, compared with the prior art, this embodiment obtains the target image of the user interacting with the object in response to the user's interaction with the object; determines the interaction information between the user and the object based on the target image; generates the interaction effect information between the user and the object and the first three-dimensional model corresponding to the object after the interaction based on the target image and the interaction information; performs synthesis processing based on the interaction effect information and the first three-dimensional model to obtain the target three-dimensional model, which is used to display the interaction result with the object to the user; this embodiment enables the user to interact with the object without setting an interaction event through the synthesized target three-dimensional model, realizing the interaction between the user and the object without relying on the interaction event code set by the object, reducing the amount of code for setting interaction events for the object in the application, and improving the flexibility of the interaction between the object and the user in the application, thereby improving the user experience.

[0090] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0091] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An interaction method, characterized in that, include: In response to the user's interaction with the object, acquire the target image of the interaction between the user and the object; The interaction information between the user and the object is determined based on the target image; Based on the target image and the interaction information, generate interaction effect information between the user and the object and a first three-dimensional model of the object after the interaction; Based on the interactive effects information and the first 3D model, a target 3D model is obtained through synthesis processing. The target 3D model is used to display the interaction results with the object to the user.

2. The method according to claim 1, characterized in that, Before generating the interaction effect information between the user and the object and the first 3D model of the object after the interaction based on the target image and the interaction information, the method further includes: Based on the target image, a second three-dimensional model corresponding to the object before interaction is generated; The step of generating interaction effect information between the user and the object and a first 3D model of the object after interaction, based on the target image and the interaction information, includes: Based on the target image, the interaction information, and the second three-dimensional model, the interaction effects between the user and the object and the first three-dimensional model of the object after the interaction are generated.

3. The method according to claim 2, characterized in that, The step of generating the interaction effects between the user and the object and the first three-dimensional model corresponding to the object after the interaction, based on the target image, the interaction information, and the second three-dimensional model, includes: Based on the target image, the interaction information, and the second three-dimensional model, the first three-dimensional model is obtained using the first model. The first model is used to generate state information corresponding to the object after the interaction, and the second three-dimensional model is updated based on the state information to obtain the first three-dimensional model. The interactive effect information is generated using a second model based on the interactive information. The second model is used to generate effect information based on the interactive information to obtain the interactive effect information.

4. The method according to claim 3, characterized in that, After the target 3D model is obtained by compositing based on the interactive effects information and the first 3D model, the method further includes: Identify at least one interactive object in the target image; The target configuration information of the object is determined based on the configuration information of the at least one interactive object; The target configuration information and the target 3D model are sent to the user via a data stream.

5. The method according to claim 4, characterized in that, Determining the target configuration information of the object based on the configuration information of the at least one interactive object includes: Based on the interaction information, the second model is used to generate the initial configuration information of the object; Based on the initial configuration information and the configuration information of the at least one interactive object, the target configuration information of the object is determined.

6. The method according to any one of claims 1 to 5, characterized in that, The process of synthesizing the target 3D model based on the interactive special effects information and the first 3D model includes: Generate interactive effects images based on interactive effects information; The interactive special effects image is combined with the first 3D model to obtain the target 3D model.

7. An interactive device, characterized in that, include: The acquisition module is configured to acquire the target image of the interaction between the user and the object in response to the user's interaction with the object; The determination module is configured to determine the interaction information between the user and the object based on the target image; The generation module is configured to generate interaction effect information between the user and the object and a first three-dimensional model of the object after the interaction, based on the target image and the interaction information. The compositing module is configured to perform compositing processing based on the interactive effect information and the first 3D model to obtain a target 3D model, which is used to display the interaction results with the object to the user.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

9. An electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

10. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images