AR-based card interaction method and related device

Abstract

The invention relates to the technical field of the Internet, in particular to an AR-based card interaction method and a related device, which are used for improving the interactivity and playability of an entity card in an online chat scene. The method comprises the following steps: presenting a chat interface containing an interactive control; the interaction control is used for arousing an AR card interaction function; in response to an interaction operation triggered by the initiator object for the interaction control, presenting a card interaction area in the chat interface; when a first card input by the initiator object is recognized in the card interaction area, presenting an initiator AR picture including a first virtual character corresponding to the first card, and sharing the initiator AR picture to at least one receiver object of the initiator object; the first virtual character is a virtual content generated in combination with an AR technology after entity identification is performed on the first card. In the online chat scene, the card is combined with the AR technology to generate the corresponding virtual character, so that the interactivity and playability of the entity card in the online chat scene are improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to an AR-based card interaction method and related devices. Background Art

[0002] In recent years, trading cards have become a new collectible that is popular in the market. Players can collect cards through exchanging, purchasing, drawing cards, etc., and flexibly use different cards to form decks that comply with the rules according to different strategies to play games. The entire card market is becoming increasingly popular.

[0003] Taking card drawing and collecting as an example, whether in live broadcast rooms or various game applications, the existing card drawing gameplay is only aimed at drawing cards. After the subject knows the result of the card drawing, he or she will stop paying attention to the cards. This will cause the subject to no longer explore other aspects of cards or card-related games.

[0004] On the other hand, when subjects talk about cards during online chats, they often just take photos of physical cards and send them to the chat interface. This will reduce the subjects' willingness to further discuss cards in the online chat scene.

[0005] It can be seen that under the relevant technology, the communication between the subjects about physical cards in the online chat scene is very limited, which to a certain extent reduces the discussion topic of cards in the online chat scene, is not conducive to the subjects' further exploration of cards or card-related games, thereby reducing the subjects' attention to the game and the subjects' activity in the game.

[0006] In summary, how to improve the interactivity and playability of physical cards in online chat scenarios is an urgent issue to be solved. Summary of the Invention

[0007] The embodiments of the present application provide an AR-based card interaction method and related devices to improve the interactivity and playability of physical cards in online chat scenarios.

[0008] An embodiment of the present application provides an AR-based card interaction method, including:

[0009] Presenting a chat interface including interactive controls; the interactive controls are used to invoke AR card interactive functions;

[0010] In response to an interactive operation triggered by an initiator object on the interactive control, presenting a card interactive area in the chat interface;

[0011] When the first card input by the initiator object is recognized in the card interaction area, an initiator AR screen including a first virtual character corresponding to the first card is presented, and the initiator AR screen is shared with at least one recipient object associated with the initiator object; wherein, the first virtual character is virtual content generated by combining AR technology after entity recognition of the first card.

[0012] It should be emphasized that in the specific implementation of this application, the relevant data involved in the subject's AR-based card interaction, such as the first card input by the initiating subject, the first virtual character, the data generated by the other party triggering the interaction operation, the initiating AR screen, etc. When the above embodiments of this application are applied to specific products or technologies, the subject's permission or consent must be obtained, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0013] Another AR-based card interaction method provided in an embodiment of the present application includes:

[0014] Presenting the chat interface;

[0015] In response to an acceptance operation of the invitation request triggered by the initiator object by the recipient object, a card interaction area is presented in the chat interface, wherein the card interaction area includes an initiator AR screen including a first virtual character obtained after entity recognition of a first card input by the initiator object;

[0016] Among them, the first virtual character is a virtual content generated by combining AR technology after entity recognition of the first card; the invitation request is sent by the initiator object based on the triggering of the interactive control in the chat interface.

[0017] Another AR-based card interaction method provided in an embodiment of the present application includes:

[0018] After receiving a recognition request for a first card input by an initiator object, determining first model information corresponding to an entity in the first card by performing entity recognition on the first card;

[0019] Feeding back the first model information to an initiator client corresponding to the initiator object, so that the initiator client presents, in a card interaction area in a chat interface: an initiator AR screen including a first virtual character corresponding to the first card; the card interaction area is presented based on the initiator object and triggers an interactive operation in response to an interactive control in the chat interface; the first virtual character is virtual content generated based on the first model information in combination with AR technology; and

[0020] After determining that the target recipient object accepts the invitation request, the first model information is fed back to the recipient client corresponding to the target recipient object, so that the recipient client shares the initiator's AR screen to the recipient object through the card interaction area; the invitation request is sent based on the sending object triggering an interactive operation on the interactive control in the chat interface.

[0021] An embodiment of the present application provides an AR-based card interaction device, comprising:

[0022] A first presentation unit is configured to present a chat interface including interactive controls for invoking AR card interaction functions;

[0023] A second presentation unit, configured to present a card interaction area in the chat interface in response to an interaction operation triggered by an initiator object on the interaction control;

[0024] The second presentation unit is further configured to present an initiator AR screen including a first virtual character corresponding to the first card when the first card input by the initiator object is recognized in the card interaction area, and share the initiator AR screen with at least one recipient object associated with the initiator object; wherein the first virtual character is virtual content generated by combining AR technology after entity recognition of the first card.

[0025] Optionally, the second presentation unit is specifically configured to:

[0026] In response to an interactive operation triggered by the interactive control, invoking a camera function;

[0027] Presenting a card interaction area in the chat interface, wherein the card interaction area includes the initiator's environmental information collected by the camera function;

[0028] By using the camera function, when the first card input by the initiator object is recognized in the initiator environment information, an initiator AR screen including the first virtual character is presented.

[0029] Optionally, the initiator environment information is located in a camera image layer in the initiator AR image, the first virtual character is located in a model layer in the initiator AR image, and the model layer is located above the camera image layer.

[0030] Optionally, the second presentation unit is further configured to:

[0031] Sending an invitation request triggered by the interactive control to at least one recipient object, and presenting corresponding invitation information in the chat area of ​​the chat interface;

[0032] The second presentation unit is specifically configured to:

[0033] After determining that a target recipient object among the at least one recipient object accepts the invitation, the initiator AR screen is shared with the target recipient object.

[0034] Optionally, the second presentation unit is further configured to:

[0035] Determine that the target recipient object joins this card interaction, present the initiator's AR image in the initiator window in the card interaction area, and present the recipient's AR image corresponding to the target recipient object in the recipient window;

[0036] The receiving AR screen includes: a second virtual character corresponding to the second card input by the target receiving object; the second virtual character is virtual content generated by combining AR technology after entity recognition of the second card.

[0037] Optionally, the device further includes:

[0038] The interactive unit is used to present an interactive animation corresponding to the behavioral instruction in the card interactive area in response to the behavioral instruction triggered for the first virtual character, and the interactive animation represents: at least controlling the first virtual character to perform the behavior corresponding to the behavioral instruction.

[0039] Optionally, the behavior instruction is a behavior instruction for a single character, and the interactive animation represents: controlling the first virtual character to perform a behavior corresponding to the behavior instruction;

[0040] A second virtual character is also presented in the card interaction area, and the behavior instructions are interactive behavior instructions for multiple characters. The interactive animation representation: controlling the first virtual character and the second virtual character to perform the interactive behavior corresponding to the behavior instructions; the second virtual character is a virtual content generated by combining AR technology after entity recognition of the second card input by the target recipient object.

[0041] Optionally, a second virtual character is further presented in the card interaction area, and the behavior instructions are interaction behavior instructions for multiple characters. The interaction unit is specifically configured to:

[0042] In the card interaction area, the initiator's AR screen is presented through the initiator's window, and the receiver's AR screen corresponding to the target receiver object is presented through the receiver's window;

[0043] Presenting an interactive animation corresponding to the action instruction in the model layers of the initiator's AR screen and the recipient's AR screen;

[0044] There is no window boundary restriction on the model layer between each window.

[0045] Optionally, there are multiple first virtual characters, and the interaction unit is specifically configured to:

[0046] In response to a behavioral instruction input for the first card, a target first virtual character described by the behavioral instruction is determined from a plurality of first virtual characters, and an interactive animation corresponding to the behavioral instruction is presented in the card interaction area; wherein the interactive animation represents: at least controlling the target first virtual character to perform the behavior corresponding to the behavioral instruction; or

[0047] After a target first virtual character is selected from a plurality of first virtual characters, in response to a behavior instruction triggered for the target first virtual character, an interactive animation corresponding to the behavior instruction is presented in the card interaction area; wherein, the interactive animation represents: at least controlling the target first virtual character to perform the behavior corresponding to the behavior instruction.

[0048] Optionally, the behavioral instruction in the interactive unit is triggered by at least one of the following methods:

[0049] The behavior instruction is triggered by inputting a behavior keyword; wherein the behavior keyword is used to describe: the behavior that the first virtual character needs to perform;

[0050] The behavior instruction is triggered by inputting a voice; the voice is used to describe: the behavior that the first virtual character needs to perform;

[0051] Recognizing an input action through a camera function, and triggering the behavior instruction based on the action; the action is used to describe: the behavior that the first virtual character needs to perform;

[0052] At least one behavior control is presented in response to a behavior instruction triggered by selecting a target behavior control from the at least one behavior control, wherein the behavior control is used to describe a behavior that the first virtual character needs to perform.

[0053] Optionally, the device further includes:

[0054] An adjustment unit is configured to, in response to an adjustment gesture triggered for the first virtual character, perform a screen zooming operation corresponding to the adjustment gesture on the first virtual character, and present the first virtual character after performing the screen zooming operation in the card interaction area.

[0055] Another AR-based card interaction device provided in an embodiment of the present application includes:

[0056] A third presentation unit, configured to present a chat interface;

[0057] an invitation unit, configured to, in response to an acceptance operation of an invitation request triggered by a recipient object to an initiator object, present a card interaction area in the chat interface, wherein the card interaction area includes an initiator AR screen including a first virtual character obtained after entity recognition of a first card input by the initiator object;

[0058] Among them, the first virtual character is a virtual content generated by combining AR technology after entity recognition of the first card; the invitation request is sent by the initiator object based on the triggering of the interactive control in the chat interface.

[0059] Optionally, the invitation unit is specifically configured to:

[0060] Receiving the invitation request triggered by the initiator object, presenting the corresponding invitation information sent by the initiator object and an acceptance control in the chat area of ​​the chat interface;

[0061] In response to the acceptance operation of the invitation request triggered by the recipient object based on the acceptance control, a card interaction area is presented in the chat interface.

[0062] Optionally, the invitation unit is further configured to:

[0063] In response to a join operation triggered by the join control, joining the current card interaction and invoking a camera function;

[0064] When the second card input by the receiving object is recognized through the camera function, the initiator's AR screen is presented through the initiator's window in the card interaction area, and the receiving party's AR screen including the second virtual character corresponding to the second card is presented through the receiving party's window; wherein, the second virtual character is virtual content generated after the second card is physically recognized in combination with AR technology.

[0065] Another AR-based card interaction device provided in an embodiment of the present application includes:

[0066] a receiving unit configured to, upon receiving an identification request for a first card input by an initiator object, determine first model information corresponding to an entity in the first card by performing entity identification on the first card;

[0067] a first feedback unit, configured to feed back the first model information to an initiator client corresponding to the initiator object, so that the initiator client presents, in a card interaction area in a chat interface: an initiator AR screen including a first virtual character corresponding to the first card; the card interaction area is presented based on the initiator object and triggers an interactive operation in response to an interactive control in the chat interface; the first virtual character is virtual content generated based on the first model information in combination with AR technology; and

[0068] After determining that the target recipient object accepts the invitation request, the first model information is fed back to the recipient client corresponding to the target recipient object, so that the recipient client shares the initiator's AR screen to the recipient object through the card interaction area; the invitation request is sent based on the sending object triggering an interactive operation on the interactive control in the chat interface.

[0069] Optionally, the device further includes:

[0070] a second feedback unit configured to, upon receiving a recognition request for a second card inputted by the target recipient object, determine second model information corresponding to an entity in the second card by performing entity recognition on the second card;

[0071] The second model information is fed back to the receiving client so that the receiving client presents the initiator's AR screen through the initiator's window in the card interaction area, and presents the receiving AR screen of the second virtual character corresponding to the second card through the receiving window; the second virtual character is virtual content generated based on the second model information in combination with AR technology.

[0072] Optionally, the second card is input by the target recipient object after joining this card interaction, and the second feedback unit is further configured to:

[0073] The second model information is fed back to the initiator client, so that the initiator client presents the initiator AR screen through the initiator window in the card interaction area, and presents the recipient AR screen through the recipient window.

[0074] Optionally, the device further includes:

[0075] a third feedback unit configured to receive a behavior instruction triggered by an initiator object for the first virtual character after the first feedback unit feeds back the first model information to the receiving client, and determine interactive action information corresponding to the behavior instruction;

[0076] Feedback the interactive action information to the initiator client and the recipient client, so that both the initiator client and the recipient client are in the corresponding card interaction area and present the interactive animation corresponding to the action instruction based on the interactive information;

[0077] The interactive animation indicates at least controlling the first virtual character to execute the behavior corresponding to the behavior instruction.

[0078] An electronic device provided in an embodiment of the present application includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of any one of the above-mentioned AR-based card interaction methods.

[0079] An embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to perform the steps of any one of the above-mentioned AR-based card interaction methods.

[0080] An embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform any one of the steps of the above-mentioned AR-based card interaction method.

[0081] The beneficial effects of this application are as follows:

[0082] The embodiment of the present application provides an AR-based card interaction method and related devices. The embodiment of the present application adds an AR card interaction function, which supports chat objects to share and interact with virtual characters generated based on physical cards in the chat scene. By presenting the first virtual character of the first card input by the corresponding initiator object generated in combination with AR technology in the card interaction area in the chat interface, the expression form and display effect of the physical card in the online chat scene are enriched. Furthermore, the initiator object can share the initiator AR screen containing the first virtual character to the recipient object, so that the recipient object can also view the first virtual character in the card interaction area in the recipient client, so that AR technology can be applied in multi-person online chat scenarios, expanding the use methods and usage scenarios of AR technology.

[0083] In summary, this application uses AR technology to realize card interaction and other functions in online chat scenarios, thereby improving the interactivity and playability of physical cards in online chat scenarios, which in turn facilitates the initiator object and the recipient object to further discuss cards in the online chat scenario, and helps to increase the initiator object and the recipient object's attention to card-related games, as well as the initiator object and the recipient object's activity in the game.

[0084] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0086] Figure 1 This is a schematic diagram of an application scenario in an embodiment of the present application;

[0087] Figure 2 This is a flowchart of an implementation of an AR-based card interaction method provided in an embodiment of the present application;

[0088] Figure 3A A schematic diagram of an interactive control provided in an embodiment of the present application;

[0089] Figure 3B A schematic diagram of another interactive control provided in an embodiment of the present application;

[0090] Figure 4A A schematic diagram of a card interaction area provided in an embodiment of the present application;

[0091] Figure 4B A schematic diagram of another card interaction area provided in an embodiment of the present application;

[0092] Figure 5 A schematic diagram of invoking a camera function provided in an embodiment of the present application;

[0093] Figure 6 A schematic diagram of a first virtual character provided in an embodiment of the present application;

[0094] Figure 7 A schematic diagram of an invitation message provided in an embodiment of the present application;

[0095] Figure 8A schematic diagram of a target recipient object joining an interaction provided in an embodiment of the present application;

[0096] Figure 9A A schematic diagram of controlling a virtual character provided in an embodiment of the present application;

[0097] Figure 9B A schematic diagram of another method of controlling a virtual character provided in an embodiment of the present application;

[0098] Figure 9C A schematic diagram of another method of controlling a virtual character provided in an embodiment of the present application;

[0099] Figure 10A A schematic diagram of a first virtual character of a control target provided in an embodiment of the present application;

[0100] Figure 10B A schematic diagram of another control target first virtual character provided in an embodiment of the present application;

[0101] Figure 11A A schematic diagram of a triggering behavior instruction provided in an embodiment of the present application;

[0102] Figure 11B A schematic diagram of another triggering behavior instruction provided in an embodiment of the present application;

[0103] Figure 11C A schematic diagram of another triggering behavior instruction provided in an embodiment of the present application;

[0104] Figure 11D A schematic diagram of a behavior control provided in an embodiment of the present application;

[0105] Figure 12A A schematic diagram of enlarging a first virtual character provided in an embodiment of the present application;

[0106] Figure 12B A schematic diagram of reducing the size of a first virtual character provided in an embodiment of the present application;

[0107] Figure 13 This is a flowchart of another AR-based card interaction method in an embodiment of the present application;

[0108] Figure 14 A schematic diagram of receiving an invitation request provided in an embodiment of the present application;

[0109] Figure 15 A schematic diagram of a sharing initiator's AR screen provided in an embodiment of the present application;

[0110] Figure 16A A schematic diagram of a receiving object joining an interaction provided in an embodiment of the present application;

[0111] Figure 16B A schematic diagram of another receiving object joining the interaction provided in an embodiment of the present application;

[0112] Figure 17 This is a flowchart of another AR-based card interaction method in an embodiment of the present application;

[0113] Figure 18 A schematic diagram of a layered presentation provided in an embodiment of the present application;

[0114] Figure 19 A flowchart of sharing a first card provided in an embodiment of the present application;

[0115] Figure 20 A flowchart of sharing a second card provided in an embodiment of the present application;

[0116] Figure 21 Schematic diagram of the composition structure of an AR-based card device in an embodiment of the present application;

[0117] Figure 22 Schematic diagram of the structure of another AR-based card device in an embodiment of the present application;

[0118] Figure 23 This is a schematic diagram of the structure of another AR-based card device in an embodiment of the present application;

[0119] Figure 24 A schematic diagram of the hardware structure of an electronic device to which an embodiment of the present application is applied;

[0120] Figure 25 A schematic diagram of the hardware structure of another electronic device to which an embodiment of the present application is applied. DETAILED DESCRIPTION

[0121] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of the technical solutions of this application, but not all of them. Based on the embodiments described in this application document, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the technical solutions of this application.

[0122] The following is an introduction to some concepts involved in the embodiments of this application.

[0123] Augmented Reality (AR) is a technology that combines virtual information with the real world. By calculating the position and angle of a camera image in real time and adding corresponding virtual content to real-world content such as images, videos, and three-dimensional (3D) models, the virtual image and the real scene are combined to create an augmented reality experience.

[0124] Virtual characters: refers to characters presented in the real world through AR technology. Virtual characters can be superimposed on real scenes, and objects can view, interact with and control these virtual characters through devices such as mobile phones and tablets.

[0125] Model information: refers to the information used to generate the corresponding virtual character. The model information includes but is not limited to the geometric structure, texture mapping, skeletal animation and other information of the corresponding virtual character. If the corresponding virtual character needs to have action and animation effects, the model information may also include key frames, skeletal hierarchy and animation sequence data. This data can be used to drive the action of the virtual character, enabling it to interact in real time.

[0126] Card interaction area: This can be an area within the chat interface, or a sub-interface such as a floating layer or pop-up window

[0127] Model layer: refers to the layer used to render each model, which can be simply referred to as the model layer. In this application, all models, including but not limited to the first virtual character and the second virtual character, can be displayed within the same model layer. Therefore, when multiple virtual characters interact, they all interact within the model layer without window boundary restrictions, thereby achieving visual cross-window interaction.

[0128] Camera screen layer: refers to the layer under the model layer, which is used to render the real-time environmental information collected by the object through the camera function. It can also be called the video stream screen layer.

[0129] The embodiments of the present application relate to artificial intelligence (AI) and are designed based on computer vision technology (CV), speech technology (Speech Technology), and natural language processing (NLP) in artificial intelligence.

[0130] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0131] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0132] Key technologies in speech technology include automatic speech recognition (ASR), text-to-speech (TTS), and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the future direction of human-computer interaction, with speech becoming one of the most promising methods of human-computer interaction. Large model technology is revolutionizing the development of speech technology. Pre-trained models based on the Transformer architecture, such as the language pre-trained model (WavLM) and the dedicated speech signal processing chip (UniSpeech), offer strong generalization and versatility, enabling them to effectively handle a wide range of speech processing tasks.

[0133] In an embodiment of the present application, automatic speech recognition technology can be used to perform speech recognition on the speech input by the initiator chat object, the recipient chat object, etc., thereby triggering behavioral instructions for the first virtual character, etc.

[0134] Natural language processing (NLP) is a key area of ​​research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0135] In an embodiment of the present application, behavior keywords can be analyzed based on natural language processing technology, so as to trigger behavior instructions according to the behavior keywords, so that the first virtual character and the second virtual character perform corresponding behaviors.

[0136] Computer vision (CV) is the science of making machines "see." Specifically, it refers to using cameras and computers to replace the human eye in identifying and measuring objects, and then further processing them to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems that can extract information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / action recognition, three-dimensional object reconstruction, three-dimensional (3D) technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0137] In an embodiment of the present application, the identification information in the first card and the second card can be identified based on computer vision technology, so as to determine the first virtual character corresponding to the first card and the second virtual character corresponding to the second card.

[0138] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones, digital twins, virtual humans, robots, AIGC, conversational interaction, smart medical care, smart customer service, game AI, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0139] In addition, the content screening method in the embodiment of the present application also involves database technology.

[0140] A database, in short, can be thought of as a digital filing cabinet—a place where electronic files are stored, where objects can add, query, update, and delete data within them. A "database" is a collection of data stored in a specific way, shared by multiple objects, with minimal redundancy, and independent of applications.

[0141] For example, various types of information in the embodiments of the present application (such as first model information, second model information, interaction action information, etc.) can be stored in a database for subsequent use, and so on.

[0142] The following is a brief introduction to the design concept of the embodiment of this application:

[0143] In recent years, trading cards have become a new collectible that is popular in the market. Players can collect cards through exchanging, purchasing, drawing cards, etc., and flexibly use different cards to form decks that comply with the rules according to different strategies to play games. The entire card market is becoming increasingly popular.

[0144] Take card collection as an example. Whether in livestreams or various gaming apps, existing card collection methods are solely focused on obtaining a card. Once the player learns the result of the draw, their interest in the card ends. This leads them to stop exploring other aspects of the card game or card-related games. Furthermore, when discussing cards in online chats, participants often simply take photos of the physical cards and post them in the chat interface. This reduces their willingness to further discuss the cards in the online chat setting.

[0145] It can be seen that under the relevant technology, the communication between the subjects about physical cards in the online chat scene is very limited, which to a certain extent reduces the discussion topic of cards in the online chat scene, is not conducive to the subjects' further exploration of cards or card-related games, thereby reducing the subjects' attention to the game and the subjects' activity in the game.

[0146] In addition, the AR technology used in related technologies is often only for single-person use and has relatively limited usage scenarios. Therefore, in online chat scenarios, AR technology cannot provide good interactive effects.

[0147] In view of this, an embodiment of the present application provides an AR-based card interaction method and related devices. The embodiment of the present application adds an AR card interaction function, which supports chat objects to share and interact with virtual characters generated based on physical cards in a chat scene. By presenting the first virtual character of the first card input by the corresponding initiator object generated in combination with AR technology in the card interaction area in the chat interface, the expression form and display effect of the physical card in the online chat scene are enriched. Furthermore, the initiator object can share the initiator AR screen containing the first virtual character to the recipient object, so that the recipient object can also view the first virtual character in the card interaction area in the recipient client, so that AR technology can be applied in multi-person online chat scenarios, expanding the usage and application scope of AR technology.

[0148] In summary, this application uses AR technology to realize card interaction and other functions in online chat scenarios, thereby improving the interactivity and playability of physical cards in online chat scenarios, which in turn facilitates the initiator object and the recipient object to further discuss cards in the online chat scenario, and helps to increase the initiator object and the recipient object's attention to card-related games, as well as the initiator object and the recipient object's activity in the game.

[0149] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application can be combined with each other if there is no conflict.

[0150] like Figure 1 , which is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario diagram includes two terminal devices 110 and a server 120.

[0151] In the embodiment of the present application, the terminal device 110 includes but is not limited to mobile phones, tablet computers, laptop computers, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, car terminals and other devices; a client related to card interaction can be installed on the terminal device, which can be software (such as a browser, social software, etc.), or a web page, mini-program, etc. The server 120 is a background server corresponding to the software or web page, mini-program, etc., or a server specifically used for card interaction, which is not specifically limited in this application. The server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.

[0152] It should be noted that the AR-based card interaction method in each embodiment of the present application can be executed by an electronic device, which can be a terminal device 110 or a server 120, that is, the method can be executed by the terminal device 110 or the server 120 alone, or can be executed jointly by the terminal device 110 and the server 120.

[0153] For example, when the terminal device 110 and the server 120 jointly execute, a client related to the AR card interaction function (such as a browser, social software, instant messaging software, etc.) is deployed on the terminal device 110. Assuming that in a private chat scenario between object A and object B, object A is the initiator object and object B is the target recipient object, in this case, the terminal device 110 logged in by object A can be recorded as the initiator terminal device 110, and the client related to the AR card interaction function on this terminal device is the initiator client; the terminal device 110 logged in by object B can be recorded as the recipient terminal device 110, and the client related to the AR card interaction function on this terminal device is the recipient client. When the initiator client responds to the interactive operation triggered by object A for the interactive control, the card interaction area is presented in the chat interface displayed by the initiator client through the initiator terminal device 110, and an invitation request for AR card interaction is sent to object B through the server 120 by default, and the invitation information is presented on both the initiator client and the recipient client.

[0154] Specifically, when the initiator client responds to the interactive operation triggered by object A on the interactive control, it sends an invitation request for object B to the server 120 through the initiator terminal device 110. After receiving the invitation request sent by the initiator terminal device 110, the server 120 forwards the invitation request to the recipient terminal device 110, and the recipient terminal device 110 presents the corresponding invitation information to object B through the recipient client.

[0155] When object A recognizes the first card input by it on the initiator client, the initiator terminal device 110 sends a corresponding recognition request to the server 120. Then, the server 120 performs entity recognition on the first card in response to the recognition request of the first card, determines the first model information corresponding to the entity in the first card, and then feeds the first model information back to the initiator terminal device 110, so that the initiator terminal device 110, through the initiator client, combines the received first model information to render in the card interaction area, and presents the initiator AR screen containing the first virtual character corresponding to the first card.

[0156] In addition, when object B accepts the invitation based on the invitation information presented by the receiving client, it notifies the server 120 through the receiving terminal device 110. After the server 120 confirms that the invitation is accepted, the first model information, initiator environment information, etc. related to object A are sent to the receiving client through the receiving terminal device 110. The receiving client combines the first model information and the initiator environment information to render in the card interaction area, presenting the initiator's AR screen containing the first virtual character corresponding to the first card, thereby achieving the effect of AR screen sharing.

[0157] It should be noted that the above example takes object A as the initiator object and object B as the receiver object. In addition, object A can also be the target receiver object, and object B can also be the initiator object. In this case, the terminal device logged in by object A is the receiver terminal device, and the terminal device logged in by object B is the initiator terminal device.

[0158] In an optional implementation, the terminal device 110 and the server 120 may communicate via a communication network.

[0159] In an optional implementation, the communication network is a wired network or a wireless network.

[0160] It should be noted that Figure 1 The examples shown are just for illustration. In fact, the number of terminal devices and servers is not limited and is not specifically limited in the embodiments of this application.

[0161] In an embodiment of the present application, when there are multiple servers, the multiple servers can be combined into a blockchain, and the servers are nodes on the blockchain; as disclosed in the embodiment of the present application, the AR-based card interaction method, in which the model-related data involved can be saved on the blockchain, for example, first model information, second model information, interactive action information, etc.

[0162] In addition, the embodiments of the present application can be applied to various scenarios, including not only online chat scenarios, but also including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving and other scenarios.

[0163] It should be emphasized that in the specific implementation of this application, the relevant data involved in the subject's AR-based card interaction, such as the first card input by the initiating subject, the first virtual character, the data generated by the other party triggering the interaction operation, the initiating AR screen, etc. When the above embodiments of this application are applied to specific products or technologies, the subject's permission or consent must be obtained, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0164] The following describes the AR-based card interaction method provided by the exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0165] See Figure 2 As shown in FIG, a flowchart of an implementation method of an AR-based card interaction method provided in an embodiment of the present application is shown. Taking the initiator client (such as instant messaging software) as the execution subject as an example, the specific implementation process of the method is as follows:

[0166] S21: The initiator client presents a chat interface including interactive controls.

[0167] Among them, the interactive controls are used to evoke AR card interactive functions.

[0168] Optionally, the interactive control can be presented directly on the chat interface, or it can be presented through some entrances, such as by clicking on the extension option in the chat interface, etc.

[0169] like Figure 3A As shown, it is a schematic diagram of an interactive control provided in an embodiment of the present application. Figure 3A A chat interface containing interactive controls is presented, wherein the control name S301 is "AR card interaction", which is an interactive control listed in the embodiment of the present application. Figure 3AIn the chat interface, the interactive control is directly presented in the chat interface. After the object enters the chat interface, the interactive control can be directly seen below the input box.

[0170] like Figure 3B As shown, it is a schematic diagram of another interactive control provided in an embodiment of the present application. Figure 3B In the chat interface shown, the interactive controls are not directly presented below the input box, but need to be evoked through the extended options. Among them, S302 is the chat interface presented when the object has not clicked the extended option. At this time, the interactive controls have not yet appeared in the chat interface. After the object clicks the extended option shown in S303, a chat interface S304 including an extended panel as shown in S304 will be presented. In S304, a control S305 with the name "AR Card Interaction" is presented. S305 is the interactive control.

[0171] Furthermore, the object can trigger subsequent card interactions and other operations based on the interactive control in the chat interface.

[0172] S22: The initiator client presents a card interaction area in the chat interface in response to the interactive operation triggered by the initiator object on the interactive control.

[0173] In the embodiment of the present application, the object that actively triggers the interactive operation in the online chat is called the initiator object, and the interactive operation is triggered by the initiator object to enable the AR interactive function.

[0174] The initiator can activate the AR interactive function by clicking on an interactive control, and then a card interactive area will appear in the chat interface. In addition, the initiator can also activate the AR interactive function by voice input or by entering keywords, etc. This application does not impose any specific restrictions on this.

[0175] In the embodiments of the present application, there is no specific limitation on the presentation method of the card interaction area. For example, the card interaction area can be presented in the form of a floating window at any position in the chat interface, or the card interaction area can be embedded in any area in the chat interface, such as the upper area or the lower area, etc. This application does not make specific limitations on this.

[0176] like Figure 4A As shown, it is a schematic diagram of a card interaction area provided in an embodiment of the present application. Figure 4A The card interaction area S401 is presented in the chat interface as a floating window. After the initiator clicks the interactive control, S401 will be presented in the chat interface. The initiator can drag the floating window to appear anywhere in the chat interface, or scale the floating window to make its size meet the needs of the initiator.

[0177] like Figure 4B As shown, it is a schematic diagram of another card interaction area provided in an embodiment of the present application. Figure 4B The card interaction area S402 is presented in an embedded manner in the upper area of ​​the chat interface. After the initiator clicks the interactive control, S401 will be presented in the chat interface. At this time, the area in the chat interface where conversation messages can be presented is S403. The initiator can drag the bottom edge of S402 or the top edge of S403 to adjust the size of the card interaction area.

[0178] In an embodiment of the present application, when the initiator object triggers an interactive operation for an interactive control, the real-time environmental information of the initiator object collected will be presented in the card interaction area through the camera function at the same time.

[0179] Specifically, the initiator object triggers an interactive operation based on the interactive control, and the initiator client responds to the interactive operation triggered by the interactive control to call up the camera function; then, a card interaction area is presented in the chat interface, and the card interaction area includes the initiator's environmental information collected by the camera function.

[0180] The initiator's environmental information refers to the actual environmental information of the initiator's object side collected by the camera.

[0181] For example, the initiator object can recognize the keyboard, cards, etc. through the rear camera of the terminal device. These actually recognized living environments all belong to the environmental information in this article.

[0182] Among them, when calling up the camera function, you can obtain camera permissions from the initiator object through a pop-up window in the chat interface. After the initiator object allows authorization of camera permissions, you can call up the camera function.

[0183] like Figure 5 As shown, it is a schematic diagram of invoking the camera function provided in an embodiment of the present application. Figure 5 After the object clicks the interactive control, a pop-up window requesting the use of the camera function as shown in S501 will pop up in the chat interface. After the initiator clicks to agree, the card interaction area shown in S502 containing the initiator's environmental information collected by the camera function will be presented in the chat interface.

[0184] In the above embodiment, the interactive operation is triggered by the initiator object based on the interactive control, and a card interaction area containing the initiator's environmental information is presented in the chat interface to facilitate subsequent identification of the first card.

[0185] It should be emphasized here that the above process needs to invoke the camera function based on the interactive operation triggered by the object and present the card interaction area containing the initiator's environmental information. In this case, the data generated by the interactive operation triggered by the object and the initiator's environmental information collected by the camera function will be involved. The collection, use and processing of this data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0186] Furthermore, the acquisition of this data requires the permission or consent of the initiator. Figure 5 As shown, before invoking the camera function, the initiator object can be asked whether the camera function can be used. After the initiator object agrees, the initiator environment information of the initiator object can be further collected through the camera function; alternatively, the object can be prompted to invoke the camera function to collect the initiator environment information, etc.

[0187] After the initiator object triggers the interactive operation on the interactive control, it can select the first card it wants to show to the recipient object, identify it, and then present the corresponding first virtual character.

[0188] S23: When the initiator client recognizes the first card input by the initiator object in the card interaction area, it presents the initiator AR screen including the first virtual character corresponding to the first card, and shares the initiator AR screen with at least one recipient object associated with the initiator object.

[0189] The first virtual character is virtual content generated by combining AR technology with entity recognition of the first card.

[0190] In the embodiment of the present application, the entity in the card refers to the main image in the card, which can be a character entity, a scene entity, a special effect entity, a game equipment entity, etc.

[0191] The embodiments of the present application can also be applied to generate corresponding virtual characters based on characters or objects containing certain elements in combination with AR technology. For example, a character wearing a lot of apple elements can be transformed into a corresponding "apple" virtual character. The embodiments of the present application can also be applied to converting real objects into corresponding cartoon virtual characters to achieve the anthropomorphism of real objects. The embodiments of the present application can also generate corresponding virtual images based on entities other than cartoon characters. For example, a corresponding virtual house can be generated based on entities such as houses in playing cards, etc. This application does not make any specific limitations on this.

[0192] Specifically, the initiator object can place the first card of his / her choice in the area captured by the camera function (referred to as the camera area). In this way, when the initiator client recognizes the first card input by the initiator object in the initiator environment information through the camera function, the initiator AR screen containing the first virtual character can be presented.

[0193] like Figure 6 As shown, it is a schematic diagram of a first virtual character provided by an embodiment of the present application. After the initiator object clicks the interactive control and agrees to use the camera function, the card interaction area immediately presents the initiator's environmental information collected by the camera function, and the text "Please aim at the target to be identified" will be displayed in the chat interface to prompt the initiator object that the card can be identified. After the initiator object places the first card of its choice in the corresponding camera area, the first card will be presented in the card interaction area, and the text "Identifying..." will be displayed in the chat interface to prompt the initiator object that the first card is being identified (such as scanned). After the recognition of the first card is completed, the text "Recognition successful!" will be displayed in the chat interface to prompt the initiator object that the recognition is complete, and the initiator's AR screen containing the first virtual character will be presented in the initiator's environmental information.

[0194] In addition, it should be noted that if the initiator object has placed the first card in the camera area before invoking the camera function, then after the object invokes the camera function, the card interaction area can be presented in the chat interface. At this time, the card interaction area can directly recognize the first card, and thus can directly recognize the entity in the first card, and after the recognition (such as scanning) is completed, the initiator's AR screen will be synchronously presented in the card interaction area.

[0195] In the above embodiment, the first card placed by the initiator object is presented in the initiator environment information of the card interaction area through the camera function, so as to facilitate the subsequent presentation of the first virtual character corresponding to the first card in the card interaction area.

[0196] It should be noted that the subject can place the card in the corresponding camera area after invoking the camera function. In this case, the first card needs to be identified after the camera function collects the initiator's environmental information. In addition, the subject can also place the card in the corresponding camera area in advance before invoking the camera function. In this way, the first card can be identified while collecting environmental information after the camera function is invoked. This application does not make specific restrictions on this.

[0197] In an embodiment of the present application, in order to enable subsequent interaction between multiple virtual characters in the card interaction area, when presenting environmental information and virtual characters in the card interaction area, the environmental information and virtual characters are placed in unused layers.

[0198] Specifically, the initiator's environmental information is located in the camera image layer in the initiator's AR image, the first virtual character is located in the model layer in the initiator's AR image, and the model layer is located above the camera image layer.

[0199] In addition, when the initiator object triggers an interactive operation for the interactive control, the server will send an invitation message to the other party in the chat, that is, the receiving object, to invite it to join the AR card interaction function.

[0200] In this embodiment of the present application, an optional implementation method of the initiator object sending the invitation request is as follows:

[0201] The initiator object triggers an interactive operation for the interactive control, and the initiator client sends an invitation request triggered based on the interactive control to at least one recipient object, and presents corresponding invitation information in the chat area of ​​the chat interface.

[0202] In an embodiment of the present application, the online chat scenario in which the initiator object is located can be a multi-person group chat scenario, in which case there can be one or more recipient objects; or it can be a two-person private chat scenario, in which case there is only one recipient object.

[0203] Specifically, in a private chat scenario between two objects, object A triggers the above interactive operation in the chat interface with object B, then object A is the initiator object and object B is the recipient object.

[0204] In a multi-person group chat scenario, object A can trigger the above interactive operations in a group chat interface. In this case, object A is the initiator, and the recipient object can be divided into the following two cases:

[0205] Object A can select the recipient object with which he wants to interact with the card in the group;

[0206] Object A may not make a selection, and then all other members in the group will serve as the recipient objects.

[0207] Specifically, after the initiator object triggers the interactive operation, the initiator client sends an invitation request to the server, and then the server forwards the invitation request to the receiving client corresponding to the receiving object. In addition, the corresponding invitation information will be presented on both the initiator side and the receiving side, such as Figure 7 The message "Waiting for the other party to accept (30s)..." is displayed in the card interaction area (on the initiator's side). Figure 14 The message "The other party invites you to join AR card interaction" is displayed in the chat area (on the receiving side).

[0208] Based on the above, the recipient can choose to accept the invitation, reject the invitation, or ignore the invitation.

[0209] If one or more recipient objects accept the invitation, the initiator's AR image can be shared with the recipient object(s) who accepted the invitation. The recipient object(s) who accepted the invitation are referred to herein as the target recipient object. The target recipient object(s) can be one or more, and this is not a specific limitation.

[0210] In an embodiment of the present application, after determining that a target recipient object among at least one recipient object accepts the invitation, the initiator client shares the initiator's AR screen with the target recipient object.

[0211] Specifically, to share the initiator's AR screen with the target recipient object, the server needs to send the initiator's object environment information, the first model information corresponding to the first card, etc. to the recipient's terminal device. The recipient's terminal device then renders according to the initiator's object environment information, the first model information, etc. to present the initiator's AR screen on the recipient's terminal device.

[0212] Among them, a preset waiting time can be added to the invitation request. If the target recipient object does not make any feedback to the invitation request within the preset waiting time, then when the preset waiting time is reached, it is defaulted that the target recipient object will not join this AR card interaction function.

[0213] In the embodiment of the present application, the invitation information in the chat area can be actively presented by the initiator's client, or it can be the relevant invitation information entered by the initiator himself. This application does not make specific limitations on this.

[0214] like Figure 7 As shown, it is a schematic diagram of an invitation message provided by an embodiment of the present application. After the initiator clicks the interactive control, the client automatically sends an invitation message to the recipient, with a preset waiting time of 30 seconds. A text prompt is displayed in the card interaction area: "Waiting for the other party to accept (30s)...", and an invitation message "Accept quickly! Show you my new card!" is displayed in the chat area.

[0215] If the target recipient accepts the invitation, a text prompt will be displayed in the chat area: The other party has accepted your invitation. At this time, the initiator's AR screen is simultaneously displayed on the initiator's terminal device and the recipient's terminal device. If the initiator adjusts the initiator's environmental information collected by the camera area at this time, the initiator's environmental information presented in the card interaction area of ​​the recipient's terminal device will also change accordingly; when the initiator places the first card in the camera area, the recipient's card interaction area will also present the initiator's prevention behavior and the first card. That is, after the initiator shares the initiator's AR screen with the recipient, the recipient can synchronously perceive all operations performed by the initiator in the camera area and the card interaction area through the card interaction area.

[0216] It should be noted that the invitation information can be sent to the target recipient object first, and the target recipient object recognizes the first card after receiving the invitation, or the first card can be recognized first and then the invitation information is sent to the target recipient. This application does not make specific limitations on this.

[0217] In the above embodiment, by sending an invitation request to the recipient object and sharing the initiator's AR screen with the target recipient object after the target recipient object accepts the invitation, the playability of the physical card in the online chat scenario is improved, which is conducive to the initiator object and the target recipient object to further explore the card.

[0218] It should be emphasized that the acquisition of data such as interactive operations, invitation requests, invitation information, receiving operations of the recipient object, and initiator AR screens triggered by the above-mentioned initiator object has also been authorized or agreed by the initiator object and the target recipient object, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0219] In addition, in order to further improve the interactivity and playability of the cards, in this application, the target recipient object can not only view the initiator's AR screen shared by the initiator object, but also join the card interaction and show the initiator object the second card of his or her choice.

[0220] In this embodiment of the present application, an optional implementation method for the target recipient object to join this card interaction is as follows:

[0221] The target recipient object can join this card interaction by adding a control. After the initiator client confirms that the target recipient object has joined this card interaction, it can present the initiator's AR screen through the initiator window in the card interaction area, and present the recipient's AR screen corresponding to the target recipient object through the recipient window.

[0222] The receiving AR screen includes: a second virtual character corresponding to the second card input by the target receiving object; the second virtual character is virtual content generated by combining AR technology after entity recognition of the second card.

[0223] In an embodiment of the present application, when the initiator window and the receiver window are presented in the card interaction area, the window presentation layout can be presented side by side left and right, side by side top and bottom, or in other arrangement modes, etc. This application does not make any specific restrictions on this.

[0224] Specifically, after receiving the invitation, the target recipient object can only view the first card and the first virtual character shared by the initiator object, or can choose to join the AR card interaction function and show the second card of his / her choice to the initiator object.

[0225] After the target recipient object joins this card interaction, the first virtual character corresponding to the initiator object side and the second virtual character corresponding to the target recipient object side can be simultaneously presented in the card interaction area.

[0226] like Figure 8 As shown, it is a schematic diagram of a target recipient object joining the interaction provided by an embodiment of the present application. After the target recipient object receives the invitation from the initiator object, it sends a dialogue message "Great!" in the chat area and chooses to join the AR card interaction function. At this time, a text prompt is displayed in the chat area of ​​the chat interface: the other party has joined the AR card interaction, and the recipient AR screen of the second virtual character corresponding to the second card input by the target recipient object is displayed in the card interaction area. Figure 8 In the card interaction area, the initiator window and the receiver window are presented side by side. The left side is the initiator window, which presents the initiator's AR screen, and the right side is the receiver window, which presents the receiver's AR screen.

[0227] In the above embodiment, the target recipient object selects the second card for identification, and the identified second virtual character is presented in the card interaction area, so that both the target recipient object and the initiator object can view the second virtual character, thereby improving the playability of the physical card in the online chat scenario, and facilitating the initiator object and the target recipient object to further explore the card.

[0228] It should be emphasized that the acquisition of data generated by the joining operation of the above-mentioned target recipient object, the second card, the second virtual character, etc., has also been obtained with the permission or consent of the target recipient object, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0229] In the above embodiment, the initiator object and the target recipient object each share their own cards and present corresponding virtual characters, thereby improving the interactivity and playability of physical cards in the online chat scene.

[0230] In addition, in order to further improve the interactivity and playability of the cards, this application also supports the object to control the virtual characters presented in the card interaction area to perform corresponding interactive behaviors in a variety of ways.

[0231] After recognizing that the first card input by the initiator object presents the first virtual character, the first virtual character in the card interaction area, or the first virtual character and the second virtual character, can be caused to perform corresponding behaviors through behavioral instructions.

[0232] In the embodiment of the present application, an optional implementation method of controlling the virtual character to interact is as follows:

[0233] The initiator object triggers a behavior instruction for the first virtual character, and the initiator client presents an interactive animation corresponding to the behavior instruction in the card interaction area in response to the behavior instruction triggered for the first virtual character.

[0234] Specifically, if the behavior instruction is a behavior instruction for a single character, then the interactive animation indicates: controlling the first virtual character to perform the behavior corresponding to the behavior instruction;

[0235] like Figure 9A As shown, it is a schematic diagram of controlling a virtual character provided by an embodiment of the present application. Figure 9A Continued use Figure 6 Assume that the card interaction area only includes the first virtual character of the initiator object. At this time, the initiator object triggers the action instruction "#ARCardInteractionTurn around and take a look" for the first virtual character, then the first virtual character in the card interaction area will execute the action instruction and turn around.

[0236] When a second virtual character is also presented in the card interaction area, if the behavior instruction is an interactive behavior instruction for multiple characters, then the interactive animation representation is: controlling the first virtual character and the second virtual character to execute the interactive behavior corresponding to the behavior instruction.

[0237] like Figure 9B As shown, it is a schematic diagram of another method of controlling a virtual character provided by an embodiment of the present application. Figure 9B Continued use Figure 8 Assume that the card interaction area includes the first virtual character of the initiator object and the second virtual character of the recipient object. At this time, the initiator object triggers the "#AR card interaction duel" behavior instruction for the first virtual character, and at the same time, the recipient object triggers the "#AR card interaction duel" behavior instruction for the second virtual character. Then, the first virtual character and the second virtual character in the card interaction area will execute the duel behavior instruction, and the interactive animation presented in the card interaction area is the duel behavior corresponding to the execution of the behavior instruction by the first virtual character and the second virtual character.

[0238] like Figure 9C As shown, it is a schematic diagram of another method of controlling a virtual character provided by an embodiment of the present application. Figure 9C Continued use Figure 9BAssume that after triggering the behavior instruction, the initiator object selects a new first card for identification and presents the corresponding new first virtual character in the card interaction area. The initiator object triggers "#ARCardInteractionHugHer" for the new first virtual character. Since this behavior instruction requires two virtual characters to hug at the same time, this behavior instruction is actually a behavior instruction for two characters. At this time, the interactive animation presented in the card interaction area is the first virtual character and the second virtual character performing the hugging behavior corresponding to the behavior instruction.

[0239] In the above embodiment, by triggering behavioral instructions and controlling the first virtual character, or the first virtual character and the second virtual character to perform corresponding behaviors, the playability of physical cards in online chat scenarios is improved, which is conducive to the initiator object and the target recipient object to further explore the card, and expands the application scope and usage of AR technology in online chat scenarios.

[0240] It should be emphasized that the acquisition of data generated by the above-mentioned behavioral instructions, interactive animations, etc. is also subject to the permission or consent of the subjects, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0241] Depend on Figure 8 、 Figure 9B and Figure 9C It can be seen that in the embodiment of the present application, the initiator's AR screen and the recipient's AR screen are presented through different windows in the card interaction area.

[0242] Specifically, in the card interaction area, the initiator's AR screen is presented through the initiator's window, and the receiver's AR screen corresponding to the target receiver object is presented through the receiver's window.

[0243] When the initiator object or the recipient object triggers a behavior instruction for the first virtual character and the second virtual character, requiring the first virtual character and the second virtual character to perform an interactive behavior in the card interaction area, the interactive animation corresponding to the behavior instruction is presented in the model layer of the initiator's AR screen and the recipient's AR screen;

[0244] Among them, there is no window boundary restriction on the model layer between each window.

[0245] Therefore, see Figure 9B and Figure 9C As shown in the schematic diagram of controlling virtual characters, when the first virtual character and the second virtual character are performing interactive behaviors, the animation of the first virtual character and the animation of the second virtual character both break through their respective windows.

[0246] In the above embodiment, since the first virtual character and the second virtual character are both rendered in the model layer, interactive behaviors between any virtual characters can be realized, breaking through the spatial limitations of visual concepts and achieving "cross-screen" visual effects. Furthermore, the interactivity of physical cards in online chat scenarios is improved, and the exploration needs of the initiator and the recipient objects for the cards are stimulated.

[0247] It should be noted that the above examples are based on the initiator object inputting one card at a time, or a card containing only one entity. In addition, the initiator object can also input multiple cards at a time, or the initiator object can input a card containing multiple entities. In this case, there can also be multiple first virtual characters.

[0248] Similarly, the recipient object may also input multiple cards at one time, or the recipient object may input a card containing multiple entities, in which case there may also be multiple second virtual characters.

[0249] The following takes the initiator object as an example:

[0250] In an embodiment of the present application, there may be multiple first virtual characters. That is, the initiator object may simultaneously identify multiple first cards during a single first card identification process, thereby presenting multiple first virtual characters in the card interaction area. The initiator object may also select to identify a first card containing multiple entities during a single first card identification process, thereby presenting a corresponding first virtual character for each entity in the first card in the card interaction area, and so on. This application does not impose specific limitations on this.

[0251] In the embodiment of the present application, when controlling at least one target first virtual character, an optional implementation method is as follows:

[0252] The initiator object triggers a behavior instruction for at least one target virtual character among multiple first virtual characters. The initiator client responds to the behavior instruction input for the first card, determines the target first virtual character described by the behavior instruction from the multiple first virtual characters, and presents the interactive animation corresponding to the behavior instruction in the card interaction area.

[0253] The interactive animation representation is: at least controlling the target first virtual character to execute the behavior corresponding to the behavior instruction.

[0254] At this time, the behavior instruction triggered by the initiator object includes a description of the target first virtual character for executing the instruction. For example, the target first virtual character can be described by the location information of each first virtual character, or by the default name of each first virtual character, etc. This application does not make specific restrictions on this.

[0255] like Figure 10A As shown, it is a schematic diagram of a control target first virtual character provided by an embodiment of the present application. Figure 10A During the process of identifying the first card, the initiator object simultaneously identified two first cards, and then two different first virtual characters were presented side by side in the card interaction area. The initiator object needed to turn the first virtual character on the left. It triggered "#ARCardInteractionTurn to see the left one" in the chat area for the first virtual character. By adding the location information "left" to the action instruction, the first virtual character on the left was designated as the target first virtual character. Then, the first virtual character on the left in the card interaction area would execute the action instruction and turn around.

[0256] In the embodiment of the present application, when controlling at least one target first virtual character, another optional implementation is as follows:

[0257] After the initiator object selects a target first virtual character from multiple first virtual characters, it triggers a behavior instruction for the target virtual character. In response to the behavior instruction triggered for the target first virtual character, the client presents an interactive animation corresponding to the behavior instruction in the card interaction area.

[0258] The interactive animation representation is: at least controlling the target first virtual character to execute the behavior corresponding to the behavior instruction.

[0259] In an embodiment of the present application, the initiator can select the target first virtual character by directly clicking on the first virtual character that the initiator wants to select. Alternatively, preset default names corresponding to various first virtual characters may be presented in the card interaction area, and the initiator can select the target first virtual character by clicking on the preset default name of the first virtual character that the initiator wants to select, etc. This application does not make any specific limitations on this.

[0260] like Figure 10B As shown, it is a schematic diagram of another control target first virtual character provided by an embodiment of the present application. Figure 10B During the process of identifying the first card, the initiator object simultaneously identified two first cards. Consequently, two different first virtual characters were presented side by side in the card interaction area. Each first virtual character presented its preset default name: the first virtual character on the left was Xiaohong, and the first virtual character on the right was Xiaohuang. The initiator object needed to turn the first virtual character on the left around. Therefore, it clicked on the name "Xiaohong," thereby selecting the first virtual character on the left as the target first virtual character. It also triggered "#ARCardInteractionTurn around on the left" for the first virtual character in the chat area. The first virtual character on the left in the card interaction area will then execute the action instruction and turn around.

[0261] In addition, in the embodiments of the present application, the number of virtual characters that can be presented in the card interaction area is not limited to 2. Any number of virtual characters can be loaded within the range allowed by the performance. Moreover, only the virtual characters presented in the card interaction area, that is, the visible areas of the initiator object and the recipient object, are rendered, and other virtual characters are not rendered or calculated, further reducing power consumption and improving performance and experience. Thereby, the models and special effects of scenes, backgrounds, etc. can be further increased, further enhancing the performance and performance effects of card interaction.

[0262] In the above implementation, the initiator object can identify multiple first virtual characters at one time and can select the target first virtual character it wants to control through various methods, thereby expanding the playability of the physical card in the online chat scenario.

[0263] It should be emphasized that the acquisition of data such as the above-mentioned multiple first virtual characters, target first virtual characters, and behavior instructions is also permitted or consented to by the initiator object, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0264] In the embodiments of the present application, there are many ways for the above-mentioned initiator object and recipient object to trigger behavior instructions. Optionally, the behavior instructions are triggered by at least one of the following methods:

[0265] Method 1: Trigger the behavior instruction by inputting a behavior keyword.

[0266] [[ID=1,7]]Among them, the behavior keyword is used to describe the behavior that the first virtual character needs to execute.

[0267] For details, please refer to Figure 9A the schematic diagram of controlling virtual characters shown in Figure 9A In the figure, the behavior keyword input by the initiator object is "#AR Card Interaction Turn around and take a look". Among them, "AR Card Interaction" is used to determine that the keyword input this time is for AR card interaction, and "Turn around and take a look" is the behavior keyword, which is used to describe the behavior that the first virtual character needs to execute, that is, to turn around.

[0268] Method 2: Trigger the behavior instruction by inputting voice.

[0269] Among them, the voice is used to describe the behavior that the first virtual character needs to execute.

[0270] As shown in Figure 11A this is a schematic diagram of triggering a behavior instruction provided by an embodiment of the present application. Figure 11A For continuity with Figure 6Assume that the initiator invites the recipient to view the first virtual character it recognizes through a 5-second voice message, with the voice content being "AR card interaction, turn around and have a look." "AR card interaction" is used to determine that the voice input this time is for AR card interaction, and "turn around and have a look" is used to describe that the first virtual character needs to perform the action of turning around.

[0271] Method 3: Trigger interactive instructions through actions.

[0272] Specifically, the input action is recognized through the camera function, and the behavioral instruction is triggered based on the action.

[0273] The action is used to describe the behavior that the first virtual character needs to perform.

[0274] like Figure 11B As shown, it is a schematic diagram of another triggering behavior instruction provided in an embodiment of the present application. Figure 11B After the initiator object and the recipient object each identify a card, the first virtual character and the second virtual character are presented in the card interaction area. At this time, the initiator object and the recipient object recognize the "heart" actions input by each of them through the camera function. Based on the initiator object's heart action, the first virtual character presents the corresponding heart special effect, and based on the recipient object's heart action, the second virtual character also presents the corresponding heart special effect.

[0275] like Figure 11C As shown, it is a schematic diagram of another triggering behavior instruction provided in an embodiment of the present application. Figure 11C After the initiator object and the recipient object each identify a card, the first virtual character and the second virtual character are presented in the card interaction area. At this time, the initiator object and the recipient object recognize the "heart" actions input by each of them through the camera function. Based on the heart actions presented by the initiator object and the recipient in cooperation, corresponding heart special effects are presented at the first virtual character and the second virtual character in the card interaction area.

[0276] It should be noted that the actions input by the initiating object and the receiving object can be pre-set actions. For example, a pre-set "heart" action can trigger the virtual character to display a heart-shaped special effect. A pre-set "fist" action can also be pre-set to trigger the virtual character to perform a "duel" behavior, and so on. This application does not specifically limit this. Based on the pre-set actions, the first virtual character and the second virtual character can be controlled to perform corresponding behaviors.

[0277] Method 4: Triggering action instructions through action controls.

[0278] Specifically, at least one behavior control is presented in response to a behavior instruction triggered by selecting a target behavior control from among the at least one behavior control.

[0279] The behavior control is used to describe the behavior that the first virtual character needs to perform.

[0280] like Figure 11D As shown, it is a schematic diagram of a behavior control provided in an embodiment of the present application. Figure 11D The card interaction area in the game displays a first virtual character and a behavior control for controlling the virtual character. The initiator can click the behavior control to display multiple sub-controls: Turn, Duel, and Heart. The initiator drags the behavior control to the desired sub-control. For example, if the initiator wants the first virtual character to turn, they can drag the behavior control to the Turn sub-control to control the first virtual character in the card interaction area.

[0281] In the above implementation, the initiator object can trigger the behavior instruction in a variety of ways, which expands the control method and interactivity of the initiator object over the first virtual character in the online chat scenario.

[0282] It should be emphasized that the acquisition of data on the above-mentioned behavioral keywords, voice, actions, etc. has also been authorized or agreed by the initiator object and the target recipient object, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0283] In addition, the initiator object can also adjust the size of the virtual character in the card interaction area by directly using gestures.

[0284] In the embodiment of the present application, when the initiator object scales the first virtual character, an optional implementation method is as follows:

[0285] The initiator object triggers an adjustment gesture for the first virtual character. In response to the adjustment gesture triggered for the first virtual character, the initiator client performs a screen zooming operation corresponding to the adjustment gesture on the first virtual character, and presents the first virtual character after the screen zooming operation is performed in the card interaction area.

[0286] like Figure 12A As shown, it is a schematic diagram of enlarging a first virtual character provided by an embodiment of the present application. Figure 12A For continued use Figure 6 Assume that after the first virtual character is presented in the card interaction area, the initiator object enlarges the first virtual character through an adjustment gesture, performs a screen zooming operation corresponding to the adjustment gesture on the first virtual character, and presents the first virtual character after the screen zooming operation in the card interaction area.

[0287] like Figure 12B As shown, it is a schematic diagram of reducing the size of a first virtual character provided by an embodiment of the present application. Figure 12B For continued use Figure 6 Assume that after the first virtual character is presented in the card interaction area, the initiator object shrinks the first virtual character through an adjustment gesture, performs a screen zooming operation corresponding to the adjustment gesture on the first virtual character, and presents the first virtual character after the screen zooming operation in the card interaction area.

[0288] In the above embodiment, the initiator object can adjust the gesture to enlarge the first virtual character to facilitate viewing of more details of the first virtual character, or can adjust the gesture to reduce the first virtual character to facilitate viewing of the full body image of the first virtual character.

[0289] It should be emphasized that the acquisition of data such as the above-mentioned adjustment gestures has also been authorized or agreed by the initiator, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0290] In addition, embodiments of the present application can also be applied to generate corresponding virtual characters by combining certain characters or objects containing certain elements with AR technology. For example, a character wearing a lot of apple elements can be transformed into a corresponding "apple" virtual character. It can also be applied to converting real objects into corresponding cartoon virtual characters to achieve the anthropomorphism of real objects. It can also generate corresponding virtual images based on entities other than cartoon characters. For example, a corresponding virtual house can be generated based on entities such as houses in playing cards, etc. This application does not make specific limitations on this.

[0291] It should be noted that the above mainly illustrates the AR-based card interaction method in the embodiment of the present application from the initiator client side. The following further illustrates the AR-based card interaction method in the embodiment of the present application from the recipient client side.

[0292] See Figure 13 As shown in FIG, it is an implementation flow chart of another AR-based card interaction method in an embodiment of the present application. Taking the client as the execution subject as an example, the specific implementation process of this method is as follows S131-S132:

[0293] S131: The receiving client presents a chat interface.

[0294] S132: The receiving client responds to the receiving object's acceptance operation of the invitation request triggered by the initiating object by presenting a card interaction area in the chat interface. The card interaction area includes an initiator's AR screen containing a first virtual character obtained after entity recognition of the first card input by the initiating object.

[0295] Among them, the first virtual character is a virtual content generated by combining AR technology after entity recognition of the first card; the invitation request is sent by the initiator object based on the interactive control in the chat interface.

[0296] Specifically, when the initiator object triggers an invitation request, the recipient object will receive the invitation information. At the same time, an acceptance control will be presented in the chat interface. After the recipient object agrees to the invitation based on the acceptance control, the card interaction area will be presented in the chat interface, and the first virtual character shared by the initiator object can be viewed.

[0297] In the embodiment of the present application, when the recipient object accepts the invitation request, an optional implementation method is as follows:

[0298] The receiving object receives the invitation request triggered by the initiating object, and the receiving client presents the corresponding invitation information sent by the initiating object and the acceptance control in the chat area of ​​the chat interface; the receiving object triggers an acceptance operation for the invitation request based on the acceptance control, and the receiving client responds to the acceptance operation for the invitation request triggered by the receiving object based on the acceptance control, and presents the card interaction area in the chat interface.

[0299] Among them, a preset waiting time can be added to the invitation request, and the acceptance control presented in the chat area of ​​the recipient object is also accompanied by the preset waiting time. If the recipient object does not make any feedback to the invitation request within the preset waiting time, then when the preset waiting time is reached, it is assumed that the recipient object will not join this AR card interaction function.

[0300] like Figure 14 As shown, it is a schematic diagram of receiving an invitation request provided in an embodiment of the present application. Figure 14 and Figure 7 Corresponding to the assumption, Figure 7 The initiator object sends an invitation request to the recipient object. Figure 14 The middle part shows the invitation message received by the recipient, which reads, "Accept now! Show me my new card!" An acceptance control and a text prompt appear in the chat area: "The other party has invited you to join the AR card interaction. The default waiting time is 30 seconds." After the recipient triggers the acceptance action based on the acceptance control, the card interaction area appears in the chat interface, along with a text prompt: "You have accepted the invitation."

[0301] In the above embodiment, the recipient object can view the card interaction area in the chat interface by accepting the invitation request sent by the initiator object, so as to facilitate subsequent viewing of the first virtual character shared by the initiator object.

[0302] After the recipient object accepts the invitation, the initiator object places the first card selected by himself in the corresponding camera area, and then the recipient object can view the initiator's AR screen containing the first virtual character corresponding to the first card in the card interaction area.

[0303] like Figure 15 , which is a schematic diagram of a shared initiator's AR screen provided by an embodiment of the present application. After the recipient triggers an acceptance operation for the invitation request based on the acceptance control, the initiator places the first card selected by it in the corresponding camera area. The recipient can then view the first card and the initiator's AR screen containing the first virtual character in the card interaction area.

[0304] In addition, the chat interface also includes a join control. In addition to being able to view the initiator's AR screen shared by the initiator, the recipient object can also join the card interaction through the join control and show the initiator object the second card of their choice.

[0305] The receiving object triggers a join operation based on the join control, and the receiving client responds to the join operation triggered by the join control, joins this card interaction, and calls up the camera function; when the receiving client recognizes the second card input by the receiving object through the camera function, in the card interaction area, the initiator's AR screen is presented through the initiator's window, and the receiving AR screen containing the second virtual character corresponding to the second card is presented through the receiving window.

[0306] The second virtual character is virtual content generated by combining AR technology with entity recognition of the second card.

[0307] In an embodiment of the present application, when the card interaction area presents the initiator window and the receiver window, the window presentation layout can be presented side by side left and right, or side by side top and bottom. This application does not make any specific restrictions on this.

[0308] like Figure 16A As shown in FIG, it is a schematic diagram of a receiving object joining the interaction provided by an embodiment of the present application. After the receiving object receives the invitation from the initiating object, it sends a dialogue message "Great!" in the chat area and chooses to join the AR card interaction function. At this time, a text prompt is displayed in the chat area of ​​the chat interface: You have joined the AR card interaction, and the receiving object's AR picture of the second virtual character corresponding to the second card input by the receiving object is displayed in the card interaction area. Figure 16A In the card interaction area, the initiator window and the receiver window are presented side by side. The left side is the initiator window, which presents the initiator's AR screen, and the right side is the receiver window, which presents the receiver's AR screen.

[0309] After the receiving object receives the first virtual character shared by the initiating object and recognizes the second card input by the receiving object, the first virtual character in the card interaction area, or the first virtual character and the second virtual character, can be caused to perform corresponding actions through behavioral instructions.

[0310] like Figure 16B As shown, it is a schematic diagram of another receiving object joining the interaction provided by an embodiment of the present application. Figure 16B Continued use Figure 8 Assume that the card interaction area includes the first virtual character of the initiator object and the second virtual character of the recipient object. At this time, the initiator object triggers the "#AR card interaction duel" behavior instruction for the first virtual character, and at the same time, the recipient object triggers the "#AR card interaction duel" behavior instruction for the second virtual character. Then, the first virtual character and the second virtual character in the card interaction area will execute the duel behavior instruction, and the interactive animation presented in the card interaction area is the duel behavior corresponding to the execution of the behavior instruction by the first virtual character and the second virtual character.

[0311] In the above embodiment, the initiator object and the recipient object each share their own cards and present corresponding virtual characters. After triggering behavioral instructions for the virtual characters, the corresponding virtual characters are controlled to perform corresponding behaviors, thereby improving the interactivity and playability of physical cards in online chat scenarios.

[0312] It should be emphasized that the acquisition of data generated by the above-mentioned recipient object's joining operation, second card, second virtual character, behavioral instructions, etc. has also been obtained with the permission or consent of the recipient object, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0313] In addition, the receiving object can also trigger behavior instructions for the first virtual character and the second virtual character in the above manner, thereby presenting interactive animation. For specific implementation methods, please refer to the above embodiments, and repeated parts will not be repeated.

[0314] The above mainly illustrates the AR-based card interaction method in the embodiment of the present application from the initiator client side and the recipient client side. The following further illustrates the AR-based card interaction method in the embodiment of the present application from the server side.

[0315] See Figure 17 As shown in FIG. 1 , it is an implementation flow chart of another AR-based card interaction method in an embodiment of the present application. Taking the server as the execution subject as an example, the specific implementation process of the method is as follows S171 to S173:

[0316] S171: After receiving the recognition request for the first card input by the initiator object, the server determines first model information corresponding to the entity in the first card by performing entity recognition on the first card.

[0317] Among them, card identification information can be set in the card. If the server detects that the card contains card identification information, it can be determined that a virtual character can be generated based on the entity in the card. If the card is not recognized to contain card identification information, it can be determined that a virtual character cannot be generated based on the entity in the card, and the unrecognizable information is fed back to the client.

[0318] Each card has a corresponding card identification information that uniquely identifies the entity and other information contained in the card. Card identification information can be in the form of a barcode, QR code, card serial number, entity identification number, etc., which is not specifically limited in this article.

[0319] Specifically, after the server recognizes the card identification information of the card, it determines the entities in the first card. Each entity corresponds to an entity identification number (Identity Document, ID). After determining the entity ID, the server returns the corresponding content delivery network (CDN) download address and obtains the corresponding model information from the CDN download address.

[0320] For example, you can use Figure 6 Assume that the initiator object will Figure 6 After the first card shown in is placed in the corresponding camera area, the initiator client sends an identification request of the initiator object for the first card to the server. After receiving the identification request, the server identifies the first card. After identifying the card identification information, it determines the entity in the first card. For example, the entity ID of the entity may be 17. Then, the server returns the CDN download address with the entity identification number 17 to obtain the first model information.

[0321] It should be noted that the entity recognition in the embodiments of the present application can be performed solely by the server or by the client. When performed solely by the client, the client specifically identifies the entity ID of the entity in the card and then sends the entity ID to the server. The server returns the corresponding CDN download address based on the entity ID and obtains the model information. Alternatively, the client can perform preliminary recognition and the server further recognizes it. In this case, the client can identify the entity in the card and send the entity to the server. The server determines the entity ID of the entity, returns the corresponding CDN download address, obtains the model information, and so on. This application does not make specific limitations on this.

[0322] It should be emphasized that the acquisition of data such as the first card, invitation request, and acceptance request mentioned above has also been obtained with the permission or consent of the initiator, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0323] S172: The server feeds back the first model information to the initiator client corresponding to the initiator object, so that the initiator client presents in the card interaction area in the chat interface: an initiator AR screen including the first virtual character corresponding to the first card.

[0324] Among them, the card interaction area is presented based on the initiator object and triggers interactive operations for the interactive controls in the chat interface; the first virtual character is virtual content generated based on the first model information combined with AR technology.

[0325] The model information corresponds one-to-one to the virtual character. The model information includes but is not limited to the geometric structure, texture mapping, skeletal animation and other information of the corresponding virtual character. If the corresponding virtual character needs to have action and animation effects, the model information may also include key frames, skeletal hierarchy and animation sequence data. These data can be used to drive the action of the virtual character, enabling it to interact in real time.

[0326] Specifically, after the server feeds back the first model information to the initiator client of the initiator object, the initiator client renders the screen according to the first model information and presents the first virtual character corresponding to the first model information in the card interaction area.

[0327] Among them, the card interaction area also includes the initiator's environmental information collected through the camera function. The initiator's environmental information is located in the camera screen layer in the initiator's AR screen in the card interaction area. The first virtual character is located in the model layer in the initiator's AR screen in the card interaction area, and the model layer is located above the camera screen layer.

[0328] like Figure 18 As shown, it is a schematic diagram of a layered presentation provided in an embodiment of the present application. Figure 18 A model layer containing virtual objects and a camera image layer containing initiator environment information and receiver environment information are presented, wherein the model layer is located above the camera image layer.

[0329] S173: After the server determines that the target recipient object accepts the invitation request, it feeds back the first model information to the recipient client corresponding to the target recipient object, so that the recipient client shares the initiator's AR screen with the recipient object through the card interaction area.

[0330] The invitation request is sent based on the sending object triggering an interactive operation on the interactive control in the chat interface.

[0331] Specifically, after the server feeds back the first model information to the receiving client of the receiving object, the receiving client renders the picture according to the first model information and presents the first virtual character corresponding to the first model information in the model layer of the card interaction area.

[0332] For example, you can use Figure 15 Assuming that the recipient object receives the invitation request, the server feeds back the first model information to the recipient client, and the recipient client generates the first model information based on the first model information. Figure 15 The first virtual character shown in is presented in the model layer of the card interaction area.

[0333] In the above embodiment, the initiator object and the target recipient object each share their own cards and present corresponding virtual characters, thereby improving the interactivity and playability of physical cards in the online chat scene.

[0334] It should be emphasized that the acquisition of data such as the first card, invitation request, and acceptance request mentioned above has also been obtained with the permission or consent of the initiator, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0335] In addition to viewing the initiator's AR screen shared by the initiator, the target recipient can also join the card interaction and show the initiator a second card of their choice.

[0336] In the embodiment of the present application, when the server feeds back the second model information corresponding to the second card, an optional implementation method is as follows:

[0337] The receiving client sends a second card recognition request for the target receiving object input to the client. After receiving the recognition request for the second card input for the target receiving object, the server determines the second model information corresponding to the entity in the second card by performing entity recognition on the second card; then, the server feeds back the second model information to the receiving client, so that the receiving client presents the initiator's AR screen through the initiator's window in the card interaction area, and presents the receiver's AR screen of the second virtual character corresponding to the second card through the receiver's window.

[0338] The second virtual character is virtual content generated based on the second model information in combination with AR technology.

[0339] Specifically, after receiving the invitation, the target recipient object can choose to join the AR card interaction function. Then, when the target recipient places the second card that he or she wants to share with the initiator object for viewing in the corresponding camera area, the recipient client will send an identification request for the second card to the server. After receiving the identification request, the server will identify the second card. After identifying the card identification information, it will determine the entity and entity ID in the second card. Then, the server returns the CDN download address corresponding to the entity ID, obtains the second model information, and feeds it back to the recipient client.

[0340] For example, you can use Figure 8 Assume that the receiving object will Figure 6 After the second card shown in is placed in the corresponding camera area, the receiving client sends a recognition request of the receiving object for the second card to the server. After receiving the recognition request, the server recognizes the second card, and after recognizing the card identification information, determines the entity in the second card. For example, the entity ID of the entity may be 36. Then, the server returns the CDN download address with the entity identification number 36, obtains the second model information, and feeds it back to the receiving client.

[0341] In the above embodiment, the target recipient object selects the second card for identification, and the server feeds back the second model information to the recipient client based on the received identification request, so that the recipient window can present the second virtual character, thereby improving the playability of the physical card in the online chat scenario and facilitating the initiator object and the target recipient object to further explore the second card.

[0342] It should be emphasized that the acquisition of data generated by the second card, second virtual character, second model information, etc. of the above-mentioned target recipient object has also been authorized or agreed by the target recipient object, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0343] like Figure 19 As shown, it is a flow chart of sharing a first card provided by an embodiment of the present application.

[0344] S1901: triggering an interactive operation for an interactive control;

[0345] Specifically, the initiator object triggers an interactive operation on the interactive control in the chat interface of the initiator client.

[0346] S1902: switch to camera function;

[0347] S1903: Collect initiator's environment information;

[0348] S1904: Send invitation request;

[0349] Specifically, when the initiator object triggers an interactive operation on the interactive control, the server automatically sends an invitation request to the recipient client.

[0350] S1905: triggering an acceptance operation for the invitation request;

[0351] Specifically, the recipient object triggers an acceptance operation based on an acceptance control for the invitation request on the recipient client.

[0352] S1906: Stream the encoded video data.

[0353] Specifically, the initiator client encodes the initiator environment information and pushes the encoded video data to the server.

[0354] S1907: Distribute video stream;

[0355] Specifically, after the recipient object accepts the invitation, the server distributes the video stream of the initiator client to the recipient client.

[0356] S1908: Pull the stream and render the image;

[0357] Specifically, the receiving client receives and reads the video stream distributed by the server and renders the picture.

[0358] S1909: Input the first card;

[0359] Specifically, the initiator object places the first card that it wants to show to the recipient object in the corresponding camera area.

[0360] S1910: Identification request for the first card;

[0361] Specifically, the initiator client sends the initiator object's identification request for the first card to the server.

[0362] S1911: Feedback first model information;

[0363] Specifically, after receiving the identification request for the first card, the server determines the entity in the first card, and then determines the entity ID, returns the corresponding CDN download address, obtains the corresponding first model information from the CDN download address, and then sends the first model information to the initiator client.

[0364] S1912: presenting a first virtual character;

[0365] S1913: Feedback first model information;

[0366] S1914: presenting a first virtual character;

[0367] S1915: triggering behavior instructions;

[0368] Specifically, the initiator object triggers a behavior instruction for the first virtual character on the initiator client.

[0369] S1916: Transmission behavior instructions;

[0370] S1917: Transmit interaction action information;

[0371] Specifically, after receiving the behavior instruction, the server returns the CDN download address corresponding to the entity ID according to the behavior instruction and the entity ID of the entity in the first card, obtains the corresponding interactive behavior information, and transmits it to the initiator client.

[0372] S1918: Present interactive animation;

[0373] S1919: Transmitting interactive action information;

[0374] Specifically, after receiving the behavior instruction, the server returns the CDN download address corresponding to the entity ID according to the behavior instruction and the entity ID of the entity in the first card, obtains the corresponding interactive behavior information, and transmits it to the receiving client.

[0375] S1920: Present interactive animation.

[0376] Figure 19 For specific implementation in the flow chart, please refer to Figures 3A to 7 、 Figure 9A The embodiments of the present invention will not be repeated.

[0377] In addition, after feeding back the first model information to the receiving client, the server may also feed back corresponding interaction action information to the initiating client and the receiving client according to the behavior instruction triggered by the initiating object for the first virtual character.

[0378] Similarly, the behavior instructions triggered for the second virtual character are handled in the same manner as for the first virtual character, and the repeated parts are not repeated here.

[0379] In the embodiment of the present application, when the server feeds back the interaction action information, an optional implementation method is as follows:

[0380] The client sends the behavior instruction triggered by the initiator object for the first virtual character to the server. The server receives the behavior instruction triggered by the initiator object for the first virtual character and determines the interactive action information corresponding to the behavior instruction. Then, the server feeds back the interactive action information to the initiator client and the recipient client, so that the initiator client and the recipient client are both in the corresponding card interaction area and present the interactive animation corresponding to the behavior instruction based on the interactive information.

[0381] The interactive animation represents: at least controlling the first virtual character to execute a behavior corresponding to the behavior instruction.

[0382] Specifically, after receiving the behavior instruction, the server returns the CDN download address corresponding to the entity ID according to the behavior instruction and the entity ID of the entity in the first card, and obtains the corresponding interactive behavior information.

[0383] Continued use Figure 9A Assume that the initiator object triggers the action instruction "#AR card interaction turn over and take a look" for the first virtual character. The initiator client sends the action instruction to the server. The server returns the CDN download address corresponding to the entity ID based on the instruction and the entity ID of the first card, obtains the corresponding interactive behavior information, and feeds the interactive action information back to the initiator client.

[0384] Similarly, the behavior instructions triggered for the second virtual character are handled in the same manner as for the first virtual character, and the repeated parts are not repeated here.

[0385] like Figure 20 As shown, it is a flow chart of sharing a second card provided by an embodiment of the present application.

[0386] S2001: triggering a join operation for joining;

[0387] Specifically, the recipient object triggers a join operation for a join control in the chat interface of the recipient's client.

[0388] S2002: switch to camera function;

[0389] S2003: Collecting receiver environment information;

[0390] S2004: Pushing the encoded video data;

[0391] Specifically, the receiving client encodes the receiving environment information and pushes the encoded video data to the server.

[0392] S2005: Distribute video stream;

[0393] Specifically, the server distributes the video stream of the receiving client to the initiating client.

[0394] S2006: Pull the stream and render the image;

[0395] Specifically, the initiator client receives and reads the video stream distributed by the server and renders the picture.

[0396] S2007: input the second card;

[0397] Specifically, the receiving object places the second card that the receiving object wants to show to the initiating object in the corresponding camera area.

[0398] S2008: Identification request for the second card;

[0399] Specifically, the receiving client sends the receiving object's identification request for the second card to the server.

[0400] S2009: Feedback the second model information;

[0401] Specifically, after receiving the identification request for the second card, the server determines the entity in the second card, and then determines the entity ID, returns the corresponding CDN download address, obtains the corresponding second model information from the CDN download address, and then sends the first model information to the receiving client.

[0402] S2010: presenting a second virtual character;

[0403] S2011: Feedback the second model information;

[0404] S2012: presenting a second virtual character;

[0405] S2013: triggering behavioral instructions;

[0406] Specifically, the initiator object triggers a behavior instruction for the first virtual character on the initiator client.

[0407] S2014: Transmission Behavior Directive;

[0408] S2015: Transmitting interactive action information;

[0409] Specifically, after receiving the behavior instruction, the server returns the CDN download address corresponding to the entity ID according to the behavior instruction and the entity ID of the entity in the first card, obtains the corresponding interactive behavior information, and transmits it to the initiator client.

[0410] S2016: Presenting interactive animation;

[0411] S2017: Transmit interactive action information;

[0412] Specifically, after receiving the behavior instruction, the server returns the CDN download address corresponding to the entity ID according to the behavior instruction and the entity ID of the entity in the first card, obtains the corresponding interactive behavior information, and transmits it to the receiving client.

[0413] S2018: Presenting interactive animation.

[0414] Figure 20 For specific implementation in the flow chart, please refer to Figure 8 、 Figures 9B to 11C The embodiments of the present invention will not be repeated.

[0415] It should be noted that Figure 19 and Figure 20 The transmission in the video is real-time transmission, and its real-time transmission protocols include but are not limited to Real Time Messaging Protocol (RTMP), Real Time Streaming Protocol (RTSP), Web Real-Time Communications (WebRTC), etc. The video stream encoding method is not limited to video encoding standards such as H.264 and H.265. The specific implementation of the push-stream image rendering is not limited to the advanced three-dimensional (3D) graphics application programming interface (OpenGL for Embedded Systems, Embedded Systems, OpenGL ES), advanced graphics engine (Core Graphics), etc.

[0416] In the above embodiment, through the received behavioral instructions, namely the entity ID, the server feeds back the corresponding interactive action information to the initiator client and the recipient client, thereby controlling the first virtual character, or the first virtual character and the second virtual character to perform corresponding behaviors in the card interaction area, thereby improving the playability of the physical cards in the online chat scene, facilitating the initiator object and the target recipient object to further explore the card, and expanding the application scope and usage of AR technology in the online chat scene.

[0417] It should be emphasized that the acquisition of data generated by the above-mentioned behavioral instructions, interactive animations, interactive action information, etc. has also been authorized or agreed by the initiator object and the recipient object, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0418] In an embodiment of the present application, when the server obtains model information and interactive action information, it will first check whether it exists in the local cache. If not, it will return the corresponding CDN download address to load the model information or interactive action information. The loaded information resources will be cached locally to facilitate subsequent repeated calls, thereby greatly reducing the loading time of virtual characters and interactive animations in the card interaction area, and bringing a better user experience to the initiator object and the recipient object.

[0419] Furthermore, in this embodiment of the present application, the number of virtual characters that can be displayed in the card interaction area is not limited to two; any number of virtual characters can be loaded within the performance range. Furthermore, only the virtual characters displayed in the card interaction area, i.e., within the visible area of ​​the initiator and recipient objects, are rendered; rendering calculations for other virtual characters are not performed, further reducing energy consumption and improving performance and user experience. This allows for the addition of scene, background, and other models and special effects, further enhancing the expressiveness and performance of card interactions.

[0420] Furthermore, since the visible area can be detected in the embodiment of the present application and panoramic capabilities can be achieved in conjunction with the gyroscope and camera functions, when the initiator object or the recipient object moves the camera and zooms in or out on the virtual character, different angles of the virtual character can be seen. New virtual characters can also be loaded at different positions in the card interaction area according to the identification position and product requirements.

[0421] This embodiment of the application can divide the card interaction area into specified shapes based on the screen width, height, resolution, and product layout requirements of the initiator or recipient's terminal device, and fill these shapes with the video stream images of the participating objects. This allows for free play in the shape, number, and size of the card interaction areas, and they can be presented at any location in the chat interface. It can also perform operations such as partial cutting and zooming of the video stream image based on product requirements.

[0422] In an embodiment of the present application, by presenting a first virtual character generated using AR technology corresponding to the first card input by the initiating object in the card interaction area of ​​the chat interface, the presentation and display effect of the physical card in the online chat scene are enriched. Furthermore, the initiating object can share the initiating AR screen containing the first virtual character with the receiving object, so that the receiving object can also view the first virtual character in the card interaction area of ​​the receiving client. This allows AR technology to be applied in multi-person online chat scenarios, expanding the usage and application scope of AR technology.

[0423] In summary, this application uses AR technology to realize card interaction and other functions in online chat scenarios, thereby improving the interactivity and playability of physical cards in online chat scenarios, which in turn facilitates the initiator object and the recipient object to further discuss cards in the online chat scenario, thereby increasing the initiator object and the recipient object's attention to card-related games, and the initiator object and the recipient object's activity in the game.

[0424] Based on the same inventive concept, the embodiment of the present application also provides an AR-based card interaction device. Figure 21 As shown, it is a schematic diagram of the structure of an AR-based card device, which may include:

[0425] The first presentation unit 2101 is configured to present a chat interface including interactive controls, wherein the interactive controls are configured to activate AR card interaction functions.

[0426] The second presentation unit 2102 is configured to present a card interaction area in the chat interface in response to an interaction operation triggered by an initiator object on an interactive control;

[0427] The second presentation unit 2102 is further configured to present an initiator's AR screen including a first virtual character corresponding to the first card when the initiator object recognizes the first card input in the card interaction area, and share the initiator's AR screen with at least one recipient object associated with the initiator object; wherein the first virtual character is virtual content generated by combining AR technology after entity recognition of the first card.

[0428] Optionally, the second presenting unit 2102 is specifically configured to:

[0429] In response to an interactive operation triggered by the interactive control, a camera function is invoked;

[0430] A card interaction area is presented in the chat interface, and the card interaction area includes the initiator's environmental information collected by the camera function;

[0431] The second presenting unit 2102 is specifically configured to:

[0432] Through the camera function, when the first card input by the initiator object is recognized in the initiator's environmental information, the initiator's AR screen including the first virtual character is presented.

[0433] Optionally, the initiator's environmental information is located in a camera image layer in the initiator's AR image, the first virtual character is located in a model layer in the initiator's AR image, and the model layer is located above the camera image layer.

[0434] Optionally, the second presenting unit 2102 is further configured to:

[0435] Sending an invitation request triggered by an interactive control to at least one recipient object, and presenting corresponding invitation information in the chat area of ​​the chat interface;

[0436] The second presenting unit 2102 is specifically configured to:

[0437] After determining that a target recipient object among the at least one recipient object accepts the invitation, the initiator's AR screen is shared with the target recipient object.

[0438] Optionally, the second presenting unit 2102 is further configured to:

[0439] The target recipient object is determined to join this card interaction. In the card interaction area, the initiator's AR screen is presented in the initiator's window, and the recipient's AR screen corresponding to the target recipient object is presented in the recipient's window.

[0440] The receiving AR screen includes: a second virtual character corresponding to the second card input by the target receiving object; the second virtual character is virtual content generated by combining AR technology after entity recognition of the second card.

[0441] Optionally, the device further comprises:

[0442] The interactive unit 2103 is configured to present an interactive animation corresponding to the behavioral instruction in the card interactive area in response to the behavioral instruction triggered for the first virtual character, wherein the interactive animation represents: at least controlling the first virtual character to perform the behavior corresponding to the behavioral instruction.

[0443] Optionally, the behavior instruction is a behavior instruction for a single character, and the interactive animation indicates: controlling the first virtual character to perform the behavior corresponding to the behavior instruction;

[0444] A second virtual character is also presented in the card interaction area. The behavioral instructions are interactive behavioral instructions for multiple characters. The interactive animation represents: controlling the first virtual character and the second virtual character to perform interactive behaviors corresponding to the behavioral instructions; the second virtual character is a virtual content generated by combining AR technology after entity recognition of the second card input by the target recipient object.

[0445] Optionally, a second virtual character is also presented in the card interaction area, and the behavior instructions are interaction behavior instructions for multiple characters. The interaction unit 2103 is specifically configured to:

[0446] In the card interaction area, the initiator's AR image is presented through the initiator's window, and the receiver's AR image corresponding to the target receiver object is presented through the receiver's window;

[0447] Presenting interactive animations corresponding to the action instructions in the model layers of the initiator's AR screen and the recipient's AR screen;

[0448] Among them, there is no window boundary restriction on the model layer between each window.

[0449] Optionally, there are multiple first virtual characters, and the interaction unit 2103 is specifically configured to:

[0450] In response to a behavioral instruction input for a first card, a target first virtual character described by the behavioral instruction is determined from a plurality of first virtual characters, and an interactive animation corresponding to the behavioral instruction is presented in the card interaction area; wherein the interactive animation represents: at least controlling the target first virtual character to perform the behavior corresponding to the behavioral instruction; or

[0451] After a target first virtual character is selected from a plurality of first virtual characters, an interactive animation corresponding to the behavioral instruction is presented in the card interaction area in response to a behavioral instruction triggered for the target first virtual character; wherein the interactive animation represents: at least controlling the target first virtual character to perform the behavior corresponding to the behavioral instruction.

[0452] Optionally, the behavior instruction in the interaction unit 2103 is triggered by at least one of the following methods:

[0453] The behavior instruction is triggered by inputting a behavior keyword, wherein the behavior keyword is used to describe: the behavior that the first virtual character needs to perform;

[0454] Action instructions are triggered by voice input; the voice is used to describe: the action that the first virtual character needs to perform;

[0455] The camera function recognizes the input action and triggers the behavior instruction based on the action; the action is used to describe: the behavior that the first virtual character needs to perform;

[0456] At least one behavior control is presented in response to a behavior instruction triggered by selecting a target behavior control from the at least one behavior control, where the behavior control is used to describe a behavior that the first virtual character needs to perform.

[0457] Optionally, the device further includes an adjustment unit 2104 for performing a screen zooming operation corresponding to the adjustment gesture on the first virtual character in response to an adjustment gesture triggered for the first virtual character, and presenting the first virtual character after the screen zooming operation is performed in the card interaction area.

[0458] Based on the same inventive concept, the embodiment of the present application also provides an AR-based card interaction device. Figure 22 As shown, it is a schematic diagram of the structure of an AR-based card device, which may include:

[0459] The third presentation unit 2201 is used to present a chat interface;

[0460] Inviting unit 2202, configured to present a card interaction area in the chat interface in response to an acceptance operation of the invitation request triggered by the receiving object to the initiating object, wherein the card interaction area includes an initiating AR image including a first virtual character obtained after entity recognition with a first card input by the initiating object;

[0461] Among them, the first virtual character is a virtual content generated by combining AR technology after entity recognition of the first card; the invitation request is sent by the initiator object based on the interactive control in the chat interface.

[0462] Optionally, the invitation unit 2202 is specifically configured to:

[0463] Upon receiving the invitation request triggered by the initiator object, the corresponding invitation information sent by the initiator object and the acceptance control are presented in the chat area of ​​the chat interface;

[0464] In response to the recipient object's acceptance operation on the invitation request triggered by the acceptance control, a card interaction area is presented in the chat interface.

[0465] Optionally, the invitation unit 2202 is further configured to:

[0466] In response to a join operation triggered by a join control, join the current card interaction and invoke the camera function;

[0467] When the second card input by the receiving object is recognized through the camera function, the initiator's AR screen is presented through the initiator's window in the card interaction area, and the receiving party's AR screen containing the second virtual character corresponding to the second card is presented through the receiving party's window; wherein, the second virtual character is virtual content generated by combining AR technology after the second card is physically recognized.

[0468] Based on the same inventive concept, the embodiment of the present application also provides another AR-based card interaction device. Figure 23 As shown, it is a schematic diagram of the structure of an AR-based card device, which may include:

[0469] The receiving unit 2301 is configured to, upon receiving a recognition request for a first card input by an initiator object, determine first model information corresponding to an entity in the first card by performing entity recognition on the first card;

[0470] The first feedback unit 2302 is configured to feed back the first model information to the initiator client corresponding to the initiator object, so that the initiator client presents, in the card interaction area of ​​the chat interface: an initiator AR screen including a first virtual character corresponding to the first card; the card interaction area is presented based on the initiator object and triggers an interactive operation in response to an interactive control in the chat interface; the first virtual character is virtual content generated based on the first model information in combination with AR technology; and

[0471] The first feedback unit 2302 is used to determine that the target recipient object accepts the invitation request, and then feed back the first model information to the recipient client corresponding to the target recipient object, so that the recipient client can share the initiator's AR screen to the recipient object through the card interaction area; the invitation request is sent based on the sending object triggering an interactive operation on the interactive control in the chat interface.

[0472] Optionally, the device further comprises:

[0473] The second feedback unit 2303 is configured to, upon receiving a recognition request for a second card inputted by a target recipient object, determine second model information corresponding to an entity in the second card by performing entity recognition on the second card;

[0474] The second model information is fed back to the receiving client so that the receiving client presents the initiator's AR screen through the initiator's window in the card interaction area, and presents the receiving client's AR screen of the second virtual character corresponding to the second card through the receiving window; the second virtual character is virtual content generated based on the second model information in combination with AR technology.

[0475] Optionally, the second card is input by the target recipient object after joining the current card interaction, and the second feedback unit 2303 is further configured to:

[0476] The second model information is fed back to the initiator client, so that the initiator client presents the initiator AR picture through the initiator window in the card interaction area, and presents the recipient AR picture through the recipient window.

[0477] Optionally, the device further comprises:

[0478] The third feedback unit 2304 is configured to receive a behavior instruction triggered by the initiator object for the first virtual character after the first feedback unit feeds back the first model information to the receiving client, and determine interactive action information corresponding to the behavior instruction;

[0479] Feedback the interactive action information to the initiator client and the recipient client, so that both the initiator client and the recipient client are in the corresponding card interaction area, and the interactive animation corresponding to the action instruction is presented based on the interactive information;

[0480] The interactive animation represents: at least controlling the first virtual character to execute a behavior corresponding to the behavior instruction.

[0481] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0482] After introducing the AR-based card interaction method and device according to an exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0483] Those skilled in the art will appreciate that various aspects of the present application can be implemented as systems, methods, or program products. Therefore, various aspects of the present application can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0484] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application. In one embodiment, the electronic device may be a server, such as Figure 1 In this embodiment, the structure of the electronic device can be as follows: Figure 24 As shown, it includes a memory 2401, a communication module 2403 and one or more processors 2402.

[0485] Memory 2401 is used to store computer programs executed by processor 2402. Memory 2401 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and programs required for running instant messaging functions, while the data storage area may store various instant messaging messages and operating instruction sets.

[0486] Memory 2401 may be a volatile memory, such as random-access memory (RAM); a non-volatile memory, such as read-only memory, flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or any other medium capable of carrying or storing a desired computer program in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 2401 may be a combination of the aforementioned memories.

[0487] The processor 2402 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 2402 is configured to implement the aforementioned AR-based card interaction method when calling the computer program stored in the memory 2401 .

[0488] The communication module 2403 is used to communicate with terminal devices and other servers.

[0489] The specific connection medium between the memory 2401, the communication module 2403 and the processor 2402 is not limited in the embodiment of the present application. Figure 24 In the embodiment, the memory 2401 and the processor 2402 are connected via a bus 2404. Figure 24 The connections between the other components are shown in bold lines for illustration only and are not intended to be limiting. The bus 2404 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Figure 24 The diagram shows a single thick line, but this does not indicate that there is only one bus or one type of bus.

[0490] The memory 2401 stores a computer storage medium, which stores computer executable instructions. The computer executable instructions are used to implement the AR-based card interaction method of the embodiment of the present application. The processor 2402 is used to execute the above-mentioned AR-based card interaction method, such as Figure 2 shown.

[0491] In another embodiment, the electronic device may also be other electronic devices, such as Figure 1 The terminal device 110 shown in FIG. In this embodiment, the structure of the electronic device can be as follows: Figure 25 As shown, it includes: a communication component 2510, a memory 2520, a display unit 2530, a camera 2540, a sensor 2550, an audio circuit 2560, a Bluetooth module 2570, a processor 2580 and other components.

[0492] The communication component 2510 is used to communicate with the server. In some embodiments, it may include a wireless fidelity (WiFi) module. The WiFi module is a short-range wireless transmission technology. Electronic devices can help users send and receive information through the WiFi module.

[0493] The memory 2520 can be used to store software programs and data. The processor 2580 executes various functions and data processing of the terminal device 110 by running the software programs or data stored in the memory 2520. The memory 2520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 2520 stores an operating system that enables the terminal device 110 to run. In the present application, the memory 2520 can store an operating system and various application programs, and can also store a computer program that executes the AR-based card interaction method of the embodiment of the present application.

[0494] The display unit 2530 can also be used to display information input by the user or provided to the user, as well as a graphical user interface (GUI) of various menus of the terminal device 110. Specifically, the display unit 2530 may include a display screen 2532 disposed on the front of the terminal device 110. The display screen 2532 may be configured in the form of a liquid crystal display, a light-emitting diode, etc. The display unit 2530 can be used to display the chat interface in the embodiments of the present application, etc.

[0495] The display unit 2530 can also be used to receive input digital or character information and generate signal input related to the user settings and function control of the terminal device 110. Specifically, the display unit 2530 may include a touch screen 2531 arranged on the front of the terminal device 110, which can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc.

[0496] The touch screen 2531 can be covered on the display screen 2532, or the touch screen 2531 and the display screen 2532 can be integrated to realize the input and output functions of the terminal device 110. The integrated display screen can be simply called a touch screen. In this application, the display unit 2530 can display applications and corresponding operation steps.

[0497] Camera 2540 can be used to capture still images, and users can publish images captured by camera 2540 through an application. Camera 2540 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to processor 2580 for conversion into a digital image signal.

[0498] The terminal device may further include at least one sensor 2550, such as an acceleration sensor 2551, a distance sensor 2552, a fingerprint sensor 2553, and a temperature sensor 2554. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0499] The audio circuit 2560, speaker 2561, and microphone 2562 provide an audio interface between the user and the terminal device 110. The audio circuit 2560 can convert the received audio data into an electrical signal and transmit it to the speaker 2561, which converts it into a sound signal for output. The terminal device 110 may also be equipped with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 2562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 2560 and converted into audio data. The audio data is then output to the communication component 2510 for transmission to, for example, another terminal device 110, or the audio data is output to the memory 2520 for further processing.

[0500] The Bluetooth module 2570 is used to exchange information with other Bluetooth devices having a Bluetooth module through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 2570 to exchange data.

[0501] The processor 2580 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. By running or executing software programs stored in the memory 2520 and calling data stored in the memory 2520, it performs various functions of the terminal device and processes data. In some embodiments, the processor 2580 may include one or more processing units; the processor 2580 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 2580. In this application, the processor 2580 can run the operating system, application programs, user interface display and touch response, as well as the AR-based card interaction method of the embodiment of the application. In addition, the processor 2580 is coupled to the display unit 2530.

[0502] In some possible implementations, various aspects of the AR-based card interaction method provided in this application may also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the AR-based card interaction method according to various exemplary embodiments of the present application described above in this specification. For example, the electronic device may execute the following steps: Figure 2 or Figure 13 Follow the steps shown in .

[0503] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0504] The program product of the embodiment of the present application may be a portable compact disc read-only memory (CD-ROM) and include a computer program, and can be run on an electronic device. However, the program product of the present application is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with a command execution system, apparatus, or device.

[0505] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a readable computer program. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with a command execution system, apparatus, or device.

[0506] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0507] The computer program for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer program can be executed entirely on the user electronic device, partially on the user electronic device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the user electronic device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external electronic device (for example, using an Internet service provider to connect through the Internet).

[0508] It should be noted that although several units or subunits of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the application, the features and functions of two or more units described above can be embodied in a single unit. Conversely, the features and functions of a single unit described above can be further divided and embodied by multiple units.

[0509] Furthermore, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0510] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain a computer-usable computer program.

[0511] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program commands. These computer program commands can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0512] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising a command device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0513] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0514] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0515] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A card interaction method based on augmented reality (AR), characterized in that: The method comprises: Presenting a chat interface including interactive controls; the interactive controls are used to invoke AR card interactive functions; In response to an interactive operation triggered by an initiator object on the interactive control, presenting a card interactive area in the chat interface; When the first card input by the initiator object is recognized in the card interaction area, an initiator AR screen including a first virtual character corresponding to the first card is presented, and the initiator AR screen is shared with at least one recipient object associated with the initiator object; wherein, the first virtual character is virtual content generated by combining AR technology after entity recognition of the first card.

2. The method according to claim 1, wherein The presenting of a card interaction area in the chat interface in response to the interactive operation triggered by the interactive control includes: In response to an interactive operation triggered by the interactive control, invoking a camera function; Presenting a card interaction area in the chat interface, wherein the card interaction area includes the initiator's environmental information collected by the camera function; When the first card input by the initiator object is recognized in the card interaction area, presenting an initiator AR screen including a first virtual character corresponding to the first card includes: By using the camera function, when the first card input by the initiator object is recognized in the initiator environment information, an initiator AR screen including the first virtual character is presented.

3. The method according to claim 2, wherein The initiator environment information is located in a camera image layer in the initiator AR image, the first virtual character is located in a model layer in the initiator AR image, and the model layer is located above the camera image layer.

4. The method according to claim 1, wherein The presenting of a card interaction area in the chat interface in response to the interactive operation triggered by the initiator object on the interactive control further includes: Sending an invitation request triggered by the interactive control to at least one recipient object, and presenting corresponding invitation information in the chat area of ​​the chat interface; The step of sharing the initiator AR screen with at least one recipient object associated with the initiator object includes: After determining that a target recipient object among the at least one recipient object accepts the invitation, the initiator AR screen is shared with the target recipient object.

5. The method according to claim 4, wherein The method further comprises: Determine that the target recipient object joins this card interaction, present the initiator's AR image in the initiator window in the card interaction area, and present the recipient's AR image corresponding to the target recipient object in the recipient window; The receiving AR screen includes: a second virtual character corresponding to the second card input by the target receiving object; the second virtual character is virtual content generated by combining AR technology after entity recognition of the second card.

6. The method according to any one of claims 1 to 5, wherein: The method further comprises: In response to a behavior instruction triggered for the first virtual character, an interactive animation corresponding to the behavior instruction is presented in the card interaction area, and the interactive animation represents: at least controlling the first virtual character to perform the behavior corresponding to the behavior instruction.

7. The method according to claim 6, wherein The behavioral instruction is a behavioral instruction for a single character, and the interactive animation represents: controlling the first virtual character to perform the behavior corresponding to the behavioral instruction; A second virtual character is also presented in the card interaction area, and the behavior instructions are interactive behavior instructions for multiple characters. The interactive animation representation: controlling the first virtual character and the second virtual character to perform the interactive behavior corresponding to the behavior instructions; the second virtual character is a virtual content generated by combining AR technology after entity recognition of the second card input by the target recipient object.

8. The method according to claim 7, wherein A second virtual character is also presented in the card interaction area. The behavior instruction is an interactive behavior instruction for multiple characters. Then, an interactive animation corresponding to the behavior instruction is presented in the card interaction area, including: In the card interaction area, the initiator's AR screen is presented through the initiator's window, and the receiver's AR screen corresponding to the target receiver object is presented through the receiver's window; Presenting an interactive animation corresponding to the action instruction in the model layers of the initiator's AR screen and the recipient's AR screen; There is no window boundary restriction on the model layer between each window.

9. The method according to claim 6, wherein There are multiple first virtual characters, and in response to the behavior instruction triggered for the first virtual character, presenting the interactive animation corresponding to the behavior instruction in the card interaction area includes: In response to a behavioral instruction input for the first card, a target first virtual character described by the behavioral instruction is determined from a plurality of first virtual characters, and an interactive animation corresponding to the behavioral instruction is presented in the card interaction area; wherein the interactive animation represents: at least controlling the target first virtual character to perform the behavior corresponding to the behavioral instruction; or After a target first virtual character is selected from a plurality of first virtual characters, in response to a behavior instruction triggered for the target first virtual character, an interactive animation corresponding to the behavior instruction is presented in the card interaction area; wherein, the interactive animation represents: at least controlling the target first virtual character to perform the behavior corresponding to the behavior instruction.

10. The method according to claim 6, wherein The action instruction is triggered by at least one of the following methods: The behavior instruction is triggered by inputting a behavior keyword; wherein the behavior keyword is used to describe: the behavior that the first virtual character needs to perform; The behavior instruction is triggered by inputting a voice; the voice is used to describe: the behavior that the first virtual character needs to perform; Recognizing an input action through a camera function, and triggering the behavior instruction based on the action; the action is used to describe: the behavior that the first virtual character needs to perform; At least one behavior control is presented in response to a behavior instruction triggered by selecting a target behavior control from the at least one behavior control, wherein the behavior control is used to describe a behavior that the first virtual character needs to perform.

11. The method according to any one of claims 1 to 5, wherein: The method further comprises: In response to an adjustment gesture triggered for the first virtual character, a screen zooming operation corresponding to the adjustment gesture is performed on the first virtual character, and the first virtual character after the screen zooming operation is performed is presented in the card interaction area.

12. A card interaction method based on AR, characterized in that: The method comprises: Presenting the chat interface; In response to an acceptance operation of the invitation request triggered by the initiator object by the recipient object, a card interaction area is presented in the chat interface, wherein the card interaction area includes an initiator AR screen including a first virtual character obtained after entity recognition of a first card input by the initiator object; Among them, the first virtual character is a virtual content generated by combining AR technology after entity recognition of the first card; the invitation request is sent by the initiator object based on the triggering of the interactive control in the chat interface.

13. The method according to claim 12, wherein: The presenting of a card interaction area in the chat interface in response to an acceptance operation of the invitation request triggered by the initiator object by the recipient object includes: Receiving the invitation request triggered by the initiator object, presenting the corresponding invitation information sent by the initiator object and an acceptance control in the chat area of ​​the chat interface; In response to the acceptance operation of the invitation request triggered by the recipient object based on the acceptance control, a card interaction area is presented in the chat interface.

14. The method according to claim 13, wherein The chat interface further includes a join control, and the method further includes: In response to a join operation triggered by the join control, joining the current card interaction and invoking a camera function; When the second card input by the receiving object is recognized through the camera function, the initiator's AR screen is presented through the initiator's window in the card interaction area, and the receiving party's AR screen including the second virtual character corresponding to the second card is presented through the receiving party's window; wherein, the second virtual character is virtual content generated after the second card is physically recognized in combination with AR technology.

15. A card interaction method based on AR, characterized in that: The method comprises: After receiving a recognition request for a first card input by an initiator object, determining first model information corresponding to an entity in the first card by performing entity recognition on the first card; Feeding back the first model information to an initiator client corresponding to the initiator object, so that the initiator client presents, in a card interaction area in a chat interface: an initiator AR screen including a first virtual character corresponding to the first card; the card interaction area is presented based on the initiator object and triggers an interactive operation in response to an interactive control in the chat interface; the first virtual character is virtual content generated based on the first model information in combination with AR technology; and After determining that the target recipient object accepts the invitation request, the first model information is fed back to the recipient client corresponding to the target recipient object, so that the recipient client shares the initiator's AR screen to the recipient object through the card interaction area; the invitation request is sent based on the sending object triggering an interactive operation on the interactive control in the chat interface.

16. The method according to claim 15, wherein The method further comprises: After receiving a recognition request for a second card inputted by the target recipient object, determining second model information corresponding to an entity in the second card by performing entity recognition on the second card; The second model information is fed back to the receiving client so that the receiving client presents the initiator's AR screen through the initiator's window in the card interaction area, and presents the receiving AR screen of the second virtual character corresponding to the second card through the receiving window; the second virtual character is virtual content generated based on the second model information in combination with AR technology.

17. The method according to claim 16, wherein The second card is input by the target recipient object after joining the current card interaction; the method further includes: The second model information is fed back to the initiator client, so that the initiator client presents the initiator AR screen through the initiator window in the card interaction area, and presents the recipient AR screen through the recipient window.

18. The method according to any one of claims 15 to 17, wherein: After feeding back the first model information to the receiving client, the method further includes: receiving a behavior instruction triggered by an initiator object for the first virtual character, and determining interactive action information corresponding to the behavior instruction; Feedback the interactive action information to the initiator client and the recipient client, so that both the initiator client and the recipient client are in the corresponding card interaction area and present the interactive animation corresponding to the action instruction based on the interactive information; The interactive animation indicates at least controlling the first virtual character to execute the behavior corresponding to the behavior instruction.

19. An AR-based card interactive device, characterized in that: include: A first presentation unit, configured to present a chat interface including interactive controls; The interactive control is used to invoke the AR card interactive function; A second presentation unit, configured to present a card interaction area in the chat interface in response to an interaction operation triggered by an initiator object on the interaction control; The second presentation unit is further configured to present an initiator AR screen including a first virtual character corresponding to the first card when the first card input by the initiator object is recognized in the card interaction area, and share the initiator AR screen with at least one recipient object associated with the initiator object; wherein the first virtual character is virtual content generated by combining AR technology after entity recognition of the first card.

20. An AR-based card interactive device, characterized in that: include: A third presentation unit, configured to present a chat interface; an invitation unit, configured to, in response to an acceptance operation of an invitation request triggered by a recipient object to an initiator object, present a card interaction area in the chat interface, wherein the card interaction area includes an initiator AR screen including a first virtual character obtained after entity recognition of a first card input by the initiator object; Among them, the first virtual character is a virtual content generated by combining AR technology after entity recognition of the first card; the invitation request is sent by the initiator object based on the triggering of the interactive control in the chat interface.

21. An AR-based card interactive device, characterized in that: include: a receiving unit configured to, upon receiving an identification request for a first card input by an initiator object, determine first model information corresponding to an entity in the first card by performing entity identification on the first card; a first feedback unit, configured to feed back the first model information to an initiator client corresponding to the initiator object, so that the initiator client presents, in a card interaction area in a chat interface, an initiator AR image including a first virtual character corresponding to the first card; The card interaction area is presented based on the initiator object, triggering an interactive operation for the interactive control in the chat interface; The first virtual character is virtual content generated based on the first model information in combination with AR technology; as well as After determining that the target recipient object accepts the invitation request, the first model information is fed back to the recipient client corresponding to the target recipient object, so that the recipient client shares the initiator's AR screen to the recipient object through the card interaction area; the invitation request is sent based on the sending object triggering an interactive operation on the interactive control in the chat interface.

22. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor is enabled to perform the steps of any one of the methods of claims 1 to 18.

23. A computer-readable storage medium, characterized in that The method comprises a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any one of the methods of claims 1 to 18.

24. A computer program product, characterized in that The method comprises a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of any one of the methods described in claims 1 to 18.

Images