Information interaction display method and system
By setting up the synchronous display and highlighting of two screens in digital human technology, combining user input and explanation text generated by the knowledge base, the problem of rigid content broadcast in digital human technology is solved, the user experience and interactivity are improved, and the user experience and interactivity are realized Flexibility and immersion in information interaction.
Patent Information
- Application Number
- CN202410947163.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-29
AI Technical Summary
The existing digital human technology shows rigidity and poor flexibility in content broadcasting, resulting in poor user experience and urgently needs to improve the immersion and interactivity of information interaction.
By setting two screens in the system, the first screen displays the image of the target object, and the second screen plays the digital person's video. The digital person performs corresponding actions when broadcasting the anchor text, and highlights the anchor text on the first screen The corresponding target object area combines the information input by the user and the knowledge base to generate explanation text, and uses a large language model and a preset keyword database to synchronize actions and highlights.
Improve the flexibility and fun of information interaction, users can quickly locate relevant information, improve user experience and interactivity, ensure the smoothness and consistency of interaction, and provide intuitive, flexible and highly personalized multi-modal users. Interactive experience.
Smart Images

Figure CN119045930A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of Internet application technology, and in particular to a method and system for interactive information display. Background Art
[0002] Screen-based interaction is a common digital interaction technology that uses high-resolution large-size displays as interactive interfaces to provide users with immersive visual experiences and intuitive operations. Digital human technology combines computer graphics, artificial intelligence, motion capture, and speech synthesis to create lifelike digital human images that are not only close to real people in appearance, but can also simulate natural human movements and speech. They are widely used in movies, games, advertising, social media, education, and other fields.
[0003] Currently, digital human technology is only used to achieve content broadcasting, which is rigid and inflexible, and there is an urgent need to improve user experience. Summary of the invention
[0004] The present application provides an information interactive display method for realizing synchronization between a digital human broadcast video and a target object.
[0005] This application provides the following solutions:
[0006] According to a first aspect, an information interactive display method is provided, which is applied to a system including at least a first screen and a second screen, wherein the first screen and the second screen are both arranged toward a user area, and the angle between the first screen and the second screen is within a preset angle range; the method comprises: obtaining a text to be explained for a target object, wherein the text to be explained comprises at least one anchor point text; generating a digital human broadcast video using the text to be explained; providing an image of the target object to the first screen to display the image of the target object on the first screen, and providing the digital human broadcast video to the second screen to play the digital human broadcast video on the second screen, wherein when the digital human broadcasts the anchor point text in the digital human broadcast video, an action corresponding to the anchor point text is executed and highlighted on the first screen.
[0007] According to an achievable manner in an embodiment of the present application, the method further includes: acquiring an image of the target object based on information of the target object input by a user; and providing the image of the target object to the first screen to display the image of the target object on the first screen.
[0008] According to an achievable method in an embodiment of the present application, obtaining the text to be explained for the target object includes: obtaining the question text input by the user for the target object or the question text corresponding to the input question voice; using the knowledge base of the target object to generate a reply text for the question text, and using the reply text as the text to be explained.
[0009] According to an achievable method in an embodiment of the present application, using the knowledge base of the target object to generate a reply text for the question text includes: using the knowledge base information of the target object and the question text to generate a prompt instruction Prompt; providing the Prompt to a large language model to obtain the reply text generated by the large language model for the question text.
[0010] According to an achievable manner in an embodiment of the present application, the method further includes: using a pre-set keyword library to determine the content in the text to be explained that matches the keywords in the keyword library as the anchor text, and the keywords in the keyword library correspond to digital human actions and target object areas; using the text to be explained to generate a digital human broadcast video includes: using the text to be explained and the digital human actions corresponding to the anchor text to generate the digital human broadcast video; when the digital human broadcasts the anchor text, sending a highlight instruction to the first screen, the highlight instruction carrying the target object area information corresponding to the anchor text, so as to highlight the partial area of the target object corresponding to the target object area information.
[0011] According to an achievable method in an embodiment of the present application, the action corresponding to the anchor text includes: using a gesture or a tool to point to the partial area highlighted on the first screen.
[0012] According to an achievable method in an embodiment of the present application, the image of the target object is pre-divided into N grids corresponding to the first screen, where N is a positive integer greater than 1; highlighting the partial area of the target object corresponding to the anchor text on the first screen includes: based on a pre-set correspondence between the anchor text and at least one of the N grids, highlighting the image area of the grid corresponding to the anchor text on the first screen.
[0013] According to an achievable method in an embodiment of the present application, highlighting the partial area of the target object corresponding to the anchor text on the first screen includes: highlighting the partial area of the target object corresponding to the anchor text; or, identifying the partial area of the target object corresponding to the anchor text with a preset element; or, displaying the partial area of the target object corresponding to the anchor text with a preset color, wherein the preset color is different from the color used for other areas except the partial area; or, enlarging and displaying the partial area of the target object corresponding to the anchor text; or, displaying the partial area of the target object corresponding to the anchor text in the center on the first screen.
[0014] According to an achievable method in the embodiment of the present application, the target object includes: cultural relics, ancient books or murals.
[0015] According to a second aspect, an information interaction display system is provided, the system comprising: a service providing device, a first screen and a second screen, the first screen and the second screen are both arranged toward a user area, and the angle between the first screen and the second screen is within a preset angle range; the service providing device is used to obtain a text to be explained for a target object, the text to be explained comprising at least one anchor point text; a digital human broadcast video is generated using the text to be explained and provided to the second screen; an image of the target object is provided to the first screen; the first screen is used to display the image of the target object; the second screen is used to play the digital human broadcast video; wherein, when the digital human broadcasts the anchor point text in the digital human broadcast video, an action corresponding to the anchor point text is executed and a partial area of the target object corresponding to the anchor point text is highlighted on the first screen.
[0016] According to an achievable method in an embodiment of the present application, the system also includes: a first user input device, used to provide a user interface to the user, and provide the target object information input by the user through the user interface to the service providing device; the service providing device is also used to obtain an image of the target object based on the target object information input by the user.
[0017] According to an achievable method in an embodiment of the present application, the system also includes: a second user input device, used to obtain a question text or a question voice input by a user for the target object, and provide the question text or the question voice to the service providing device; the service providing device is used to generate a reply text for the question text by using the knowledge base of the target object, or to call a speech recognition system to convert the question voice into a question text, and generate a reply text for the question text by using the knowledge base of the target object; and use the generated reply text as the text to be explained.
[0018] According to an achievable method in the embodiment of the present application, the service providing device is further used to use a pre-set keyword library to determine the content in the text to be explained that matches the keywords in the keyword library as anchor text, and the keywords in the keyword library correspond to digital human actions and target object areas; when the digital human broadcasts the anchor text, a highlight instruction is sent to the first screen, and the highlight instruction carries the target object area information corresponding to the anchor text, so as to highlight the partial area of the target object corresponding to the target object area information.
[0019] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.
[0020] According to a fourth aspect, there is provided an electronic device, comprising:
[0021] one or more processors; and
[0022] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the first aspects above.
[0023] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0024] 1) This application sets up a first screen for displaying the target object and a second screen for broadcasting the video of the digital human. Through the anchor text, the corresponding action of the digital human in the second screen corresponds to the partial area of the target object displayed in the first screen. While realizing the synchronous playback of the target object and the video broadcast by the digital human, the highlighted content of the target object is combined with the action of the digital human, which further improves the interactivity of the information. The information display method is more flexible and interesting. Users can quickly locate relevant information according to the actions of the digital human and the highlighted content, which greatly improves the user experience.
[0025] 2) This application obtains the image of the target object by the user inputting the target object information, so that the user can display the target object according to his or her needs, thereby improving the interactivity between the information interaction system and the user.
[0026] 3) This application obtains the question text input by the user for the target object or the question text corresponding to the input question voice, and uses the question text and the answer text generated by the knowledge base of the target object as the explanation text, and provides a method for generating the explanation text, so that the digital human can explain the target object with the user in a question-and-answer manner, which is more interactive and the explanation content is more accurate and targeted.
[0027] 4) This application uses knowledge base information and question text to generate prompt instructions for a large language model, and the large language model generates a reply text based on the prompt instructions, further improving the accuracy of reply text generation.
[0028] 5) This application pre-sets a keyword library, and the keywords correspond to the digital human's actions and target object areas. The content in the text to be explained that matches the keywords in the keyword library is used as the anchor text. The digital human's actions corresponding to the anchor text are used to generate the digital human's broadcast video, and when the digital human broadcasts the anchor text, a highlighting instruction is sent to the first screen, and the highlighting instruction carries the target object area information corresponding to the anchor text. The coordination between the digital human's actions and the highlighted target object area is achieved, ensuring the smoothness and consistency of the interaction, so that users can experience a seamless interactive experience.
[0029] 6) The present application divides the first screen into N grids, and based on the correspondence between the pre-set anchor text and at least one of the N grids, highlights the image area of the grid corresponding to the anchor text on the first screen, and provides a highlighting method to achieve precise synchronous response of the digital human and these areas, thereby providing an intuitive, flexible and highly personalized multimodal user interaction experience.
[0030] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0032] Figure 1 A system architecture diagram applicable to the embodiments of the present application.
[0033] Figure 2 This is a system architecture diagram applicable to another embodiment of the present application.
[0034] Figure 3 A flowchart of an information interaction method provided in an embodiment of the present application.
[0035] Figure 4 A schematic diagram of a first screen divided into nine grids to highlight a target area provided in an embodiment of the present application.
[0036] Figure 5 A schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0038] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.
[0039] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0040] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.
[0041] There are already some technologies that use digital humans to explain information on large screens. Among these technologies, there are still some defects in the performance of digital humans, such as the possible delay and inaccuracy of motion capture, the limitations of speech synthesis in emotional expression, and the inability to only play preset video content and interact with users in real time. These technical defects sometimes make the presentation of digital humans unnatural and affect the overall user experience.
[0042] In view of this, the present application provides a new idea. In order to facilitate the understanding of the present application, the system architecture on which the present application is based is first described. Figure 1An exemplary system architecture to which the embodiments of the present application can be applied is shown. Figure 1 As shown in , the system architecture may include: a service providing device, a first screen and a second screen.
[0043] Among them, the first screen is used to display the image of the target object. The second screen is used to play the digital human broadcast video. The first screen and the second screen are both set towards the user area, and the angle between the first screen and the second screen is within a preset angle range. The user area is the area where the user stays, and the user can easily view the contents of the first screen and the second screen in the user area. The preset angle range of the angle between the first screen and the second screen can be determined based on factors such as the size of the first screen and the second screen, the size of the area where the two screens are located, and the size of the user area. The setting of the preset angle range can meet the interaction needs between the two screens and make the user have a strong sense of reality when watching.
[0044] In the present application, the target object is the object to be displayed by the information interaction display system. The target object can be input by the user or preset by the technician according to the demand. The present application can be applied to a variety of fields, and the target object can be the object to be displayed in the field. In particular, the method of the present application can be used in museums, and the target object can include: cultural relics, ancient books or murals. According to the information of the target object input by the user, an image of the target object is obtained; the image of the target object is provided to the first screen to display the image of the target object on the first screen.
[0045] The service providing device is used to obtain the text to be explained for the target object, the text to be explained includes at least one anchor text; generate a digital human broadcast video using the text to be explained and provide it to the second screen; and provide an image of the target object to the first screen. When the digital human broadcasts the anchor text in the digital human broadcast video, the action corresponding to the anchor text is performed and a partial area of the target object corresponding to the anchor text is highlighted on the first screen.
[0046] The service provider device can be set up as an independent server, or in a server group, or in a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system to solve the defects of difficult management and weak service scalability in traditional physical hosts and virtual private servers (VPs) services. The service provider device can also be set up as a computer terminal with strong computing power.
[0047] The system architecture of the embodiment of the present application may also include a first user input device. The first user input device is used to provide a user interface to the user, and provide the information of the target object input by the user through the user interface to the service providing device; the first user device may be a tablet, PC (Personal Computer) and other devices. For example, a tablet device is set up in a museum, and the user can input the information of the target object through the tablet device. It can also be a user's mobile phone or a wearable device. For example, the user uses a mobile phone to scan a code to connect to the service and inputs the information of the target object through the user interface provided by the service. The service providing device can obtain an image of the target object based on the information of the target object input by the user.
[0048] The system architecture of the embodiment of the present application may also include a second user input device ( Figure 1 The second user input device is used to obtain the question text or question voice input by the user for the target object, and provide the question text or question voice to the service providing device. The second user input device can be the same device as the first user input device, or a device with voice input function such as a microphone can be used as the second user input device.
[0049] Figure 2 This is a system architecture diagram applicable to another embodiment of the present application, such as Figure 2 As shown, the tablet in the figure is the first user input device, and the microphone is the second user input device. The first user input device and the second user input device respectively provide the acquired inputs to the service providing device for processing.
[0050] At this time, the service provider uses the knowledge base of the target object to generate a reply text for the question text, or calls a speech recognition system to convert the question speech into question text, and uses the knowledge base of the target object to generate a reply text for the question text; the generated reply text is used as the text to be explained.
[0051] As one of the feasible ways, using the knowledge base of the target object, generating a reply text for the question text can be as follows: generating a Prompt (prompt instruction) using the knowledge base information of the target object and the question text; providing the Prompt to a large language model, and obtaining the reply text generated by the large language model for the question text.
[0052] This implementation method utilizes the powerful natural language understanding and text generation capabilities of the large language model. In addition to this method, you can also use an ordinary text generation model to generate reply text, or you can use the target object's knowledge base information and question text to perform knowledge retrieval, and then use the retrieval results, knowledge base information, and question text to generate prompts, and provide the prompts to the ordinary reply model to generate reply text.
[0053] The service providing device is also used to use a pre-set keyword library to determine the content in the text to be explained that matches the keywords in the keyword library as the anchor text, and the keywords in the keyword library correspond to the digital human actions and target object areas; when the digital human broadcasts the anchor text, a highlighting instruction is sent to the first screen, and the highlighting instruction carries the target object area information corresponding to the anchor text, so as to highlight the partial area of the target object corresponding to the target object area information.
[0054] The image of the target object corresponds to the first screen being pre-divided into N grids, where N is a positive integer greater than 1; highlighting the partial area of the target object corresponding to the anchor text on the first screen includes: based on the correspondence between the pre-set anchor text and at least one grid in the N grids, highlighting the image area of the grid corresponding to the anchor text on the first screen.
[0055] Highlighting a partial area of the target object corresponding to the anchor text on the first screen includes: highlighting the partial area of the target object corresponding to the anchor text; or identifying the partial area of the target object corresponding to the anchor text with a preset element; or displaying the partial area of the target object corresponding to the anchor text in a preset color, wherein the preset color is different from the color used for other areas except the partial area; or displaying the partial area of the target object corresponding to the anchor text in an enlarged manner; or displaying the partial area of the target object corresponding to the anchor text in the center on the first screen.
[0056] Figure 3 The information interactive display method provided in the embodiment of the present application is a flow chart. The method can be performed by Figure 1 The service provider in the system shown is executed. Figure 3 As shown in , the method may include the following steps:
[0057] Step 301: Obtain a text to be explained for a target object, where the text to be explained includes at least one anchor text.
[0058] Step 302, generating a digital human broadcast video using the text to be explained.
[0059] Step 303: providing the image of the target object to the first screen so as to display the image of the target object on the first screen, and providing the digital human broadcasting video to the second screen so as to play the digital human broadcasting video on the second screen, wherein when the digital human broadcasts the anchor text in the digital human broadcasting video, the action corresponding to the anchor text is performed and the partial area of the target object corresponding to the anchor text is highlighted on the first screen.
[0060] It can be seen from the above process that the present application sets up a first screen for displaying the target object and a second screen for broadcasting the video broadcast by the digital human. The anchor text is used to make the corresponding action of the digital human in the second screen correspond to the partial area of the target object displayed in the first screen. While realizing the synchronous playback of the target object and the video broadcast by the digital human, the highlighted content of the target object is combined with the action of the digital human, further enhancing the interactivity and fun of the information.
[0061] The following is a detailed description of each step in the above process and the effects that can be further produced in conjunction with the embodiments. It should be noted that the limitations such as "first" and "second" involved in the present disclosure do not have limitations in terms of size, order and quantity, and are only used to distinguish them in name, for example, "first screen" and "second screen" are used to distinguish two screens.
[0062] First, the above step 301, namely, "obtaining a text to be explained for a target object, wherein the text to be explained includes at least one anchor text", is described in detail in conjunction with an embodiment.
[0063] In this application, the target object is the object to be displayed by the information interactive display system. The target object can be input by the user or preset by the technician according to the demand. This application can be applied to various fields, and the target object can be the object to be displayed in the field. In particular, the method of this application can be used in museums, and the target object can include: cultural relics, ancient books or murals, etc.
[0064] The present application obtains the text to be explained for the target object. The text to be explained can be obtained in a variety of ways. For example, a library of texts to be explained can be pre-established for multiple target objects. When an explanation is required for a certain target object, the required text to be explained is extracted from the library of texts to be explained. As a preferred implementation, the question text input by the user for the target object or the question text corresponding to the input question voice can be obtained; the knowledge base of the target object is used to generate a reply text for the question text, and the reply text is used as the text to be explained. Furthermore, when using the knowledge base to generate the reply text, a large language model can also be used. Specifically, a prompt instruction Prompt is generated using the knowledge base information of the target object and the question text; in the large language model, the prompt instruction (Prompt) is an input text used to guide the model to generate a specific type of text or perform a specific task, and the Prompt is provided to the large language model to obtain the reply text generated by the large language model for the question text.
[0065] The text to be explained in the present application includes at least one anchor text, which is the key content in the explanation text. The anchor text can be used as a kind of positioning information. When the digital human explains the text to be explained to the position of the anchor text, the first screen and the second screen can be triggered to perform related operations.
[0066] Anchor text can be determined in many ways. For example, a pre-trained machine learning model can be used to extract keywords from the text to be explained, and the extracted keywords can be used as anchor text. Alternatively, the part of speech and word frequency of the text to be explained can be analyzed, and nouns with high frequency of occurrence can be used as anchor text. Anchor text can also be determined based on user behavior, such as using keywords in the content of users' frequently asked questions as anchor text.
[0067] As a preferred implementation, the present application can use a pre-set keyword library when determining the anchor text, the keyword library contains multiple keywords, and the keywords in the keyword library correspond to digital human actions and target object areas. The content in the text to be explained that matches the keywords in the keyword library is used as the anchor text.
[0068] The above step 302, namely "generating a digital human broadcasting video using the text to be explained", is described in detail below in conjunction with an embodiment.
[0069] There are many ways to generate a digital human broadcast video using the text to be explained. For example, some automatic video generation platforms can be used, which can automatically convert text content into video, including speech synthesis and digital human animation; deep learning models such as Runway Gen-3 and Sora can also be used to generate more natural speech and more realistic facial animation.
[0070] The text to be explained in the present application includes an anchor text. When the anchor text is determined by using a keyword library, the anchor text corresponds to a digital human action. The text to be explained and the digital human action corresponding to the anchor text can be used to generate a digital human broadcast video. In the generated broadcast video, the digital human performs the action corresponding to the anchor text when broadcasting the anchor text.
[0071] The above step 303, i.e., "providing the image of the target object to the first screen so as to display the image of the target object on the first screen, and providing the digital human broadcasting video to the second screen so as to play the digital human broadcasting video on the second screen, wherein when the digital human broadcasts the anchor text in the digital human broadcasting video, the action corresponding to the anchor text is executed and the partial area of the target object corresponding to the anchor text is highlighted on the first screen" is described in detail in conjunction with an embodiment.
[0072] In this application, the first screen is used to display the target object. The image of the target object can be preset, or the image of the target object can be obtained based on the information of the target object input by the user. The image of the target object can be obtained from a local image library, or obtained by network download, large model generation, etc.
[0073] After acquiring the image of the target object, the image of the target object is provided to the first screen so as to display the image of the target object on the first screen. The digital human broadcast video is provided to the second screen so as to play the digital human broadcast video on the second screen.
[0074] When the digital human reads the anchor text in the digital human reading video, the action corresponding to the anchor text is performed and a partial area of the target object corresponding to the anchor text is highlighted on the first screen.
[0075] As an implementable method, when the digital human reads the anchor text, a highlighting instruction is sent to the first screen, and the highlighting instruction carries the target object area information corresponding to the anchor text, so as to highlight a partial area of the target object corresponding to the target object area information.
[0076] The action corresponding to the anchor text can be set according to needs, for example, emphasizing key information on the first screen through gestures, such as clapping, nodding or shaking the head, simulating interaction with the screen content, such as clicking, dragging, scrolling, etc.; in particular, the action corresponding to the anchor text includes: using gestures or tools to point to the highlighted partial area on the first screen.
[0077] As another practicable manner, the image of the target object is pre-divided into N grids corresponding to the first screen, where N is a positive integer greater than 1. According to the correspondence between the preset anchor text and at least one grid in the N grids, the image area of the grid corresponding to the anchor text is highlighted on the first screen. Figure 4 The first screen provided in the embodiment of the present application is divided into a nine-square grid to highlight the target area, such as Figure 4 As shown, the first screen is divided into nine grids, the image area in the grid in the upper left corner is highlighted, and the digital person on the second screen uses gestures to point to the grid in the upper left corner of the first screen, thereby presenting a strong sense of interaction between the first screen and the second screen to the user.
[0078] Among them, highlighting the partial area of the target object corresponding to the anchor text on the first screen can be, highlighting the partial area of the target object corresponding to the anchor text; or marking the partial area of the target object corresponding to the anchor text with a preset element, for example, framing the partial area with a square box, pointing to the partial area with an arrow, etc.; or displaying the partial area of the target object corresponding to the anchor text with a preset color, the preset color is different from the color used for other areas except the partial area; or, the partial area of the target object corresponding to the anchor text is enlarged and displayed; or, the partial area of the target object corresponding to the anchor text is centered on the first screen.
[0079] The above method provided by the embodiment of the present application can be applied to a variety of application scenarios, including but not limited to: in classroom teaching or online education, through digital human explanation and image display, students' learning experience and knowledge acquisition effect can be enhanced. At product launch conferences or exhibitions, digital human videos are used to introduce product features, while product images are displayed on the screen, so that users can understand the products more intuitively. In museums or exhibition halls, by setting up screens next to exhibits, displaying images related to the exhibits and digital human explanation videos, visitors can be provided with a richer exhibition experience.
[0080] Taking the method in this application for explaining the Qingming Shanghe Tu in a museum as an example, if the target object information input by the user is: "Please help me introduce the Qingming Shanghe Tu", the target object is "Qingming Shanghe Tu", and the generated explanation text content is a text about the Qingming Shanghe Tu, and the text contains the keyword "Hongqiao". The image of Qingming Shanghe Tu is displayed on the first screen. When the digital human explains Hongqiao, a highlighting instruction is sent to the first screen. The instruction contains the location information of Hongqiao. The first screen highlights the area of "Hongqiao" according to the instruction. On the second screen, the digital human can use a flashlight to point to the location of "Hongqiao" on the first screen, linking the area where "Hongqiao" is on the first screen, and highlighting the location of "Hongqiao" in a highlighted manner, presenting the user with an effect that the digital human illuminates this area and explains it.
[0081] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0082] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.
[0083] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0084] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.
[0085] And an electronic device, comprising:
[0086] one or more processors; and
[0087] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.
[0088] The present application also provides a computer program product, including a computer program, which implements the steps of any one of the methods in the aforementioned method embodiments when executed by a processor.
[0089] in, Figure 5The architecture of the electronic device is shown as an example, which may include a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, and a memory 520. The processor 510, the video display adapter 511, the disk drive 512, the input / output interface 513, the network interface 514, and the memory 520 may be communicatively connected via a communication bus 530.
[0090] The processor 510 may be implemented by a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in this application.
[0091] The memory 520 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 520 can store an operating system 521 for controlling the operation of the electronic device 500, and a basic input and output system (BIOS) 522 for controlling the low-level operation of the electronic device 500. In addition, a web browser 523, a data storage management system 524, and an information interaction display system 525, etc. can also be stored. The above-mentioned information interaction display system 525 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided in the present application is implemented by software or firmware, the relevant program code is stored in the memory 520 and is called and executed by the processor 510.
[0092] The input / output interface 513 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0093] The network interface 514 is used to connect to a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).
[0094] The bus 530 comprises a pathway for transmitting information between the various components of the device (eg, the processor 510, the video display adapter 511, the disk drive 512, the input / output interface 513, the network interface 514, and the memory 520).
[0095] It should be noted that, although the above device only shows a processor 510, a video display adapter 511, a disk drive 512, an input / output interface 513, a network interface 514, a memory 520, a bus 530, etc., in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.
[0096] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a computer program product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.
[0097] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.
Claims
1. An information interactive display method, applied to a system comprising at least a first screen and a second screen, wherein the first screen and the second screen are both arranged toward a user area, and an angle between the first screen and the second screen is within a preset angle range; characterized in that: The method comprises: Acquire a text to be explained for a target object, wherein the text to be explained includes at least one anchor text; Generating a digital human broadcast video using the text to be explained; The image of the target object is provided to the first screen to display the image of the target object on the first screen, and the digital human broadcast video is provided to the second screen to play the digital human broadcast video on the second screen, wherein when the digital human broadcasts the anchor text in the digital human broadcast video, the action corresponding to the anchor text is performed and the partial area of the target object corresponding to the anchor text is highlighted on the first screen.
2. The method according to claim 1, characterized in that The method further comprises: Acquire an image of the target object according to information of the target object input by the user; The image of the target object is provided to the first screen to display the image of the target object on the first screen.
3. The method according to claim 1, characterized in that The obtaining of the text to be explained for the target object comprises: Obtaining a question text input by a user for the target object or a question text corresponding to an input question voice; The knowledge base of the target object is used to generate a reply text for the question text, and the reply text is used as the text to be explained.
4. The method according to claim 3, characterized in that Using the knowledge base of the target object to generate a reply text for the question text includes: Generate a prompt instruction Prompt using the knowledge base information of the target object and the question text; The Prompt is provided to a large language model, and a reply text generated by the large language model for the question text is obtained.
5. The method according to claim 1, characterized in that The method further comprises: using a preset keyword library, determining content in the text to be explained that matches a keyword in the keyword library as an anchor text, wherein the keywords in the keyword library correspond to digital human actions and target object areas; Generating a digital human broadcast video using the text to be explained includes: generating the digital human broadcast video using the text to be explained and the digital human action corresponding to the anchor text; When the digital human reads the anchor text, a highlighting instruction is sent to the first screen, wherein the highlighting instruction carries the target object region information corresponding to the anchor text, so as to highlight a partial region of the target object corresponding to the target object region information.
6. The method according to claim 1, characterized in that The action corresponding to the anchor text includes: using a gesture or a tool to point to the partial area highlighted on the first screen.
7. The method according to claim 1, characterized in that The image of the target object is pre-divided into N grids corresponding to the first screen, where N is a positive integer greater than 1; Highlighting a partial area of the target object corresponding to the anchor text on the first screen includes: According to a preset correspondence between the anchor point text and at least one of the N grids, an image area of the grid corresponding to the anchor point text is highlighted on the first screen.
8. The method according to claim 1, characterized in that Highlighting a partial area of the target object corresponding to the anchor text on the first screen includes: highlighting a portion of the target object corresponding to the anchor text; or, Using a preset element to identify a partial area of the target object corresponding to the anchor text; or, Displaying a partial area of the target object corresponding to the anchor text in a preset color, where the preset color is different from a color used in other areas except the partial area; or Enlarging and displaying a partial area of the target object corresponding to the anchor text; or, A partial area of the target object corresponding to the anchor text is displayed in the center of the first screen.
9. The method according to any one of claims 1 to 8, characterized in that The target objects include: cultural relics, ancient books or murals.
10. An information interactive display system, characterized in that: The system comprises: a service providing device, a first screen and a second screen, wherein the first screen and the second screen are both arranged toward a user area, and an angle between the first screen and the second screen is within a preset angle range; The service providing device is used to obtain a text to be explained for a target object, wherein the text to be explained includes at least one anchor point text; generate a digital human broadcast video using the text to be explained and provide it to the second screen; and provide an image of the target object to the first screen; The first screen is used to display the image of the target object; The second screen is used to play the digital human broadcast video; Wherein, when the digital human reads the anchor text in the digital human reading video, the action corresponding to the anchor text is executed and a partial area of the target object corresponding to the anchor text is highlighted on the first screen.
11. The system according to claim 10, characterized in that The system further comprises: A first user input device, used to provide a user interface to the user, and provide the target object information input by the user through the user interface to the service providing device; The service providing device is further used to obtain an image of the target object according to the information of the target object input by the user.
12. The system according to claim 10, characterized in that The system further comprises: a second user input device, configured to obtain a question text or a question voice input by a user for the target object, and provide the question text or the question voice to the service providing device; The service providing device is used to generate a reply text for the question text by using the knowledge base of the target object, or to call a speech recognition system to convert the question speech into question text, and to generate a reply text for the question text by using the knowledge base of the target object; and the generated reply text is used as the text to be explained.
13. The system according to claim 10, characterized in that The service providing device is further used to use a preset keyword library to determine the content in the text to be explained that matches the keywords in the keyword library as anchor text, and the keywords in the keyword library correspond to digital human actions and target object areas; when the digital human broadcasts the anchor text, a highlighting instruction is sent to the first screen, and the highlighting instruction carries the target object area information corresponding to the anchor text, so as to highlight the partial area of the target object corresponding to the target object area information.
14. An electronic device, characterized in that: include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of claims 1 to 9.
15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.
16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.
