Conversational advertising system and method of use

Abstract

The invention relates to the field of advertisement systems, in particular to a dialogue type advertisement system and a use method. The advertisement system comprises a language processing and generating module used for processing and generating natural languages, a knowledge base used for supplementing knowledge for the language processing and generating module, a client side used for providing local services for users, a server side used for remotely providing services for the users, and terminal equipment used for interacting with advertisements. And the server side provides data and functions for the client side through the interface program. According to the invention, question-answer communication between the advertisement body and the user through voice is realized. Voice recognition is adopted, then answers are obtained through a language processing generation module and a knowledge base established by product data, and customers are answered by digital persons with actions and expressions. The interaction between the user and the advertisement is realized, the attention of the user to the advertisement is improved, the feedback information of the user is obtained, and an advertiser can conveniently improve an advertisement putting channel.

Description

Technical Field

[0001] The present invention relates to the field of advertising systems, and in particular to a conversational advertising system and a method of using the same. Background Art

[0002] Economic advertising refers to advertising for profit, usually commercial advertising, which is a means of disseminating information about goods or services to consumers or users through advertising media in a paid manner in order to promote goods or provide services. Existing advertisements do not have the ability to interact, but simply broadcast information content, which often does not attract the attention of users. Even if users have questions, they cannot ask questions or interact. Merchants who place advertisements also cannot obtain feedback data on their placements. Summary of the invention

[0003] The technical problem to be solved by the present invention is: In order to solve the technical problem described in the background technology, the present invention provides a conversational advertising system and a method of use. It realizes question-and-answer communication between the advertisement and the user through voice. First, voice recognition is used, and then the answer is obtained through the language processing generation module and the knowledge base established by the product information, and the digital person with actions and expressions answers the customer. It realizes the interaction between the user and the advertisement, improves the user's attention to the advertisement, and obtains the user's feedback information, which is convenient for advertisers to improve the advertising delivery channels.

[0004] The technical solution adopted by the present invention to solve its technical problem is: A conversational advertising system includes a language processing generation module for processing and generating natural language, a knowledge base for supplementing knowledge for the language processing generation module, a client for providing local services to users, a server for providing services to users remotely, and a terminal device for interacting with advertisements. The server provides data and functions to the client through an interface program.

[0005] Specifically, the language processing generation module includes a large language model, a speech recognition model, and a speech production model.

[0006] Specifically, the language processing generation module includes a speech multimodal large language model.

[0007] Specifically, the conversational advertising system includes a device for playing an image model and an action image model of a digital human on a display of a terminal device; The source of the digital human is a real person or a virtual person generated by AIGC; the digital human expresses facial expressions in the reply according to the user's emotional label identified by the language processing generation module; The establishment of digital human includes the method of making digital human modules and the realization of the actions of digital human during playback; The method of making a digital human module is to first make a video of the action template of the digital human. For a digital human from a real person, you can record it; for a virtual human generated by AIGC, use prompt words to generate a video to let the virtual human make the required action; after obtaining the video, extract the position vectors of the hand, lower arm, upper arm, head, torso, foot, calf, thigh, and eye at the beginning and end of the template video; The method for implementing the actions of the digital human during playback is to first find out based on the content that if there is a product name or company name in the text to be spoken, or if corresponding actions are required based on the text script, call the corresponding template; insert an intermediate value between the current character's action and the starting action of the template, so that the digital human transitions from one action to another.

[0008] A method for using a conversational advertising system, the specific steps of the method are: S1. The user interacts with the advertisement supported by the advertisement system through the terminal device; S2, the advertisement body asks the user whether he is interested in a certain product; S3. If the user indicates through the terminal device that he is not interested, the advertisement body will change the advertisement product; if the user indicates through the terminal device that he is interested or ignores it, the advertisement will start playing; S4. The advertisement encourages users to ask questions; S5. If the user does not ask a question, the advertisement product is changed. If the user asks a question, the questioning phase is entered; S6. The user's question enters the advertising system through voice, and is answered after being processed by the language processing generation module; S7. Finally, collect data for consultation and provide feedback.

[0009] Specifically, the answering steps processed by the language processing generation module are: S61, the user sends a voice message when asking a question; S62: The speech uttered by the user is converted into text through the speech recognition model and merged with the historical answer text into a text collection; S63, the text collection is input into the large language model and the knowledge base, and the knowledge base supplements the content of the large language model; S64, the large language model outputs the text to the speech sound model, thereby converting the text into speech; S65. Finally, the text is replied to the user.

[0010] Specifically, in step S64, after the speech production model converts the text into speech, it first generates facial image frames from the speech through the image model, then generates body movement frames through the action image model, and finally generates a digital human to reply to the user.

[0011] Specifically, the answering steps processed by the language processing generation module are: S71, the user sends a voice message when asking a question; S72: The speech uttered by the user is converted into text through the speech recognition model and merged with the historical answer text into a text collection; S73, the text set is input into the speech multimodal large language model and the knowledge base, the knowledge base supplements the speech multimodal large language model with content, and generates a reply speech; S74, the answering voice is replied to the user; the answering voice first generates a facial image frame through the image model, then generates a body movement frame through the action image model, and finally generates a digital human to reply to the user.

[0012] Specifically, the terminal device includes a screen, a voice input device, a voice output device, and a camera; the data information of each customer is obtained through the camera, and the customer attributes are identified and marked and stored in the database to form a record.

[0013] Specifically, the knowledge base includes document information, image information, and table information. The knowledge base uses retrieval enhancement generation technology to perform retrieval and answer; the retrieval enhancement generation technology includes two parts: establishing a knowledge base and retrieval; The specific steps to build a knowledge base are: S11, dividing the input document into blocks by using a text segmentation technology, and converting the blocks into text blocks; S12, analyzing each text block and extracting graph elements to generate a knowledge graph, where the graph elements include entities, relationships, and descriptions; S13. Apply recursive scene clustering to the knowledge graph using the hierarchical Leiden algorithm until a scene size threshold is reached to understand the scene hierarchy of the graph. S14. Use the node vector algorithm to generate vector representations of nodes in the graph, thereby realizing knowledge graph embedding; S15. Generate a summary of each scenario using a large language model. Each scenario report is summarized by a large language model for summary use. Generate a vector representation of the scenario by generating text embeddings of the scenario report, scenario report summary, and scenario report title. S16, creating a text block graph and vectorizing the document; S17, finally generating a node table; Retrieval includes detailed retrieval, global retrieval, and answer selection; Detailed retrieval: Entity-based reasoning local retrieval methods generate answers by combining relevant data from the knowledge graph extracted by a large language model with text chunks from the original document; Global retrieval is based on reasoning over the entire dataset. Global retrieval methods generate answers by searching all generated scenario reports in a map-reduce manner. The answer selection is to obtain the answers output by the large language model, the answers output by the global retrieval, and the answers output by the detailed retrieval, make a selection based on the answer content, and then get the result answer through the large language model; the answer output by the detailed retrieval has a higher priority than the answer output by the global retrieval, and the answer output by the global retrieval has a higher priority than the answer output by the large language model.

[0014] The beneficial effects of the present invention are as follows: the present invention provides a conversational advertising system and a method of using the same. The system realizes question-and-answer communication between the advertisement and the user through voice. Voice recognition is first used, and then the answer is obtained through the language processing generation module and the knowledge base established by the product information, and the digital person with actions and expressions answers the customer. The system realizes the interaction between the user and the advertisement, improves the user's attention to the advertisement, and obtains the user's feedback information, which is convenient for advertisers to improve the advertising delivery channels. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0016] Figure 1 It is a structural block diagram of the present invention; Figure 2 It is a structural block diagram of the present invention using a large language model; Figure 3 It is a structural block diagram of the present invention using a speech multimodal large language model; Figure 4 is a flow chart of the method of using the present invention; Figure 5 is a flow chart of a method for using a large language model in the present invention; Figure 6 It is a flow chart of the method of using the speech multimodal large language model of the present invention; DETAILED DESCRIPTION

[0017] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0018] Figure 1 It is a structural block diagram of the present invention; Figure 2 It is a structural block diagram of the present invention using a large language model; Figure 3 It is a structural block diagram of the present invention using a speech multimodal large language model; Figure 4 is a flow chart of the method of using the present invention; Figure 5 is a flow chart of a method for using a large language model in the present invention; Figure 6 It is a flow chart of the method of using the speech multimodal large language model of the present invention.

[0019] The technical solution adopted by the present invention to solve its technical problem is: As attached Figure 1 As shown, a conversational advertising system includes a language processing generation module for processing and generating natural language, a knowledge base for supplementing knowledge for the language processing generation module, a client for providing local services to users, a server for providing services to users remotely, and a terminal device for interacting with advertisements. The server provides data and functions to the client through an interface program.

[0020] Terminal devices include screens, voice input devices, voice output devices, and cameras.

[0021] In conversational advertising, since there are cameras on large screens or mobile phones, it is possible to obtain videos of the environment before the advertisement. Obtain information such as customer attributes, customer quantity, customer distance, customer stay time, and customer eye focus time in front of the advertisement screen. This information is of great significance to product manufacturers who place advertisements, as it allows them to know where to place advertisements, compare the effects of different advertisement contents, and evaluate the effects of advertisement placement. After obtaining the video, image recognition is performed directly on the advertising device (advertising screen or mobile phone, computer). This image recognition does not process every frame of the video, but performs frame extraction recognition. In the obtained video, statistics are performed for each customer who is close to the distance (the distance is configured as a parameter in the advertisement body). However, the obtained video is not saved, so it does not infringe on the privacy of customers, but only counts the number.

[0022] The data information of each customer is obtained through the camera, and the customer attributes are identified and marked and stored in the database to form a record. If necessary, the customer's facial feature vector is also recorded. When the next frame of the image is processed, the customer in this frame is marked again. When the facial feature vector is found to match the previous n frames, only the record of the customer is updated without adding a new record. In this way, the number and attributes of customers, as well as the stay time of each customer, can be counted. When the user pays attention to the advertisement, both eyes will naturally stare at the advertisement at the same time. First, train an artificial intelligence model, label many pictures that the eyeballs pay attention to and pictures that they don't pay attention to, and input these label data into the model for training. The model obtained after training can identify whether the customer pays attention to specific content, and make corresponding annotations when obtaining customer attributes. According to multiple frames in the video, the length of time that the customer's eyes pay attention can be calculated.

[0023] The knowledge base is an additional database built to make up for the lack of specific knowledge in the language processing generation module, or to meet the developer's requirements for the language processing generation module to make specific answers. When building a knowledge base, you only need to input documents in various formats, and the language processing generation module will be able to understand and build the knowledge base by itself.

[0024] As attached Figure 2 As shown, the language processing generation module includes a large language model, a speech recognition model, and a speech sound model. The large language model is a deep learning model that can generate natural language or understand the meaning of language. The model generally has high general self-knowledge and can output text or audio.

[0025] As attached Figure 3 As shown, the language processing generation module includes a speech multimodal large language model. The speech multimodal large language model is a large language model that can process and generate speech and text information. The model not only has the ability to understand and generate natural language text, but also can process speech signals to achieve speech recognition and speech synthesis.

[0026] The conversational advertising system includes an image model and an action image model for playing digital humans on the terminal device display. Digital humans are digital human images with human appearance created using digital technology. Digital humans come from real people or virtual humans generated by AIGC; digital humans express facial expressions in replies based on user emotion tags identified by the language processing generation module; The establishment of digital human includes the method of making digital human modules and the realization of the actions of digital human during playback; The method of making a digital human module is to first make a video of the action template of the digital human. For a digital human from a real person, you can record it; for a virtual human generated by AIGC, use prompt words to generate a video to let the virtual human make the required action; after obtaining the video, extract the position vectors of the hand, lower arm, upper arm, head, torso, foot, calf, thigh, and eye at the beginning and end of the template video; The method for implementing the actions of the digital human during playback is to first find out based on the content that if there is a product name or company name in the text to be spoken, or if corresponding actions are required based on the text script, call the corresponding template; insert an intermediate value between the current character's action and the starting action of the template, so that the digital human transitions from one action to another.

[0027] As attached Figure 4 As shown, a method for using a conversational advertising system, the specific steps of the method are: S1. The user interacts with the advertisement supported by the advertisement system through the terminal device; S2, the advertisement body asks the user whether he is interested in a certain product; S3. If the user indicates through the terminal device that he is not interested, the advertisement body will change the advertisement product; if the user indicates through the terminal device that he is interested or ignores it, the advertisement will start playing; S4. The advertisement encourages users to ask questions; S5. If the user does not ask a question, the advertisement product is changed. If the user asks a question, the questioning phase is entered; S6. The user's question enters the advertising system through voice, and is answered after being processed by the language processing generation module; S7. Finally, collect data for consultation and provide feedback.

[0028] As attached Figure 5 As shown, the answer steps processed by the language processing generation module are: S61, the user sends a voice message when asking a question; S62: The speech uttered by the user is converted into text through the speech recognition model and merged with the historical answer text into a text collection; S63, the text collection is input into the large language model and the knowledge base, and the knowledge base supplements the content of the large language model; S64, the large language model outputs the text to the speech sound model, thereby converting the text into speech; S65. Finally, the text is replied to the user.

[0029] In step S64, after the speech production model converts the text into speech, it first generates facial image frames from the speech through the image model, then generates body movement frames through the action image model, and finally generates a digital human to reply to the user.

[0030] As attached Figure 6 As shown, the answer steps processed by the language processing generation module are: S71, the user sends a voice message when asking a question; S72: The speech uttered by the user is converted into text through the speech recognition model and merged with the historical answer text into a text collection; S73, the text set is input into the speech multimodal large language model and the knowledge base, the knowledge base supplements the speech multimodal large language model with content, and generates a reply speech; S74. The answer voice is replied to the user.

[0031] The answering voice first generates facial image frames through the image model, then generates body movement frames through the action image model, and finally generates a digital person to reply to the user.

[0032] In the voice question-and-answer communication between the advertising entity and the user, a digital human image can be used for voice dialogue, or a direct voice dialogue can be used. The system processes the user's voice and enters the large language model for processing to obtain an answer. The large language model can be a text input and output, or it can be a direct voice input and output of an end-to-end multi-model large language model. In the large language model processing, a vector knowledge base generated by the advertising provider from the product information is also added. Such a knowledge base is retrieved in the form of associated queries or knowledge graphs based on the materials provided by the advertiser, so that the advertising entity can answer consultation questions related to specific products according to the requirements of the advertising provider. After the answer is generated, the advertising entity answers the consultation question to the user through a digital human or direct voice.

[0033] Through conversational digital humans, speech recognition, large language models, and knowledge base technologies, digital humans are placed in advertisements to build conversational advertisements. When playing advertisements, users can be asked about their intentions before playing the corresponding advertisements. Users can also interrupt and ask merchants for information about the products in the advertisements.

[0034] Such advertisements can fully mobilize users' attention, involve users in the advertisement playback, and answer users' questions. For merchants, they can collect information about users' questions and obtain real data on the effectiveness of the delivery channels to enhance the improvement of advertising delivery.

[0035] In advertisements, if digital humans are used, the digital images and voices of celebrities can be adopted, so that users can have direct conversations with celebrities, greatly enhancing the effectiveness of the advertisements.

[0036] The application scenarios of conversational advertising include outdoor large-screen advertising, indoor screen advertising, staircase advertising, online advertising, and mobile phone advertising.

[0037] The knowledge base includes document information, image information, and table information. The knowledge base uses retrieval enhancement generation technology to search and answer questions. The retrieval enhancement generation technology includes two parts: establishing a knowledge base and searching. The specific steps to build a knowledge base are: S11, dividing the input document into blocks by using a text segmentation technology, and converting the blocks into text blocks; S12. Analyze each text block and extract graph elements to generate a knowledge graph. The graph elements include entities, relationships, and descriptions.

[0038] Regarding entity and relation extraction, each unit of text is processed in the first step of graph extraction to extract entities and relations from the raw text using a large language model. The output of this step is a subgraph for each unit of text, containing a list of entities with name, type, and description, and a list of relations with source, target, and description. These subgraphs are merged together, and any entities with the same name and type are merged by creating an array of their descriptions. Similarly, any relations with the same source and target are merged by creating an array of their descriptions.

[0039] Regarding entity and relationship summaries, once we have a graph of entities and relationships, each with a list of descriptions, we can summarize these lists into a single description for each entity and relationship. This can be done by asking the large language model to provide a short summary that captures all the different information in each description. This allows all entities and relationships to have a concise description.

[0040] Regarding entity disambiguation, the last step of knowledge graph extraction is to resolve any entities that represent the same real-world entity but have different names. Since this is done with a large language model and we don't want to lose information, we want to take a conservative, non-destructive approach. However, the current implementation of entity resolution is destructive. It will provide a list of entities to the large language model and ask it to determine which entities should be merged. These entities will then be merged into a single entity and their relationships will be updated. Other entity resolution techniques are currently being explored. In the near future, entity resolution will be performed by creating an edge between entity variants, indicating that the entity has been resolved by the indexing engine. This will allow end users to undo index-side resolution and add their own non-destructive resolution using a similar process.

[0041] Regarding narrative extraction, finally, claims are extracted from source text blocks. These claims represent positive factual statements with evaluation status and time constraints.

[0042] S13. Apply recursive scenario clustering to the knowledge graph using the hierarchical Leiden algorithm until a scenario size threshold is reached, thereby understanding the scenario hierarchy of the graph; and provide a method to navigate and summarize the graph at different levels of granularity.

[0043] S14. Use the node vector algorithm to generate vector representations of nodes in the graph to achieve knowledge graph embedding; this will enable understanding of the implicit structure of the graph and provide additional vector space for searching related concepts during the query phase.

[0044] S15. Generate a summary of each scenario using a large language model. Each scenario report is summarized by a large language model for summary use. Generate a vector representation of the scenario by generating text embeddings of the scenario report, scenario report summary, and scenario report title. S16. Create a text block graph and vectorize the document. In the step of connecting to text blocks, the application connects each document to the text unit created in the first stage and creates contextual relationships between text blocks. This enables the application to understand which documents are related to which text units. In the document embedding step, the average embedding of the document slices is used to generate a vector representation of the document. Re-block the documents for non-overlapping blocks, and then generate embeddings for each block. Create an average of these blocks, weighted by the tag count, and use it as a document embedding. This will help to intuitively understand the relationship between nodes in the graph in two-dimensional space.

[0045] S17. Finally, a node table is generated; at this stage of the workflow, the application performs some steps to support network visualization in the high-dimensional vector space of the existing graph. There are two logical graphs in play at this time: the entity relationship graph and the document graph. For each logical graph, dimensionality reduction is performed to generate a two-dimensional representation of the graph. This will enable understanding of the relationship between nodes in the graph in a two-dimensional space. Then embedding as a node table. The rows of this table include a discriminator indicating whether the node is a document or an entity, as well as mapping coordinates.

[0046] Retrieval includes detailed retrieval, global retrieval, and answer selection; Detail retrieval is a local retrieval method based on entity reasoning, which generates answers by combining relevant data from the knowledge graph extracted by a large language model with text chunks of the original document; The specific steps of detail retrieval are: given a user query and conversation history, the detail retrieval method identifies a set of entities from the knowledge graph that are semantically related to the user input. These entities can be used as access points to the knowledge graph to extract more relevant details, such as connected entities, relationships, entity descriptions, and scenario reports. In addition, it extracts relevant text blocks from the original input documents related to the identified entities. These candidate data sources are then prioritized and filtered to fit into a single context window of a predefined size, which is used to generate a response to the user query.

[0047] Global retrieval is based on reasoning over the entire dataset. Global retrieval methods generate answers by searching all generated scenario reports in a map-reduce manner.

[0048] The specific steps of global retrieval are: 1. Split the input file into branches. Since the test file is small, each file is used as a branch and the file is split by line. This step is automatically completed by the ma-preduce framework. (Searching semantically related scenes through vectors) 2. Submit the segmented files to the user-defined map method for processing and generate<key,value> Yes. (The score of the relevance between the scenario and the user's question is generated, and the scenario node itself forms a MAP) 3. Get the output of the map method<key,value> After the comparison, the same key values ​​will be put together in the sorting process. The scenario results are sorted based on their relevance to the question, and the components can be understood.

[0049] 4. Reduce process, add up the values ​​of the same key value to get a new<key,value> After comprehensive scoring within the group, the results are sorted and items with a score of 0 are filtered out, and finally the large language model generates a response.

[0050] The answer selection is to obtain the answers output by the large language model, the answers output by the global retrieval, and the answers output by the detailed retrieval, make a selection based on the answer content, and then get the result answer through the large language model; the answer output by the detailed retrieval has a higher priority than the answer output by the global retrieval, and the answer output by the global retrieval has a higher priority than the answer output by the large language model.

[0051] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A conversational advertising system, characterized in that: It includes a language processing generation module for processing and generating natural language, a knowledge base for supplementing knowledge for the language processing generation module, a client for providing local services to users, a server for providing services to users remotely, and a terminal device for interacting with advertisements. The server provides data and functions to the client through an interface program.

2. The interactive advertising system according to claim 1, characterized in that: The language processing generation module includes a large language model, a speech recognition model, and a speech production model.

3. The interactive advertising system according to claim 1, characterized in that: The language processing generation module includes a speech multimodal large language model.

4. The interactive advertising system according to claim 1, characterized in that: The conversational advertising system includes a device for playing a digital human image model and a motion image model on a terminal device display; The source of the digital human is a real person or a virtual person generated by AIGC; the digital human expresses facial expressions in the reply according to the user's emotional label identified by the language processing generation module; The establishment of digital human includes the method of making digital human modules and the realization of the actions of digital human during playback; The method of making a digital human module is to first make a video of the action template of the digital human. For a digital human from a real person, you can record it; for a virtual human generated by AIGC, use prompt words to generate a video to let the virtual human make the required action; after obtaining the video, extract the position vectors of the hand, lower arm, upper arm, head, torso, foot, calf, thigh, and eye at the beginning and end of the template video; The method for implementing the actions of the digital human during playback is to first find out based on the content that if there is a product name or company name in the text to be spoken, or if corresponding actions are required based on the text script, call the corresponding template; insert an intermediate value between the current character's action and the starting action of the template, so that the digital human transitions from one action to another.

5. The method for using the conversational advertising system according to any one of claims 1 to 4, characterized in that: The specific steps of this method are: S1. The user interacts with the advertisement supported by the advertisement system through the terminal device; S2, the advertisement body asks the user whether he is interested in a certain product; S3. If the user indicates through the terminal device that he is not interested, the advertisement body will change the advertisement product; if the user indicates through the terminal device that he is interested or ignores it, the advertisement will start playing; S4. The advertisement encourages users to ask questions; S5. If the user does not ask a question, the advertisement product is changed. If the user asks a question, the questioning phase is entered; S6. The user's question enters the advertising system through voice, and is answered after being processed by the language processing generation module; S7. Finally, collect data for consultation and provide feedback.

6. The method for using the interactive advertising system according to claim 5, characterized in that: The answering steps processed by the language processing generation module are: S61, the user sends a voice message when asking a question; S62: The speech uttered by the user is converted into text through the speech recognition model and merged with the historical answer text into a text collection; S63, the text collection is input into the large language model and the knowledge base, and the knowledge base supplements the content of the large language model; S64, the large language model outputs the text to the speech sound model, thereby converting the text into speech; S65. Finally, the text is replied to the user.

7. The method for using the interactive advertising system according to claim 6, characterized in that: In step S64, after the speech production model converts the text into speech, it first generates facial image frames from the speech through the image model, then generates body movement frames through the action image model, and finally generates a digital human to reply to the user.

8. The method for using the interactive advertising system according to claim 5, characterized in that: The answering steps processed by the language processing generation module are: S71, the user sends a voice message when asking a question; S72: The speech uttered by the user is converted into text through the speech recognition model and merged with the historical answer text into a text collection; S73, the text set is input into the speech multimodal large language model and the knowledge base, the knowledge base supplements the speech multimodal large language model with content, and generates a reply speech; S74, the answering voice is replied to the user; the answering voice first generates a facial image frame through the image model, then generates a body movement frame through the action image model, and finally generates a digital human to reply to the user.

9. The method for using the interactive advertising system according to claim 8, characterized in that: The terminal device includes a screen, a voice input device, a voice output device, and a camera; the data information of each customer is obtained through the camera, the customer attributes are identified and marked and stored in the database to form a record.

10. The method for using the interactive advertising system according to claim 1, characterized in that: The knowledge base includes document information, image information, and table information. The knowledge base uses retrieval enhancement generation technology to search and answer questions. The retrieval enhancement generation technology includes two parts: establishing a knowledge base and searching. The specific steps to build a knowledge base are: S11, dividing the input document into blocks by using a text segmentation technology, and converting the blocks into text blocks; S12, analyzing each text block and extracting graph elements to generate a knowledge graph, where the graph elements include entities, relationships, and descriptions; S13. Apply recursive scene clustering to the knowledge graph using the hierarchical Leiden algorithm until a scene size threshold is reached to understand the scene hierarchy of the graph. S14. Use the node vector algorithm to generate vector representations of nodes in the graph, thereby realizing knowledge graph embedding; S15. Generate a summary of each scenario using a large language model. Each scenario report is summarized by a large language model for summary use. Generate a vector representation of the scenario by generating text embeddings of the scenario report, scenario report summary, and scenario report title. S16, creating a text block graph and vectorizing the document; S17, finally generating a node table; Retrieval includes detailed retrieval, global retrieval, and answer selection; Detailed retrieval: Entity-based reasoning local retrieval methods generate answers by combining relevant data from the knowledge graph extracted by a large language model with text chunks from the original document; Global retrieval is based on reasoning over the entire dataset. Global retrieval methods generate answers by searching all generated scenario reports in a map-reduce manner. The answer selection is to obtain the answers output by the large language model, the answers output by the global retrieval, and the answers output by the detailed retrieval, make a selection based on the answer content, and then get the result answer through the large language model; the answer output by the detailed retrieval has a higher priority than the answer output by the global retrieval, and the answer output by the global retrieval has a higher priority than the answer output by the large language model.

Images