Large screen content recall method and device, storage medium and program product

Abstract

The invention provides a large screen content recall method and device, a storage medium and a program product, and relates to the technical field of semantic understanding. The large-screen content recall method comprises the following steps: importing large-screen item data for display into a large-screen database, extracting a keyword and an abstract summary corresponding to each item in a targeted manner, and generating a semantic vector code corresponding to the abstract summary; performing verbal skill intention judgment on the input of the user by using a large language model; performing mixed retrieval coarse recall in a keyword hit and semantic vector matching mode, and performing deep semantic analysis and screening by using a large language model to realize accurate recall; and a judger is used for overall planning and allocation of overall intention recognition and project recall, and response output is carried out through a large screen. According to the method, the recall accuracy and integrity in a complex scene can be effectively improved, meanwhile, the response efficiency and resource consumption of the system are considered, second-level response is achieved, and the real-time interaction experience and application efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of semantic understanding technology, and in particular to a method, apparatus, storage medium, and program product for recalling content on large screens. Background Technology

[0002] With the continuous advancement of artificial intelligence, natural language processing, and big data visualization technologies, large-screen display systems are playing an increasingly important role in smart cities, corporate showrooms, and other scenarios. In these systems, the key to improving display efficiency and user experience lies in how to quickly and accurately retrieve content highly relevant to the current user input (such as case studies, products, patents, awards, etc.) from a massive project database.

[0003] Most traditional large-screen content display systems rely on keyword matching or rule-driven methods for content retrieval. While this approach is simple to implement and quick to respond, it has significant limitations in semantic understanding and content relevance assessment. Because it heavily depends on the literal overlap between user input and words in the content library, it struggles to effectively handle semantic similarity matching. This results in low retrieval accuracy when faced with complex scenarios such as ambiguous expressions, polysemous terms, domain-specific terminology, or multilingual expressions, as it fails to accurately capture user intent.

[0004] More optimized retrieval and recall methods include vector matching, a combination of vector retrieval and keyword retrieval, and a fine-grained ranking model based on initial vector matching and subsequent ranking. However, these methods still fall short in their ability to deeply understand user intent. Especially when user queries are complex, the recall effectiveness of these methods will decrease. For example, if a user wants to view all content under a specific tag, vector matching may result in omissions, posing a significant challenge to the completeness of the retrieved content. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a method, device, storage medium, and program product for large-screen content retrieval, providing a control and operation system for intelligent large-screen devices. These large-screen devices are used in exhibition halls or company lobbies to showcase and introduce case studies, products, patents, awards, and other projects to visitors or clients. In a given presentation setting, the project data in the large-screen device is closed, static data, which is periodically updated according to presentation needs. This project data is stored in a database. Based on user commands, the large-screen device retrieves the most matching project from the database and displays it to the user.

[0006] The method of this invention leverages the semantic understanding capabilities of natural language models, particularly in tasks such as text matching, intent recognition, and content generation. Building upon initial keyword matching, it further utilizes large language models to perform in-depth semantic analysis and matching of user input and various types of information in the content library, thereby improving the relevance and accuracy of the recall results.

[0007] In particular, this invention integrates a large language model with conventional keyword-based and semantic vector matching methods. By using a hybrid retrieval approach, it performs coarse recall of content that users may be interested in, and then uses the powerful semantic understanding capabilities of the large language model to finely filter the recall results, ensuring the accuracy and completeness of the recalled content. This effectively improves the recall effect in complex scenarios and meets the diverse query needs of users.

[0008] Furthermore, this invention, through a reasonable model invocation mechanism and a lightweight semantic processing flow, ensures the depth of semantic understanding while also taking into account the system's response efficiency and resource consumption.

[0009] The core of the method of this invention includes two parts: data initialization construction and the construction of a recall technology framework.

[0010] In the information collection and organization phase of data initialization, a large language model is used to extract key information and summarize the content of each case project. This step significantly reduces the workload of manual data organization and enables rapid and efficient initial database construction. This data construction also provides a data foundation for subsequent recall efforts.

[0011] The recall framework significantly reduces response time during user interaction through a multi-step parallel implementation, achieving rapid and accurate content recall. In large-screen display scenarios, it can achieve second-level response times, thereby significantly improving the system's real-time interactive experience and application performance.

[0012] Specifically:

[0013] In a first aspect, the present invention provides a method for recalling content on a large screen, comprising the following steps:

[0014] S1. Data initialization and construction: Organize and build a knowledge base for the large screen data to be displayed; the large screen data contains multiple items, each item is imported into the large screen database, and the keywords and summary summaries corresponding to each item are extracted in a targeted manner to generate semantic vector codes corresponding to the summary summaries;

[0015] S2, large screen response and content recall, the specific steps include:

[0016] S21. Organize the user's input and convert the voice input into text;

[0017] S22. Overall Intent Recognition: Use a large language model to determine the user's linguistic intent and match it with the candidate operations on the large screen;

[0018] S23. Item Recall: A coarse recall is achieved by combining keyword matching with semantic vector matching. Then, a large language model is used to perform in-depth semantic analysis and screening on the candidate items output in the coarse recall stage of the coarse recall stage to achieve precise recall.

[0019] S24. Information Integration and Transfer: Steps S22 and S23 are executed simultaneously. The judgment unit coordinates and allocates information and collects the final output to reduce the response time of the large screen.

[0020] As a further improvement of the present invention, the specific steps of step S1 include:

[0021] S11. Entity extraction for projects: Use a large language model to extract keywords from the name and details of each project and store them in a regular database;

[0022] S12. Summarize the projects: Use a large language model to summarize the project details of each project. Encode the generated summaries into semantic vectors using a vectorization model and store them in a vector database.

[0023] As a further improvement of the present invention, the specific steps of step S22 include:

[0024] S221. Candidate operation intent sorting: sort out the operation instructions allowed by the large screen in the current scenario to obtain candidate instruction intents, including regular operation intents and other intents. The other intents are used to determine whether the user is asking or querying a specific item or a group of items.

[0025] S222. Construct an intent recognition prompt, allowing the large language model to match the most appropriate instruction intent based on the user's utterance.

[0026] As a further improvement of the present invention, in step S222, the following attributes of each intent are defined by intent recognition prompt:

[0027] id: uniquely corresponds to an intent;

[0028] name: Intent name;

[0029] description: Provide a detailed description of the intent;

[0030] Meanwhile, the constraint model only returns the intent ID.

[0031] As a further improvement of the present invention, step S23 includes:

[0032] S231. Hybrid retrieval and coarse recall, the specific steps of which include:

[0033] The steps for keyword matching are as follows: when the keyword corresponding to the project appears in the user's words, the project is recalled;

[0034] The steps of semantic vector matching are as follows: using the vectorization model used to generate semantic vector encoding in step S1, the user's input speech is vectorized; a similarity threshold is set, and when the similarity between the user's input text and the summary text of a certain item is greater than the similarity threshold, the item is recalled.

[0035] As a further improvement of the present invention, step S23 includes:

[0036] S232, Deep semantic analysis and screening, the specific steps of which include:

[0037] Using a large language model, based on the understanding of the natural language context, semantic intent, and potential needs of user input, we perform deep semantic analysis and screening on the candidate items output in the coarse recall stage of hybrid retrieval.

[0038] As a further improvement of the present invention, the prompt constructed for calling the large language model includes the following steps:

[0039] Enter background information;

[0040] For each candidate project, add the following attributes: ID uniquely corresponding to a candidate project, project name, and project keywords;

[0041] Based on the input dialogue, return the IDs of one or more of the most relevant items, and do not return any other content.

[0042] As a further improvement of the present invention, the specific steps of step S24 include:

[0043] The judge makes a preliminary judgment on the results of the overall intent recognition step S22.

[0044] If the judgment result is a regular operation intention on the large screen, the operation instruction on the large screen will be sent directly to the large screen without waiting for the result of the S23 project recall step to return.

[0045] If the determination result indicates other intentions, wait for the return of the result from the S23 project recall step, perform simple sorting, and then send it to the large screen for display of one or more items.

[0046] In a second aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0047] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0048] Fourthly, the present invention provides a computer program product that, when executed by a processor, implements the steps of the method described in the first aspect.

[0049] Compared with existing technologies, this invention, based on preliminary keyword matching, further utilizes a large language model to perform in-depth semantic analysis and matching of user input and various types of information in the content library, thereby improving the relevance and accuracy of the recall results. This invention also ensures the depth of semantic understanding while taking into account the system's response efficiency and resource consumption through a reasonable model calling mechanism and a lightweight semantic processing flow. This invention is particularly effective in improving the response speed of large-screen recall for static project data. Attached Figure Description

[0050] Figure 1 This is a flowchart of a large-screen content recall method disclosed in this invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Steps S1, S2… in the described embodiments of the present invention do not limit the scope of execution of the present invention; the various models, simulation environments, and software described in the present invention are not considered as the only limiting methods of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0052] In this invention, computer device / equipment / system refers to a related entity applied to a computer, such as hardware, a combination of hardware and software, software, or software in execution. More specifically, for example, software includes, but is not limited to, a process running on a processor, a processor, an object, executable software, an execution thread, a program, and / or a computer. Furthermore, an application program or script running on a server, and the server itself, can also be software. One or more software programs may be in an execution process and / or thread, and the software may be localized on one computer and / or distributed across two or more computers, and may be run on various computer-readable media.

[0053] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0054] In a first aspect, the present invention provides an embodiment of a method for recalling content on a large screen, such as... Figure 1 As shown, the specific process can be as follows:

[0055] S1: Data initialization and construction;

[0056] The data is organized and a knowledge base is built. The data contains multiple projects. Each project is imported into the database for storage, and keywords and project descriptions are extracted for each project. This step mainly relies on the entity extraction and summarization capabilities of the large language model.

[0057] In one large-screen display example, the first step is to organize and build a knowledge base for the data that the user wants to display. The data contains multiple items, and the user inputs information via voice or text. The machine recognizes the user's input and responds, returning one or more items that match the user's ideas.

[0058] The projects can be of various types, such as products, case studies, awards, and papers. Each project has a project name and project details.

[0059] S11: Extract entities from the project;

[0060] Build a prompt so that the large language model can extract key lexical information from the name and details of each item.

[0061] In one embodiment of the present invention, the constructed prompt is:

[0062] "Please extract some of the most representative keywords from the following project names and descriptions. The keywords should accurately reflect the core content and technical direction of the project."

[0063] Project Name: {Project Name};

[0064] Project Description: {Project Details};

[0065] The keywords extracted in this step are stored in a regular database.

[0066] S12: Provide a summary of the project;

[0067] Build a prompt, allowing the large language model to summarize the project details of each project, extract the key content of the project, and describe it in relatively concise language:

[0068] In one embodiment of the present invention, the constructed prompt is:

[0069] "Please summarize the following project description in no more than 100 words. The language should be concise and clear, highlighting the core achievements and application value of the project."

[0070] Project Name: {Project Name};

[0071] Project Description: {Project Details};

[0072] The summary generated in this step will be semantically vectorized using a vectorization model (such as Sentence-BERT, bge-embedding, etc.) and stored in a vector database to support subsequent semantic matching and similarity calculation.

[0073] S2: Construction of the recall technology framework;

[0074] like Figure 1 As shown, in the recall technology framework, the user's text or voice input is analyzed and understood, the user's intent is identified, and a corresponding response is made through the large screen. The response here includes the operation of the instructions on the large screen, as well as the recall of content in the database.

[0075] S21: Convert the user's voice input;

[0076] When a user sends a command via voice, the ASR speech-to-text module converts the voice into text.

[0077] S22: Overall intent recognition;

[0078] To support users in performing touchless command operations on the large screen, an intent recognition module is used to determine the intent of the input after the user's input is received.

[0079] S221: Organizing candidate operation intentions;

[0080] The operation commands allowed on the large screen in the current scenario are sorted out to obtain the intent of candidate commands.

[0081] The candidate instruction intent includes another intent whose primary purpose is to determine whether the user is asking or querying a specific item or a set of specific items.

[0082] In a scenario involving a large screen, candidate command intentions may include: zoom in, zoom out, increase volume, decrease volume, return to the main page, previous page, next page, etc.

[0083] S222: Intent recognition prompt construction;

[0084] Build an intent recognition prompt so that the large language model can match the most appropriate user intent based on the user's utterance.

[0085] Each intention has the following attributes:

[0086] id: uniquely corresponds to an intent;

[0087] name: Intent name;

[0088] Description: Provides a detailed description of the intent; used to improve the accuracy of intent recognition.

[0089] At the same time, the aim is to identify the content returned by the prompt constraint model, thereby reducing the number of tokens returned by the large language model and shortening the overall response time.

[0090] In one embodiment of the present invention, the constructed intent recognition prompt is:

[0091] #01 Based on the candidate intents I provide and the input phrase, identify the most appropriate intent corresponding to the input phrase.

[0092] #02 The candidate intent is in JSON format. The key in the JSON represents the ID corresponding to the intent, the name in the value represents the name of the intent, and the description represents the detailed meaning and related description of the intent.

[0093] #03 Candidate Intent: {

[0094] 0: {"name":"Other", "description":"Other intentions, such as introducing a project or product, or asking about case studies, showcasing awards, etc."},

[0095] 1: {"name":"Magnify","description":"Magnify the interface"},

[0096] 2: {"name":"Shrink","description":"Shrink the interface"},

[0097] 3: {"name":"Main Page","description":"Return to the main page / main interface, go back to the homepage."}, 4: ... ... ... ...

[0100] };

[0101] #04 Input query: {query};

[0102] #05 Please determine the intent that best matches the input utterance and return the corresponding intent ID. If no candidate intent can be matched, it is judged as an "other" intent.

[0103] #06 Please return the intent ID directly, such as 0; 3; etc. Please output the number of the ID directly, without returning any additional content.

[0104] S23: Project Recall;

[0105] S231: Hybrid retrieval for coarse recall;

[0106] By combining keyword matching with semantic vector matching, relevant items are quickly retrieved based on user input. This retrieval aims to retrieve as many relevant items as possible, while allowing for a certain degree of false positives.

[0107] The steps for keyword matching are as follows: as long as the keyword corresponding to the project appears in the user's wording, the project is considered to have been recalled.

[0108] The steps of semantic vector matching are as follows: The user's input text is vectorized; the vectorization model used here is the same as the one used in the data initialization construction in step S1. Based on vector similarity as the filtering criterion, a similarity threshold K is set (generally 0.8 or 0.9, which can be adjusted according to the actual model used and the data). When the similarity between the user's input text and the summary text of a certain project is greater than the threshold, that content is recalled.

[0109] The collection of items recalled using the two methods described above will be used as the output of the coarse recall in the hybrid retrieval.

[0110] S232: Large Language Model Selection;

[0111] Using a large language model, based on an understanding of the natural language context, semantic intent, and potential needs of user input, deep semantic analysis and filtering are performed on the candidate items output from the coarse recall stage of hybrid retrieval. The goal is to accurately identify one or more items that the user truly cares about.

[0112] At the same time, by returning the ID, the length of the model's output content is reduced as much as possible, thereby improving the overall response speed.

[0113] For the selection of a large language model, a general-purpose AI large language model can be used, such as Deepseek, Tongyi Qianwen, and Doubao.

[0114] Its advantage lies in the fact that large language models can flexibly understand the user's intent and filter projects, such as the user only wanting to see projects from last year, or projects in a certain region, or projects of a certain type, etc.

[0115] In one embodiment of the present invention, the prompt for large language model screening is as follows:

[0116] #01 Based on the background information and the input dialogue, please select the appropriate content from the candidate items and return it. The input dialogue is a user asking or introducing relevant content, which may be an introduction to a specific item or several items.

[0117] #02 Background Information: This year is 2025. Other information (such as basic company information, major data categories in the database, and other background information).

[0118] #03 The input script is a text, and the set of candidate items is a dictionary. The keys of the dictionary are numbers starting from 0 and incrementing, and the values ​​are the name of each item and related keywords. Keywords generally include departments, relevant years, relevant locations, technical terms, etc. (this can be adjusted according to the specific project).

[0119] #04 Input query: {query};

[0120] #05 Candidate item: {

[0121] 0: Product xxxx (2024, Shanghai, intelligent agent, medicine...),

[0122] 1: Case study (2024, Shanghai, industrial manufacturing, intelligent inspection...)

[0123] 2: Case study (2025, Hangzhou, culture and tourism, exhibition hall...),

[0124] 3: Awards (2023, National, Specialized, Refined and Innovative...), ... ...

[0126] };

[0127] #06 Based on the input query, return the keys of the most relevant batch of items. Note that this can be one or multiple keys. Please return the relevant keys directly in list format. For example: [0] or [1,3] or [0,2,3,4], etc. Please output directly without returning any extra content.

[0128] S24: Use a judgment device for information integration and relay;

[0129] In this method, overall intent recognition and item recall are performed simultaneously, and the judge collects the final outputs of the two steps. Since overall intent recognition only requires one large language model call, its response speed is usually faster than item recall. The judge performs a preliminary judgment on the result of overall intent recognition. If the intent is a large screen operation command, it does not need to wait for the result response from the item recall module and directly sends the large screen operation command to the large screen for corresponding operation. If the judge determines that the intent is "other" after preliminary judgment, it waits for the result return from the item recall module, performs simple processing, and then sends it to the large screen for single or multiple item displays.

[0130] The above method can minimize the overall system response time, as users often have high demands for response speed in large-screen presentation scenarios. Using this method, the response time for large-screen commands can be controlled to approximately 0.5 seconds, and the response time for project recalls can be controlled to approximately 1 second. This enables fast and accurate large-screen response and content display.

[0131] In a second aspect, the present invention provides an embodiment of a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0132] Thirdly, the present invention provides an embodiment of a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0133] Fourthly, the present invention provides a computer program product embodiment, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0134] This invention proposes a method, apparatus, storage medium, and program product for recalling content on large screens, the technical effects of which are as follows:

[0135] (1) A recall mechanism combining hybrid retrieval and big language model screening is used to ensure the comprehensiveness and coverage of the recall while using the understanding ability of the big language model to analyze and screen the candidate options, thereby improving the relevance and accuracy of the recalled content and flexibly responding to the complex query requirements put forward by users, such as fuzzy queries with time, region, category and other limiting conditions.

[0136] (2) A multi-process parallel architecture is adopted to process time-consuming modules in parallel. In the response process, the judge module prioritizes the judgment and processing based on the intent recognition result with the fastest response speed, and can execute the corresponding operation without waiting for all modules to complete, thus shortening the system response time. In addition, during the large language model call process, the prompt design is optimized to guide the model to output the most concise result, further improving the system response efficiency.

[0137] (3) The technical solution proposed in this invention has strong versatility and is applicable to content retrieval needs in various large-screen display scenarios; it supports the rapid import of new project data and allows for flexible configuration of keywords, summaries, and other information. At the same time, this method also has good scalability in supporting large-screen operation commands, and can flexibly configure command types according to actual needs to quickly adapt to changes in user needs.

Claims

1. A method for recalling content on a large screen, characterized in that, Includes the following steps: S1. Data initialization and construction: Organize and build a knowledge base for the large screen data to be displayed; the large screen data contains multiple items, each item is imported into the large screen database, and the keywords and summary summaries corresponding to each item are extracted in a targeted manner to generate semantic vector codes corresponding to the summary summaries; The large-screen database is a static database that is updated regularly. S2, large screen response and content recall, the specific steps include: S21. Organize the user's input and convert the voice input into text; S22. Overall Intent Recognition: Use a large language model to determine the user's linguistic intent and match it with candidate operations on the large screen; the candidate operations include controllable operations on the large screen and other operations. In the determination of the intent of the speech, the returned content and form of the large model are subject to simplification constraints; S23. Item Recall: By combining keyword matching with semantic vector matching, a coarse recall is performed in the large screen database. Then, a large language model is used to perform deep semantic analysis and screening on the candidate items output in the coarse recall stage of the coarse recall to achieve accurate recall. In the deep semantic analysis and filtering, the returned content and format of the large model are subject to simplification constraints. S24. Information Integration and Transfer: Steps S22 and S23 are executed simultaneously. The judge is used to coordinate and allocate information, skip unnecessary project recall steps, and collect the final output to reduce the response time of the large screen.

2. The method according to claim 1, characterized in that, The specific steps of step S1 include: S11. Entity extraction for projects: Use a large language model to extract keywords from the name and details of each project and store them in a regular database; S12. Summarize the projects: Use a large language model to summarize the project details of each project. Encode the generated summaries into semantic vectors using a vectorization model and store them in a vector database.

3. The method according to claim 1, characterized in that, The specific steps of step S22 include: S221. Candidate operation intent sorting: sort out the operation instructions allowed by the large screen in the current scenario to obtain candidate instruction intents, including regular operation intents and other intents. The other intents are used to determine whether the user is asking or querying a specific item or a group of items. S222. Construct an intent recognition prompt, allowing the large language model to match the most appropriate instruction intent based on the user's utterance.

4. The method according to claim 3, characterized in that, In step S222, the following attributes are defined for each intent by identifying the prompt: id: uniquely corresponds to an intent; name: Intent name; description: Provide a detailed description of the intent; Meanwhile, the constraint model only returns the intent ID.

5. The method according to claim 1, characterized in that, Step S23 includes: S231. Hybrid retrieval and coarse recall, the specific steps of which include: The steps for keyword matching are as follows: when the keyword corresponding to the project appears in the user's words, the project is recalled; The steps of semantic vector matching are as follows: using the vectorization model used to generate semantic vector encoding in step S1, the user's input speech is vectorized; a similarity threshold is set, and when the similarity between the user's input text and the summary text of a certain item is greater than the similarity threshold, the item is recalled.

6. The method according to claim 1, characterized in that, Step S23 includes: S232, Deep semantic analysis and screening, the specific steps of which include: Using a large language model, based on the understanding of the natural language context, semantic intent, and potential needs of user input, we perform deep semantic analysis and screening on the candidate items output in the coarse recall stage of hybrid retrieval.

7. The method according to claim 6, characterized in that, The prompt built for calling the large language model includes the following steps: Enter background information; For each candidate project, add the following attributes: ID uniquely corresponding to a candidate project, project name, and project keywords; Based on the input dialogue, return the IDs of one or more of the most relevant items, and do not return any other content.

8. The method according to claim 1, characterized in that, The specific steps of step S24 include: The judge makes a preliminary judgment on the results of the overall intent recognition step S22. If the judgment result is a regular operation intention on the large screen, the operation instruction on the large screen will be sent directly to the large screen without waiting for the result of the S23 project recall step to return. If the determination result indicates other intentions, wait for the return of the result from the S23 project recall step, perform simple sorting, and then send it to the large screen for display of one or more items.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-8.

11. A computer program product, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-8.

Images