Intelligent pushing method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to the medical field and the financial field, and discloses an intelligent pushing method and device, computer equipment and a storage medium, and the method comprises the steps: receiving a call voice signal, and converting the call voice signal into text data in a streaming recognition mode; performing multi-label intention recognition on the text data to obtain an intention recognition result, and generating a semantic triple based on the intention recognition result; judging that the client has a new retrieval intention according to the intention recognition result, and when judging that the client has the new retrieval intention, generating a structured query question based on the semantic triple; based on the structured query question, retrieving a preset knowledge base through a mixed retrieval mechanism to obtain at least one candidate knowledge fragment; performing knowledge fusion and noise filtering processing on the candidate knowledge fragment to obtain a target knowledge fragment; and carrying out multi-modal intelligent pushing on the target knowledge fragment. The knowledge service capability of the customer service industry is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an intelligent push method, device, computer equipment, and storage medium. Background Technology

[0002] Currently, most customer service knowledge base systems in the customer service industry adopt keyword matching or FAQ search models. However, this model has certain drawbacks. For example, in financial service scenarios, if a customer raises multi-dimensional or cross-domain questions during a call (such as simultaneously inquiring about the returns and claims process of financial products), customer service personnel often need to manually enter keywords to search, which can lead to response delays, incomplete information, and a decline in service experience. Similarly, in medical consultation scenarios, patients often inquire about disease interpretation, medication guidance, appointment booking, and medical insurance reimbursement. When patients raise combined needs (such as simultaneously inquiring about chronic disease medication plans and medical insurance reimbursement rates), customer service personnel also need to manually search for keywords. This not only affects consultation response efficiency but may also affect the accuracy and guidance of medical consultations due to untimely information matching.

[0003] Specifically, existing systems of this type generally suffer from the following core problems: First, the retrieval patterns are static. Most systems rely on fixed indexes or keyword matching, making it difficult to adapt to the semantic diversity and implicit intent in natural language. For example, in the financial field, problems such as "cannot find" or "searching incorrectly" are prone to occur; in the medical field, it is unable to accurately identify the implicit needs of patients (for example, a patient mentions "dizziness after taking medication," but is actually inquiring about drug side effects), thus leading to the matching of irrelevant knowledge content.

[0004] Secondly, there is a high response time. For example, customer service representatives in the financial and medical fields need to manually switch system interfaces and enter search terms during calls, which takes an average of 2-5 seconds. In financial scenarios, this affects the continuity of service; in medical scenarios, for emergency consultations (such as consultations on sudden drug reactions), delays may affect the timeliness of patients' handling of emergencies.

[0005] Third, there is a lack of contextual understanding. The system cannot understand the contextual intent based on real-time voice during a call, nor can it automatically generate query content. For example, in financial scenarios, this can cause knowledge delivery to lag behind customer needs; in medical scenarios, it is difficult to link the patient's step-by-step description of their condition (e.g., first describing symptoms, then supplementing medication information), requiring repeated searches and resulting in the knowledge being disjointed from the patient's consultation progress.

[0006] Fourth, the knowledge base is not updated in a timely manner. Updates to knowledge content rely on manual review and lack self-learning and intelligent update mechanisms. For example, in the financial sector, knowledge can easily become outdated after policy adjustments and product iterations; in the medical field, drug catalogs, treatment guidelines, and medical insurance policies are frequently updated, and new treatment standards or reimbursement policies cannot be synchronized to the knowledge base in a timely manner. This may lead to customer service providing outdated information or even misleading patients.

[0007] Therefore, how to improve the knowledge service capabilities of the customer service industry is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0008] This invention provides an intelligent push method, apparatus, computer device, and storage medium, aiming to improve customer service knowledge service capabilities.

[0009] In a first aspect, embodiments of the present invention provide an intelligent push method, including: The system receives customer call voice signals using speech recognition technology and converts the call voice signals into text data using streaming recognition. Multi-label intent recognition is performed on the text data to obtain the corresponding intent recognition results, and semantic triples are generated based on the intent recognition results; Based on the intent recognition result, it is determined whether the customer has a new search intent, and when a new search intent is determined, a structured query question is generated based on the semantic triple; Based on the structured query question, a hybrid retrieval mechanism is used to search the preset knowledge base to obtain at least one candidate knowledge fragment; The candidate knowledge fragments are subjected to knowledge fusion and noise filtering to obtain the target knowledge fragment; Multimodal intelligent push is performed on the target knowledge fragments.

[0010] Secondly, embodiments of the present invention provide an intelligent push device, comprising: The voice receiving unit is used to receive the customer's call voice signal through voice recognition technology, and to convert the call voice signal into text data using streaming recognition. An intent recognition unit is used to perform multi-label intent recognition on the text data, obtain the corresponding intent recognition result, and generate semantic triples based on the intent recognition result; The question generation unit is used to determine whether the customer has a new search intent based on the intent recognition result, and when it is determined that a new search intent exists, to generate a structured query question based on the semantic triple; The fragment retrieval unit is used to retrieve at least one candidate knowledge fragment from the preset knowledge base based on the structured query question through a hybrid retrieval mechanism. A fusion filtering unit is used to perform knowledge fusion and noise filtering on the candidate knowledge fragments to obtain the target knowledge fragment. The intelligent push unit is used to perform multimodal intelligent push of the target knowledge fragments.

[0011] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent push method as described in the first aspect.

[0012] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the intelligent push method as described in the first aspect.

[0013] This invention provides an intelligent push method, apparatus, computer device, and storage medium. The method includes: receiving a customer's call voice signal using speech recognition technology and converting the call voice signal into text data using streaming recognition; performing multi-tag intent recognition on the text data to obtain corresponding intent recognition results and generating semantic triples based on the intent recognition results; determining whether the customer has a new search intent based on the intent recognition results, and generating a structured query question based on the semantic triples when a new search intent is determined; searching a preset knowledge base based on the structured query question using a hybrid search mechanism to obtain at least one candidate knowledge fragment; performing knowledge fusion and noise filtering on the candidate knowledge fragment to obtain a target knowledge fragment; and performing multimodal intelligent push on the target knowledge fragment. This invention, through real-time reception of customer voice signals and conversion into text, can quickly capture customer intent. Simultaneously, it utilizes a hybrid retrieval mechanism to accurately search for relevant knowledge fragments from a pre-set knowledge base. After knowledge fusion and noise filtering, the target knowledge fragment is obtained, and finally, multimodal intelligent push is performed. This effectively solves the problems of static retrieval modes, high response latency, lack of contextual linkage, and untimely knowledge base updates in existing customer service knowledge base systems. Therefore, it significantly improves the knowledge service capabilities of the customer service industry, providing customers with a higher quality and more efficient service experience. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1This is a flowchart illustrating an intelligent push method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a sub-process of step S103 in an intelligent push method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a sub-process of step S104 in an intelligent push method provided in an embodiment of the present invention; Figure 4 A schematic diagram of a sub-process of step S105 in an intelligent push method provided in an embodiment of the present invention; Figure 5 A schematic block diagram of an intelligent push device provided in an embodiment of the present invention; Figure 6 This is a schematic block diagram of a first sub-section in an intelligent push device provided in an embodiment of the present invention; Figure 7 This is a schematic block diagram of a second sub-component in an intelligent push device provided by an embodiment of the present invention; Figure 8 This is a schematic block diagram of a third sub-component in an intelligent push device provided by an embodiment of the present invention; Figure 9 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0018] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0019] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0020] The intelligent push method provided in this invention can be applied in an application environment where a client and a server interact, wherein the client communicates with the server via a network. The server receives the client's call voice signal using speech recognition technology and converts the call voice signal into text data using streaming recognition; it performs multi-tag intent recognition on the text data to obtain the corresponding intent recognition result, and generates semantic triples based on the intent recognition result; it determines whether the client has a new search intent based on the intent recognition result, and if a new search intent is determined, it generates a structured query question based on the semantic triples; based on the structured query question, it searches a preset knowledge base using a hybrid search mechanism to obtain at least one candidate knowledge fragment; it performs knowledge fusion and noise filtering on the candidate knowledge fragment to obtain a target knowledge fragment; and it performs multimodal intelligent push on the target knowledge fragment. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0021] Please see below. Figure 1 This invention provides an intelligent push method, specifically including steps S101 to S106.

[0022] Step S101: Receive the customer's call voice signal through voice recognition technology, and convert the call voice signal into text data using streaming recognition. Step S102: Perform multi-label intent recognition on the text data to obtain the corresponding intent recognition result, and generate semantic triples based on the intent recognition result; Step S103: Determine whether the customer has a new search intent based on the intent recognition result, and when it is determined that a new search intent exists, generate a structured query question based on the semantic triple; Step S104: Based on the structured query question, a hybrid retrieval mechanism is used to retrieve the preset knowledge base to obtain at least one candidate knowledge fragment; Step S105: Perform knowledge fusion and noise filtering on the candidate knowledge fragments to obtain the target knowledge fragment; Step S106: Perform multimodal intelligent push on the target knowledge fragment.

[0023] In this embodiment, firstly, the customer's voice call signal is received using speech recognition technology, and then converted into text data in real time using streaming recognition to ensure the immediacy and continuity of information processing. Next, multi-label intent recognition is performed on the converted text data to accurately determine the customer's intent and generate semantic triples. Then, based on the intent recognition results, it is determined whether the customer has a new search intent. If so, a structured query question is automatically generated based on the semantic triples to ensure the accuracy and relevance of the query. Afterward, a hybrid retrieval mechanism is used to search a preset knowledge base, obtaining at least one candidate knowledge fragment by combining multiple retrieval algorithms. After obtaining the candidate knowledge fragment, further knowledge fusion and noise filtering are performed to integrate relevant information and remove redundant or irrelevant content to obtain the accurate target knowledge fragment. Finally, the target knowledge fragment is intelligently pushed in a multimodal manner.

[0024] This embodiment receives customer voice signals in real time and converts them into text, enabling rapid capture of customer intent. Simultaneously, it utilizes a hybrid retrieval mechanism to accurately search for relevant knowledge fragments from a preset knowledge base. After knowledge fusion and noise filtering, the target knowledge fragment is obtained, and finally, multimodal intelligent push is performed. This effectively solves the problems of static retrieval modes, high response latency, lack of contextual linkage, and untimely knowledge base updates in existing customer service knowledge base systems. As a result, it significantly improves the knowledge service capabilities of the customer service industry and provides customers with a better and more efficient service experience.

[0025] The intelligent push method provided in this embodiment is applicable to various customer service scenarios. For example, in the financial sector, it helps customer service personnel obtain timely information on financial products, providing accurate and detailed financial consulting services. When a customer inquires about wealth management products, it can quickly push information such as the product's returns and risk levels, improving service consistency and accuracy. In the medical field, it enables doctors or medical customer service personnel to quickly obtain treatment plans and medication information related to a patient's condition. It allows for rapid response to urgent inquiries; for example, in cases of sudden drug reactions, it can promptly push corresponding treatment measures and precautions, ensuring timely handling of emergencies. Simultaneously, it effectively avoids the problem of delayed or even incorrect information due to untimely knowledge base updates, bringing more efficient, intelligent, and accurate knowledge service support to customer service in the financial and medical fields.

[0026] In one embodiment, step S101 includes: An end-to-end Transformer architecture is used to perform streaming recognition on the call voice signal; By utilizing a speech signal framing mechanism and combining acoustic features, the semantics of the next frame of the call speech signal are predicted, so as to synchronously convert the call speech signal into text data.

[0027] This embodiment receives the customer's call voice signal in real time using speech recognition technology, and then performs streaming recognition using an end-to-end Transformer structure (such as the Conformer-CTC model), keeping the average latency within 200ms. It uses a speech signal framing mechanism combined with acoustic features to predict the semantics of the next frame, ensuring synchronization between speech recognition and text output. In traditional ASR systems, the real-time speech recognition module and intent recognition operate in separate steps, resulting in latency as semantic processing only begins after recognition is complete. This embodiment, however, achieves "listening and understanding simultaneously" through a frame-level triggering mechanism.

[0028] In practical applications, local semantics are output immediately when the speech recognition confidence exceeds a threshold (e.g., 0.92), and multi-frame predictions are fused using a sliding window strategy to dynamically update the intent. This mechanism reduces the knowledge retrieval response time from 2 seconds to 0.3 seconds, achieving millisecond-level triggering. This efficient conversion method reduces information processing time, effectively improves system response speed, avoids high response latency issues, and allows customer service representatives to obtain customer intent more promptly.

[0029] Furthermore, the streaming conversion method used in this embodiment can adapt to the continuous speech of customers, performing conversion continuously while the customer is speaking, rather than waiting for the customer to finish speaking a long passage before processing begins. This makes the system's understanding of customer intent more coherent and real-time. For example, in financial service scenarios, when a customer describes their financial needs in detail, the system can convert the speech simultaneously. Once a new search intent is identified, subsequent operations can be performed immediately, providing customer service personnel with timely and relevant knowledge fragments, improving the coherence and quality of service. At the same time, this speech-to-text method also helps to solve the problem of existing systems lacking contextual linkage. Because it can record the customer's speech content in real time, it can combine contextual information for more accurate intent recognition and querying during subsequent processing. For example, in medical consultation scenarios, when a patient describes their condition step by step, the system can obtain this information completely. When generating structured query questions and retrieving knowledge fragments, it fully considers the context, avoiding situations where knowledge delivery lags behind the patient's consultation progress, ensuring that the delivered knowledge closely matches the actual consultation situation.

[0030] In one embodiment, step S102 includes: The text data is subjected to multi-label intent recognition using a fine-tuned LLM model to obtain the intent recognition result; The contextual dialogue data corresponding to the intent recognition result is obtained, and the intent recognition result and the contextual dialogue data are input into the self-attention network, and the semantic triple is generated by the self-attention network.

[0031] This embodiment performs intent recognition and semantic layer parsing on the converted text. Specifically, it includes first-level intent classification, which uses a fine-tuned LLM (e.g., Qwen-14B-Chat) to perform multi-label intent recognition on the text and outputs intent label sets, such as {product consultation, claims inquiry, account security, payment anomalies, etc.}; and second-level semantic layer parsing, which inputs the intent results and contextual dialogue history into a self-attention network to generate semantic triples <topic, action, entity>, such as <claims, inquiry, status>.

[0032] In one embodiment, such as Figure 2 As shown, step S103 includes steps S201 to S203.

[0033] Step S201: Obtain the difference between the intent recognition result and the historical intent, and compare the difference with a preset difference threshold; Step S202: When the difference does not exceed the preset difference threshold, it is determined that there is no new search intent; Step S203: When the difference exceeds the preset difference threshold, it is determined that there is a new search intent. Then, the semantic triple is transformed into a structured query question using a domain template fusion mechanism and a semantic filling mechanism.

[0034] This embodiment first determines whether a new search intent exists by comparing the difference between the intent recognition result and historical intents. In practical applications, the difference can be calculated based on changes in the intent tag set and differences in semantic triples. For example, if a customer's previous intent was to inquire about products, but the current intent recognition result shows a claims inquiry, this significant change in intent will cause the difference to exceed a preset difference threshold, at which point the system determines that a new search intent exists.

[0035] After determining that a new search intent exists, a domain template fusion mechanism (Prompt template) and a semantic fill mechanism are used to transform semantic triples into structured query questions. For example, "A customer wants to check the claim status" is transformed into the question "Claim status query process" or "How to check the claim progress?", making the query more accurate and efficient. The domain template fusion mechanism can provide suitable query templates for semantic triples based on different business domains, such as finance, healthcare, and e-commerce. Template example: "Customer wants {action}{entity} → Generate question: 'How to operate {entity}{action}?'". For example, in the financial domain, for the semantic triple <financial product, consultation, return>, a template like "Query the {return} status of {financial product}" can be used. The semantic fill mechanism accurately fills the specific content of the semantic triple into the template to form a complete structured query question.

[0036] This method of identifying new search intents and generating structured query questions can promptly capture changes in customer needs, avoiding the use of static search patterns. It enables the system to dynamically search based on real-time customer intent, improving the targeting and effectiveness of the search. Simultaneously, by combining contextual information, such as historical intents and dialogue data, it can better understand the customer's true needs, further enhancing the system's intelligence and service quality. In actual customer service scenarios, it allows customer service personnel to quickly obtain knowledge fragments related to the customer's latest needs, providing more accurate and timely service. Combined with a large-scale model completion strategy to generate natural language queries, it can improve knowledge retrieval accuracy by approximately 40%.

[0037] In practical applications, an Encoder-Decoder structure can be used to generate structured query questions, with the input being semantic triples and intent context: ; Here, y represents the generated structured query, Encoder represents the encoder that encodes the input semantic triples, and Decoder represents the decoder that decodes the encoded information into a structured query. This structure can fully utilize the information in the semantic triples and contextual information to generate query statements that match the customer's intent. For example, in an e-commerce scenario, when the semantic triple is <product, inquiry, price> and the context indicates that the customer is interested in a specific brand, the Encoder-Decoder structure can generate a precise query statement like "Query {specific brand} {product} {price}". Topic, intent, and entity represent the topic, intent, and entity in the semantic triple, respectively.

[0038] In one embodiment, such as Figure 3As shown, step S104 includes steps S301 to S303.

[0039] Step S301: Use the Sentence-BERT model to perform vectorized embedding of the structured query question and the knowledge items in the preset knowledge base, and calculate the cosine similarity between the structured query question and the knowledge items based on the vectorized embedding results to obtain the similarity results; Step S302: Use the Elasticsearch inverted index structure to perform fast fuzzy matching between the structured query and the knowledge entries in the preset knowledge base to obtain the matching results; Step S303: Perform a weighted sorting of the similarity results and the matching results, and select the top N knowledge items as the candidate knowledge fragments based on the weighted sorting results; where N≥1.

[0040] This embodiment, after generating a structured query, employs a hybrid retrieval mechanism to search a pre-defined knowledge base for candidate knowledge fragments. Specifically, the Sentence-BERT model is first used to vectorize and embed the structured query and knowledge entries from the pre-defined knowledge base. The Sentence-BERT model can convert text into vector representations, thus mapping the structured query and knowledge entries to the same vector space. By calculating the cosine similarity of the vectorized embedding results, the semantic similarity between the query and the knowledge entries can be measured, yielding a similarity result. The closer the cosine similarity is to 1, the more semantically similar the two are.

[0041] Simultaneously, an Elasticsearch inverted index structure is used to perform fast fuzzy matching between structured query questions and knowledge entries in a pre-defined knowledge base. The Elasticsearch inverted index is a highly efficient search data structure that can quickly find knowledge entries that fuzzily match the query question, thus obtaining the matching results. This fuzzy matching mechanism can broaden the search scope and avoid missing relevant knowledge due to subtle differences in wording.

[0042] Next, the similarity and matching results are weighted and ranked. The specific weights can be assigned to these results based on the actual application scenario and requirements. For example, in scenarios with high semantic accuracy requirements, the weight of the similarity result can be appropriately increased; while in scenarios requiring broader coverage of relevant knowledge, the weight of the matching result can be increased. Based on the weighted ranking, the top N knowledge items are selected as candidate knowledge fragments, where N≥1. The specific value of N can be adjusted according to system performance and actual needs. In practical applications, BM25 and semantic relevance scores can be combined for weighted ranking, and the Top-3 highly relevant knowledge fragments can be output within 20ms.

[0043] The hybrid retrieval mechanism provided in this embodiment combines the advantages of semantic similarity matching and fuzzy matching. It ensures that the retrieved knowledge is highly semantically relevant to the query, while also expanding the retrieval scope and avoiding the omission of potentially useful knowledge. In practical applications, this retrieval method improves the accuracy and comprehensiveness of knowledge retrieval, providing richer and more relevant candidate knowledge fragments for subsequent knowledge fusion and noise filtering. For example, in a medical consultation scenario, when a customer inquires about treatment methods for a certain disease, the hybrid retrieval mechanism can retrieve relevant medical literature, clinical cases, and other knowledge entries from a pre-set knowledge base, providing customer service personnel with more comprehensive information to help them better answer customer questions.

[0044] In one embodiment, such as Figure 4 As shown, step S105 includes steps S401 to S403.

[0045] Step S401: Calculate the information gain of each candidate knowledge fragment to the explanatory power of the new retrieval intent, and filter the candidate knowledge fragments according to the information gain to obtain intermediate knowledge fragments; Step S402: Use a hierarchical clustering algorithm to perform semantic clustering and fusion on the intermediate knowledge fragments to obtain a unified response template; Step S403: Perform contextual verification on the unified response template using a large language model, and set the target knowledge fragment based on the result of the contextual verification.

[0046] For the obtained candidate knowledge fragments, this embodiment performs knowledge fusion and noise filtering to obtain the most accurate and effective target knowledge fragments that best match the customer's new search intent. First, the information gain of each candidate knowledge fragment's explanatory power for the new search intent is calculated. Information gain reflects the degree to which a candidate knowledge fragment contributes to explaining the new search intent; by calculating information gain, the importance of each candidate knowledge fragment can be measured. Then, based on the information gain, the candidate knowledge fragments are filtered by information gain ratio, removing those with low information gain ratios and little contribution to explaining the new search intent, thus obtaining intermediate knowledge fragments. This step effectively reduces noise information and improves the quality of the knowledge fragments.

[0047] Next, hierarchical clustering algorithms are used to perform semantic clustering and fusion on the intermediate knowledge fragments. Hierarchical clustering algorithms can classify knowledge fragments into different categories based on their semantic similarity and merge similar knowledge fragments to obtain a unified response template. In this process, similar knowledge fragments are merged together, eliminating redundant information and making the knowledge more concise and organized. For example, in financial service scenarios, if multiple candidate knowledge fragments all relate to the calculation method of a certain financial product's returns, clustering fusion can integrate these knowledge fragments into a unified response template for calculating the returns of that financial product.

[0048] Finally, the unified response template is validated using a large language model. The large language model possesses powerful language understanding and reasoning capabilities, enabling it to determine the accuracy and appropriateness of the unified response template based on contextual information. Target knowledge fragments are set based on the contextual validation results. If the unified response template passes validation, it is used as the target knowledge fragment; if problems exist, the unified response template needs to be adjusted or regenerated. In practical applications, this contextual validation mechanism ensures that the target knowledge fragment closely matches the customer's actual needs and contextual information, improving the accuracy and reliability of knowledge delivery. For example, in an e-commerce scenario, when a customer inquires about the after-sales service policy of a product, the unified response template obtained through clustering and fusion needs to be validated in conjunction with the customer's previous dialogue content to ensure that the pushed after-sales service policy matches the product and specific circumstances inquired about by the customer.

[0049] Through the above knowledge fusion and noise filtering process, the most valuable and accurate target knowledge fragments can be selected from the candidate knowledge fragments, providing a high-quality knowledge foundation for subsequent multimodal intelligent push, thereby better meeting customer needs and improving the quality and efficiency of customer service.

[0050] In practical applications, the information gain ratio can be calculated using the following formula: ; Where IGR represents the information gain ratio, r i Let H(C) represent the i-th candidate knowledge fragment, C be the intent tag set, and H(C) be the entropy value.

[0051] For example, candidate knowledge fragments with an IGR below a threshold (0.15) are considered noise and discarded. The remaining candidate knowledge fragments are then clustered by BERT to obtain a unified response template, which can effectively reduce duplicate pushes and ensure the uniqueness and accuracy of the answer.

[0052] In one embodiment, step S106 includes: The target knowledge fragment is displayed in a parallel mode using multiple modalities; wherein the content of the multiple modal display includes a text summary, key flowcharts, and related connections of the target knowledge fragment. The system obtains feedback on the target knowledge fragment based on an interactive feedback mechanism, and then corrects the intelligent push based on the feedback.

[0053] This embodiment, after obtaining the target knowledge fragment, employs a parallel mode for multimodal display. This display method fully leverages the advantages of different modal information, providing users with a richer and more intuitive knowledge presentation. Here, the multimodal display includes a text summary of the target knowledge fragment, key flowcharts, and related links. The text summary accurately extracts the core content of the target knowledge fragment, allowing users to quickly understand key information; the key flowcharts visually represent the logical structure and process steps of the knowledge, helping users to more clearly understand complex knowledge systems; and the related links provide links to other materials or resources closely related to the target knowledge, facilitating further in-depth exploration of related knowledge.

[0054] Furthermore, this embodiment also obtains feedback on the target knowledge fragment based on an interactive feedback mechanism. This feedback mechanism allows users to evaluate the displayed content, ask questions, or provide suggestions. The system collects this feedback information and corrects the intelligent push based on it. For example, if a user reports that the text summary is not accurate enough, the system will refine and optimize the summary; if a user indicates that the key flowchart is not clear enough, the system will adjust and improve the flowchart. By continuously correcting based on user feedback, the intelligent push can better meet users' personalized needs and improve user satisfaction with the knowledge push service.

[0055] Furthermore, this embodiment, by combining multimodal display and interactive feedback mechanisms, can also promote deep interaction between users and the system. While viewing multimodal content, users can express their thoughts and needs at any time, and the system can respond and adjust accordingly, forming a virtuous cycle of interaction. This interaction not only helps improve the quality of knowledge delivery but also enhances users' trust and reliance on the system. In various application scenarios, such as online education, intelligent customer service, and enterprise knowledge management, this multimodal intelligent delivery and interactive feedback mechanism can play a crucial role in providing users with more efficient and accurate knowledge services, thereby improving the overall service experience.

[0056] In a specific embodiment, a self-learning optimization mechanism is adopted, which updates the weight parameters by introducing reinforcement learning feedback. For example, when an employee clicks "useful" or "irrelevant" on the pushed knowledge, the weight parameters are updated according to the following formula: ; Among them, wt w represents the weight parameters before the update. t+1 This represents the updated weight parameters, where η is the learning rate, reward is the reward value, and expected is the expected value. Reward can be specifically defined as user confirmation rate or customer call satisfaction. This mechanism enables the knowledge base retrieval model to continuously optimize with actual use, achieving intelligent evolution of knowledge recommendation.

[0057] In practical applications, this embodiment adopts a layered deployment architecture, deploying the real-time speech recognition (ASR) module and semantic understanding (SLU) module on edge nodes (Edge Servers) to reduce response latency by shortening the data transmission path. Simultaneously, the knowledge retrieval and fusion module is deployed in the cloud, fully utilizing cloud computing power to support complex calculations. At the communication level, WebSocket channels and gRPC streaming protocols are used to achieve millisecond-level data transmission, ensuring real-time interaction requirements. In terms of data processing, caching and pre-fetching mechanisms are introduced. The system can establish a cache pool based on customer historical intent and preload high-frequency knowledge items, and can predict potential customer mentions and retrieve relevant knowledge in advance, further improving response speed. Regarding security and compliance, the knowledge base content undergoes anonymization and version control to ensure that fields involving sensitive information such as insurance policies and accounts are automatically masked, complying with industry data security standards. Furthermore, this embodiment uses asynchronous task queues (such as Celery) to build a parallel processing engine, enabling core processes such as speech recognition, semantic analysis, retrieval, and push to be executed in parallel, significantly improving overall processing efficiency.

[0058] Overall, this embodiment integrates AIGC and semantic retrieval technologies to build an intelligent knowledge push system that is real-time, accurate, and self-learning. It realizes the intelligent upgrade of customer service knowledge services from "passive retrieval" to "proactive understanding," providing innovative technical approaches and considerable commercial value for service industries such as finance.

[0059] Specifically, compared with the prior art, this embodiment has the following advantages: (1) Improved response speed: The automatic push is completed within 0.5 seconds, which is reduced from 3 seconds for traditional manual retrieval; (2) Improved accuracy of knowledge retrieval: Combining semantic retrieval with question generation significantly reduces the occurrence of "answers that are not what are asked"; (3) Significantly reduced workload for agents: Employees do not need to manually enter search keywords, and the system can automatically identify customer intent and push relevant knowledge; (4) Enhanced cross-business line capabilities: With the help of a dynamic knowledge fusion mechanism, unified management of knowledge in multiple fields such as insurance, banking, and credit cards has been achieved; (5) Improved customer experience: Achieve "one call, enjoy multiple services", which strongly supports the "three savings" strategy (saving time, saving worry, saving effort).

[0060] Figure 5 This is a schematic block diagram of an intelligent push device 500 provided in an embodiment of the present invention. The device 500 includes: The voice receiving unit 501 is used to receive the customer's call voice signal through voice recognition technology, and to convert the call voice signal into text data using a streaming recognition method. The intent recognition unit 502 is used to perform multi-label intent recognition on the text data, obtain the corresponding intent recognition result, and generate semantic triples based on the intent recognition result; The question generation unit 503 is used to determine whether the customer has a new search intent based on the intent recognition result, and when it is determined that a new search intent exists, to generate a structured query question based on the semantic triple; The fragment retrieval unit 504 is used to retrieve at least one candidate knowledge fragment from the preset knowledge base based on the structured query question through a hybrid retrieval mechanism. The fusion filtering unit 505 is used to perform knowledge fusion and noise filtering on the candidate knowledge fragments to obtain the target knowledge fragment. The intelligent push unit 506 is used to perform multimodal intelligent push on the target knowledge fragment.

[0061] In this embodiment, firstly, the customer's voice call signal is received using speech recognition technology, and then converted into text data in real time using streaming recognition to ensure the immediacy and continuity of information processing. Next, multi-label intent recognition is performed on the converted text data to accurately determine the customer's intent and generate semantic triples. Then, based on the intent recognition results, it is determined whether the customer has a new search intent. If so, a structured query question is automatically generated based on the semantic triples to ensure the accuracy and relevance of the query. Afterward, a hybrid retrieval mechanism is used to search a preset knowledge base, obtaining at least one candidate knowledge fragment by combining multiple retrieval algorithms. After obtaining the candidate knowledge fragment, further knowledge fusion and noise filtering are performed to integrate relevant information and remove redundant or irrelevant content to obtain the accurate target knowledge fragment. Finally, the target knowledge fragment is intelligently pushed in a multimodal manner.

[0062] This embodiment receives customer voice signals in real time and converts them into text, enabling rapid capture of customer intent. Simultaneously, it utilizes a hybrid retrieval mechanism to accurately search for relevant knowledge fragments from a preset knowledge base. After knowledge fusion and noise filtering, the target knowledge fragment is obtained, and finally, multimodal intelligent push is performed. This effectively solves the problems of static retrieval modes, high response latency, lack of contextual linkage, and untimely knowledge base updates in existing customer service knowledge base systems. As a result, it significantly improves the knowledge service capabilities of the customer service industry and provides customers with a better and more efficient service experience.

[0063] The intelligent push method provided in this embodiment is applicable to various customer service scenarios. For example, in the financial sector, it helps customer service personnel obtain timely information on financial products, providing accurate and detailed financial consulting services. When a customer inquires about wealth management products, it can quickly push information such as the product's returns and risk levels, improving service consistency and accuracy. In the medical field, it enables doctors or medical customer service personnel to quickly obtain treatment plans and medication information related to a patient's condition. It allows for rapid response to urgent inquiries; for example, in cases of sudden drug reactions, it can promptly push corresponding treatment measures and precautions, ensuring timely handling of emergencies. Simultaneously, it effectively avoids the problem of delayed or even incorrect information due to untimely knowledge base updates, bringing more efficient, intelligent, and accurate knowledge service support to customer service in the financial and medical fields.

[0064] In one embodiment, the voice receiving unit 501 includes: A signal recognition unit is used to perform streaming recognition of the call voice signal using an end-to-end Transformer structure; The semantic prediction unit is used to predict the semantics of the next frame of the call voice signal by using the speech signal framing mechanism and combining acoustic features, so as to synchronously convert the call voice signal into text data.

[0065] This embodiment receives the customer's call voice signal in real time using speech recognition technology, and then performs streaming recognition using an end-to-end Transformer structure (such as the Conformer-CTC model), keeping the average latency within 200ms. It uses a speech signal framing mechanism combined with acoustic features to predict the semantics of the next frame, ensuring synchronization between speech recognition and text output. In traditional ASR systems, the real-time speech recognition module and intent recognition operate in separate steps, resulting in latency as semantic processing only begins after recognition is complete. This embodiment, however, achieves "listening and understanding simultaneously" through a frame-level triggering mechanism.

[0066] In practical applications, local semantics are output immediately when the speech recognition confidence exceeds a threshold (e.g., 0.92), and multi-frame predictions are fused using a sliding window strategy to dynamically update the intent. This mechanism reduces the knowledge retrieval response time from 2 seconds to 0.3 seconds, achieving millisecond-level triggering. This efficient conversion method reduces information processing time, effectively improves system response speed, avoids high response latency issues, and allows customer service representatives to obtain customer intent more promptly.

[0067] Furthermore, the streaming conversion method used in this embodiment can adapt to the continuous speech of customers, performing conversion continuously while the customer is speaking, rather than waiting for the customer to finish speaking a long passage before processing begins. This makes the system's understanding of customer intent more coherent and real-time. For example, in financial service scenarios, when a customer describes their financial needs in detail, the system can convert the speech simultaneously. Once a new search intent is identified, subsequent operations can be performed immediately, providing customer service personnel with timely and relevant knowledge fragments, improving the coherence and quality of service. At the same time, this speech-to-text method also helps to solve the problem of existing systems lacking contextual linkage. Because it can record the customer's speech content in real time, it can combine contextual information for more accurate intent recognition and querying during subsequent processing. For example, in medical consultation scenarios, when a patient describes their condition step by step, the system can obtain this information completely. When generating structured query questions and retrieving knowledge fragments, it fully considers the context, avoiding situations where knowledge delivery lags behind the patient's consultation progress, ensuring that the delivered knowledge closely matches the actual consultation situation.

[0068] In one embodiment, the intent recognition unit 502 includes: A multi-label recognition unit is used to perform multi-label intent recognition on the text data using a fine-tuned LLM model to obtain the intent recognition result; The triple generation unit is used to obtain the contextual dialogue data corresponding to the intent recognition result, input the intent recognition result and the contextual dialogue data into the self-attention network, and generate the semantic triple by the self-attention network.

[0069] This embodiment performs intent recognition and semantic layer parsing on the converted text. Specifically, it includes first-level intent classification, which uses a fine-tuned LLM (e.g., Qwen-14B-Chat) to perform multi-label intent recognition on the text and outputs intent label sets, such as {product consultation, claims inquiry, account security, payment anomalies, etc.}; and second-level semantic layer parsing, which inputs the intent results and contextual dialogue history into a self-attention network to generate semantic triples <topic, action, entity>, such as <claims, inquiry, status>.

[0070] In one embodiment, such as Figure 6 As shown, the question generation unit 503 includes: The difference comparison unit 601 is used to obtain the difference between the intent recognition result and the historical intent, and compare the difference with a preset difference threshold. The first determination unit 602 is used to determine that there is no new search intent when the difference does not exceed a preset difference threshold. The second determination unit 603 is used to determine that there is a new search intent when the difference exceeds a preset difference threshold, and then use a domain template fusion mechanism and a semantic filling mechanism to convert the semantic triple into a structured query question.

[0071] This embodiment first determines whether a new search intent exists by comparing the difference between the intent recognition result and historical intents. In practical applications, the difference can be calculated based on changes in the intent tag set and differences in semantic triples. For example, if a customer's previous intent was to inquire about products, but the current intent recognition result shows a claims inquiry, this significant change in intent will cause the difference to exceed a preset difference threshold, at which point the system determines that a new search intent exists.

[0072] After determining that a new search intent exists, a domain template fusion mechanism (Prompt template) and a semantic fill mechanism are used to transform semantic triples into structured query questions. For example, "A customer wants to check the claim status" is transformed into the question "Claim status query process" or "How to check the claim progress?", making the query more accurate and efficient. The domain template fusion mechanism can provide suitable query templates for semantic triples based on different business domains, such as finance, healthcare, and e-commerce. Template example: "Customer wants {action}{entity} → Generate question: 'How to operate {entity}{action}?'". For example, in the financial domain, for the semantic triple <financial product, consultation, return>, a template like "Query the {return} status of {financial product}" can be used. The semantic fill mechanism accurately fills the specific content of the semantic triple into the template to form a complete structured query question.

[0073] This method of identifying new search intents and generating structured query questions can promptly capture changes in customer needs, avoiding the use of static search patterns. It enables the system to dynamically search based on real-time customer intent, improving the targeting and effectiveness of the search. Simultaneously, by combining contextual information, such as historical intents and dialogue data, it can better understand the customer's true needs, further enhancing the system's intelligence and service quality. In actual customer service scenarios, it allows customer service personnel to quickly obtain knowledge fragments related to the customer's latest needs, providing more accurate and timely service. Combined with a large-scale model completion strategy to generate natural language queries, it can improve knowledge retrieval accuracy by approximately 40%.

[0074] In practical applications, an Encoder-Decoder structure can be used to generate structured query questions, with the input being semantic triples and intent context: ; Here, y represents the generated structured query, Encoder represents the encoder that encodes the input semantic triples, and Decoder represents the decoder that decodes the encoded information into a structured query. This structure can fully utilize the information in the semantic triples and contextual information to generate query statements that match the customer's intent. For example, in an e-commerce scenario, when the semantic triple is <product, inquiry, price> and the context indicates that the customer is interested in a specific brand, the Encoder-Decoder structure can generate a precise query statement like "Query {specific brand} {product} {price}". Topic, intent, and entity represent the topic, intent, and entity in the semantic triple, respectively.

[0075] In one embodiment, such as Figure 7 As shown, the fragment retrieval unit 504 includes: The similarity calculation unit 701 is used to perform vectorized embedding of the structured query question and the knowledge items in the preset knowledge base using the Sentence-BERT model, and calculate the cosine similarity between the structured query question and the knowledge items based on the vectorized embedding result to obtain the similarity result; The fuzzy matching unit 702 is used to perform fast fuzzy matching between the structured query question and knowledge items in the preset knowledge base using the Elasticsearch inverted index structure to obtain the matching result; The weighted selection unit 703 is used to perform weighted sorting of the similarity results and the matching results, and select the top N knowledge items as the candidate knowledge fragments according to the weighted sorting results; where N≥1.

[0076] This embodiment, after generating a structured query, employs a hybrid retrieval mechanism to search a pre-defined knowledge base for candidate knowledge fragments. Specifically, the Sentence-BERT model is first used to vectorize and embed the structured query and knowledge entries from the pre-defined knowledge base. The Sentence-BERT model can convert text into vector representations, thus mapping the structured query and knowledge entries to the same vector space. By calculating the cosine similarity of the vectorized embedding results, the semantic similarity between the query and the knowledge entries can be measured, yielding a similarity result. The closer the cosine similarity is to 1, the more semantically similar the two are.

[0077] Simultaneously, an Elasticsearch inverted index structure is used to perform fast fuzzy matching between structured query questions and knowledge entries in a pre-defined knowledge base. The Elasticsearch inverted index is a highly efficient search data structure that can quickly find knowledge entries that fuzzily match the query question, thus obtaining the matching results. This fuzzy matching mechanism can broaden the search scope and avoid missing relevant knowledge due to subtle differences in wording.

[0078] Next, the similarity and matching results are weighted and ranked. The specific weights can be assigned to these results based on the actual application scenario and requirements. For example, in scenarios with high semantic accuracy requirements, the weight of the similarity result can be appropriately increased; while in scenarios requiring broader coverage of relevant knowledge, the weight of the matching result can be increased. Based on the weighted ranking, the top N knowledge items are selected as candidate knowledge fragments, where N≥1. The specific value of N can be adjusted according to system performance and actual needs. In practical applications, BM25 and semantic relevance scores can be combined for weighted ranking, and the Top-3 highly relevant knowledge fragments can be output within 20ms.

[0079] The hybrid retrieval mechanism provided in this embodiment combines the advantages of semantic similarity matching and fuzzy matching. It ensures that the retrieved knowledge is highly semantically relevant to the query, while also expanding the retrieval scope and avoiding the omission of potentially useful knowledge. In practical applications, this retrieval method improves the accuracy and comprehensiveness of knowledge retrieval, providing richer and more relevant candidate knowledge fragments for subsequent knowledge fusion and noise filtering. For example, in a medical consultation scenario, when a customer inquires about treatment methods for a certain disease, the hybrid retrieval mechanism can retrieve relevant medical literature, clinical cases, and other knowledge entries from a pre-set knowledge base, providing customer service personnel with more comprehensive information to help them better answer customer questions.

[0080] In one embodiment, such as Figure 8 As shown, the fusion filtering unit 505 includes: The gain filtering unit 801 is used to calculate the information gain of each candidate knowledge fragment to the explanatory power of the new retrieval intent, and to filter the candidate knowledge fragments according to the information gain to obtain intermediate knowledge fragments. Clustering fusion unit 802 is used to perform semantic clustering fusion on the intermediate knowledge fragments using a hierarchical clustering algorithm to obtain a unified response template; The verification setting unit 803 is used to perform context verification on the unified response template through a large language model, and set the target knowledge fragment according to the result of the context verification.

[0081] For the obtained candidate knowledge fragments, this embodiment performs knowledge fusion and noise filtering to obtain the most accurate and effective target knowledge fragments that best match the customer's new search intent. First, the information gain of each candidate knowledge fragment's explanatory power for the new search intent is calculated. Information gain reflects the degree to which a candidate knowledge fragment contributes to explaining the new search intent; by calculating information gain, the importance of each candidate knowledge fragment can be measured. Then, based on the information gain, the candidate knowledge fragments are filtered by information gain ratio, removing those with low information gain ratios and little contribution to explaining the new search intent, thus obtaining intermediate knowledge fragments. This step effectively reduces noise information and improves the quality of the knowledge fragments.

[0082] Next, hierarchical clustering algorithms are used to perform semantic clustering and fusion on the intermediate knowledge fragments. Hierarchical clustering algorithms can classify knowledge fragments into different categories based on their semantic similarity and merge similar knowledge fragments to obtain a unified response template. In this process, similar knowledge fragments are merged together, eliminating redundant information and making the knowledge more concise and organized. For example, in financial service scenarios, if multiple candidate knowledge fragments all relate to the calculation method of a certain financial product's returns, clustering fusion can integrate these knowledge fragments into a unified response template for calculating the returns of that financial product.

[0083] Finally, the unified response template is validated using a large language model. The large language model possesses powerful language understanding and reasoning capabilities, enabling it to determine the accuracy and appropriateness of the unified response template based on contextual information. Target knowledge fragments are set based on the contextual validation results. If the unified response template passes validation, it is used as the target knowledge fragment; if problems exist, the unified response template needs to be adjusted or regenerated. In practical applications, this contextual validation mechanism ensures that the target knowledge fragment closely matches the customer's actual needs and contextual information, improving the accuracy and reliability of knowledge delivery. For example, in an e-commerce scenario, when a customer inquires about the after-sales service policy of a product, the unified response template obtained through clustering and fusion needs to be validated in conjunction with the customer's previous dialogue content to ensure that the pushed after-sales service policy matches the product and specific circumstances inquired about by the customer.

[0084] Through the above knowledge fusion and noise filtering process, the most valuable and accurate target knowledge fragments can be selected from the candidate knowledge fragments, providing a high-quality knowledge foundation for subsequent multimodal intelligent push, thereby better meeting customer needs and improving the quality and efficiency of customer service.

[0085] In practical applications, the information gain ratio can be calculated using the following formula: ; Where IGR represents the information gain ratio, r i Let H(C) represent the i-th candidate knowledge fragment, C be the intent tag set, and H(C) be the entropy value.

[0086] For example, candidate knowledge fragments with an IGR below a threshold (0.15) are considered noise and discarded. The remaining candidate knowledge fragments are then clustered by BERT to obtain a unified response template, which can effectively reduce duplicate pushes and ensure the uniqueness and accuracy of the answer.

[0087] In one embodiment, the intelligent push unit 506 includes: A multimodal display unit is used to display the target knowledge fragment in a parallel mode in a multimodal manner; wherein, the content of the multimodal display includes a text summary, key flowcharts, and related connections of the target knowledge fragment; The feedback correction unit is used to obtain feedback on the target knowledge fragment based on an interactive feedback mechanism, and to correct the intelligent push based on the feedback.

[0088] This embodiment, after obtaining the target knowledge fragment, employs a parallel mode for multimodal display. This display method fully leverages the advantages of different modal information, providing users with a richer and more intuitive knowledge presentation. Here, the multimodal display includes a text summary of the target knowledge fragment, key flowcharts, and related links. The text summary accurately extracts the core content of the target knowledge fragment, allowing users to quickly understand key information; the key flowcharts visually represent the logical structure and process steps of the knowledge, helping users to more clearly understand complex knowledge systems; and the related links provide links to other materials or resources closely related to the target knowledge, facilitating further in-depth exploration of related knowledge.

[0089] Furthermore, this embodiment also obtains feedback on the target knowledge fragment based on an interactive feedback mechanism. This feedback mechanism allows users to evaluate the displayed content, ask questions, or provide suggestions. The system collects this feedback information and corrects the intelligent push based on it. For example, if a user reports that the text summary is not accurate enough, the system will refine and optimize the summary; if a user indicates that the key flowchart is not clear enough, the system will adjust and improve the flowchart. By continuously correcting based on user feedback, the intelligent push can better meet users' personalized needs and improve user satisfaction with the knowledge push service.

[0090] Furthermore, this embodiment, by combining multimodal display and interactive feedback mechanisms, can also promote deep interaction between users and the system. While viewing multimodal content, users can express their thoughts and needs at any time, and the system can respond and adjust accordingly, forming a virtuous cycle of interaction. This interaction not only helps improve the quality of knowledge delivery but also enhances users' trust and reliance on the system. In various application scenarios, such as online education, intelligent customer service, and enterprise knowledge management, this multimodal intelligent delivery and interactive feedback mechanism can play a crucial role in providing users with more efficient and accurate knowledge services, thereby improving the overall service experience.

[0091] In a specific embodiment, a self-learning optimization mechanism is adopted, which updates the weight parameters by introducing reinforcement learning feedback. For example, when an employee clicks "useful" or "irrelevant" on the pushed knowledge, the weight parameters are updated according to the following formula: ; Among them, w t w represents the weight parameters before the update. t+1 This represents the updated weight parameters, where η is the learning rate, reward is the reward value, and expected is the expected value. Reward can be specifically defined as user confirmation rate or customer call satisfaction. This mechanism enables the knowledge base retrieval model to continuously optimize with actual use, achieving intelligent evolution of knowledge recommendation.

[0092] In practical applications, this embodiment adopts a layered deployment architecture, deploying the real-time speech recognition (ASR) module and semantic understanding (SLU) module on edge nodes (Edge Servers) to reduce response latency by shortening the data transmission path. Simultaneously, the knowledge retrieval and fusion module is deployed in the cloud, fully utilizing cloud computing power to support complex calculations. At the communication level, WebSocket channels and gRPC streaming protocols are used to achieve millisecond-level data transmission, ensuring real-time interaction requirements. In terms of data processing, caching and pre-fetching mechanisms are introduced. The system can establish a cache pool based on customer historical intent and preload high-frequency knowledge items, and can predict potential customer mentions and retrieve relevant knowledge in advance, further improving response speed. Regarding security and compliance, the knowledge base content undergoes anonymization and version control to ensure that fields involving sensitive information such as insurance policies and accounts are automatically masked, complying with industry data security standards. Furthermore, this embodiment uses asynchronous task queues (such as Celery) to build a parallel processing engine, enabling core processes such as speech recognition, semantic analysis, retrieval, and push to be executed in parallel, significantly improving overall processing efficiency.

[0093] Overall, this embodiment integrates AIGC and semantic retrieval technologies to build an intelligent knowledge push system that is real-time, accurate, and self-learning. It realizes the intelligent upgrade of customer service knowledge services from "passive retrieval" to "proactive understanding," providing innovative technical approaches and considerable commercial value for service industries such as finance.

[0094] Specifically, compared with the prior art, this embodiment has the following advantages: (1) Improved response speed: The automatic push is completed within 0.5 seconds, which is reduced from 3 seconds for traditional manual retrieval; (2) Improved accuracy of knowledge retrieval: Combining semantic retrieval with question generation significantly reduces the occurrence of "answers that are not what are asked"; (3) Significantly reduced workload for agents: Employees do not need to manually enter search keywords, and the system can automatically identify customer intent and push relevant knowledge; (4) Enhanced cross-business line capabilities: With the help of a dynamic knowledge fusion mechanism, unified management of knowledge in multiple fields such as insurance, banking, and credit cards has been achieved; (5) Improved customer experience: Achieve "one call, enjoy multiple services", which strongly supports the "three savings" strategy (saving time, saving worry, saving effort).

[0095] Please see Figure 9 , Figure 9 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. The computer device is equipped with both wireless and wired communication capabilities.

[0096] The computer device includes a processor 902, a memory, and a network interface 905 connected via a system bus 901. The memory may include a non-volatile storage medium 903 and internal memory 904.

[0097] The non-volatile storage medium 903 can store an operating system 9031 and a computer program 9032. When the computer program 9032 is executed, it causes the processor 902 to execute a smart push method.

[0098] The processor 902 provides computing and control capabilities to support the operation of the entire computer device.

[0099] The internal memory 904 provides an environment for the execution of the computer program 9032 in the non-volatile storage medium 903. When the computer program 9032 is executed by the processor 902, the processor 902 can execute an intelligent push method.

[0100] This network interface 905 is used for network communication with other devices. Those skilled in the art will understand that... Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0101] The processor 902 is used to run a computer program 9032 stored in a memory to implement any embodiment of the above-described intelligent push method.

[0102] It should be understood that, in this embodiment of the invention, the processor 902 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0103] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: The system receives customer call voice signals using speech recognition technology and converts the call voice signals into text data using streaming recognition. Multi-label intent recognition is performed on the text data to obtain the corresponding intent recognition results, and semantic triples are generated based on the intent recognition results; Based on the intent recognition result, it is determined whether the customer has a new search intent, and when a new search intent is determined, a structured query question is generated based on the semantic triple; Based on the structured query question, a hybrid retrieval mechanism is used to search the preset knowledge base to obtain at least one candidate knowledge fragment; The candidate knowledge fragments are subjected to knowledge fusion and noise filtering to obtain the target knowledge fragment; Multimodal intelligent push is performed on the target knowledge fragments.

[0104] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0105] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: The system receives customer call voice signals using speech recognition technology and converts the call voice signals into text data using streaming recognition. Multi-label intent recognition is performed on the text data to obtain the corresponding intent recognition results, and semantic triples are generated based on the intent recognition results; Based on the intent recognition result, it is determined whether the customer has a new search intent, and when a new search intent is determined, a structured query question is generated based on the semantic triple; Based on the structured query question, a hybrid retrieval mechanism is used to search the preset knowledge base to obtain at least one candidate knowledge fragment; The candidate knowledge fragments are subjected to knowledge fusion and noise filtering to obtain the target knowledge fragment; Multimodal intelligent push is performed on the target knowledge fragments.

[0106] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0107] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0108] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0109] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An intelligent push method, characterized in that, include: The system receives customer call voice signals using speech recognition technology and converts the call voice signals into text data using streaming recognition. Multi-label intent recognition is performed on the text data to obtain the corresponding intent recognition results, and semantic triples are generated based on the intent recognition results; Based on the intent recognition result, it is determined whether the customer has a new search intent, and when a new search intent is determined, a structured query question is generated based on the semantic triple; Based on the structured query question, a hybrid retrieval mechanism is used to search the preset knowledge base to obtain at least one candidate knowledge fragment; The candidate knowledge fragments are subjected to knowledge fusion and noise filtering to obtain the target knowledge fragment; Multimodal intelligent push is performed on the target knowledge fragments.

2. The intelligent push method according to claim 1, characterized in that, The process of receiving customer call voice signals using speech recognition technology and converting the call voice signals into text data using streaming recognition includes: An end-to-end Transformer architecture is used to perform streaming recognition on the call voice signal; By utilizing a speech signal framing mechanism and combining acoustic features, the semantics of the next frame of the call speech signal are predicted, so as to synchronously convert the call speech signal into text data.

3. The intelligent push method according to claim 1, characterized in that, The step of performing multi-label intent recognition on the text data to obtain the corresponding intent recognition result, and generating semantic triples based on the intent recognition result, includes: The text data is subjected to multi-label intent recognition using a fine-tuned LLM model to obtain the intent recognition result; The contextual dialogue data corresponding to the intent recognition result is obtained, and the intent recognition result and the contextual dialogue data are input into the self-attention network, and the semantic triple is generated by the self-attention network.

4. The intelligent push method according to claim 1, characterized in that, The step of determining whether a customer has a new search intent based on the intent recognition result, and generating a structured query question based on the semantic triples when a new search intent is determined, includes: Obtain the difference between the intent recognition result and the historical intent, and compare the difference with a preset difference threshold; When the difference does not exceed a preset difference threshold, it is determined that there is no new search intent. When the difference exceeds a preset difference threshold, it is determined that there is a new search intent. Then, the semantic triple is transformed into a structured query question using a domain template fusion mechanism and a semantic filling mechanism.

5. The intelligent push method according to claim 1, characterized in that, The process involves retrieving a preset knowledge base based on the structured query question using a hybrid retrieval mechanism to obtain at least one candidate knowledge fragment, including: The Sentence-BERT model is used to embed the structured query question into vectors and knowledge items in the preset knowledge base. The cosine similarity between the structured query question and the knowledge items is calculated based on the vector embedding results to obtain the similarity results. The structured query is quickly fuzzily matched with knowledge entries in a preset knowledge base using an Elasticsearch inverted index structure to obtain the matching results. The similarity results and the matching results are weighted and sorted, and the top N knowledge items are selected as the candidate knowledge fragments based on the weighted sorting results; where N≥1.

6. The intelligent push method according to claim 1, characterized in that, The step of performing knowledge fusion and noise filtering on the candidate knowledge fragments to obtain the target knowledge fragment includes: Calculate the information gain of each candidate knowledge fragment to the explanatory power of the new retrieval intent, and filter the candidate knowledge fragments according to the information gain to obtain intermediate knowledge fragments; The intermediate knowledge fragments are semantically clustered and fused using a hierarchical clustering algorithm to obtain a unified response template; The unified response template is validated in context using a large language model, and the target knowledge fragment is set based on the results of the context validation.

7. The intelligent push method according to claim 1, characterized in that, The multimodal intelligent push of the target knowledge fragment includes: The target knowledge fragment is displayed in a parallel mode using multiple modalities; wherein the content of the multiple modal display includes a text summary, key flowcharts, and related connections of the target knowledge fragment. The system obtains feedback on the target knowledge fragment based on an interactive feedback mechanism, and then corrects the intelligent push based on the feedback.

8. An intelligent push device, characterized in that, include: The voice receiving unit is used to receive the customer's call voice signal through voice recognition technology, and to convert the call voice signal into text data using streaming recognition. An intent recognition unit is used to perform multi-label intent recognition on the text data, obtain the corresponding intent recognition result, and generate semantic triples based on the intent recognition result; The question generation unit is used to determine whether the customer has a new search intent based on the intent recognition result, and when it is determined that a new search intent exists, to generate a structured query question based on the semantic triple; The fragment retrieval unit is used to retrieve at least one candidate knowledge fragment from the preset knowledge base based on the structured query question through a hybrid retrieval mechanism. A fusion filtering unit is used to perform knowledge fusion and noise filtering on the candidate knowledge fragments to obtain the target knowledge fragment. The intelligent push unit is used to perform multimodal intelligent push of the target knowledge fragments.

9. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intelligent push method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the intelligent push method as described in any one of claims 1 to 7.

Images