Intelligent customer service implementation method, device and equipment

Abstract

The embodiment of the invention discloses an intelligent customer service implementation method, device and equipment, and belongs to the technical field of artificial intelligence. The method comprises the following steps: preprocessing business process data of any business in the field to which intelligent customer service belongs to obtain business process characters and business process tables of the business; processing the business process characters and the business process table through a multi-modal large model according to the prompt information to obtain a structured flow chart of the business; and when the intelligent customer service receives the question text of the user, replying the question text through the large language model based on the structured flow charts of the plurality of services in the field to which the intelligent customer service belongs to obtain a reply text of the question text, and feeding back the reply text to the user. According to the technical scheme, the construction efficiency of the structured flow chart can be improved, and the question and answer accuracy and response speed of the intelligent customer service staff can be remarkably enhanced based on the knowledge of the structured flow chart, so that the satisfaction degree of the user to the intelligent customer service staff is improved.

Description

Intelligent customer service implementation methods, devices and equipment Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus and device for implementing intelligent customer service. Background Technology

[0002] With the continuous development of artificial intelligence technology, intelligent customer service is widely used in practical applications. Using intelligent customer service can effectively improve the efficiency of answering user questions, providing convenience and a perfect experience for users' lives. Similar to human customer service representatives who need business process training before starting work, the implementation of intelligent customer service relies on structured process diagrams. These structured flowcharts allow intelligent customer service to accurately locate the customer's current process node and provide precise service.

[0003] Currently, the common method for constructing structured flowcharts of business processes is manual annotation. However, this manual method requires manually annotating the process nodes and relationships between them in the business documents, which consumes a lot of manpower, time, and costs, resulting in low efficiency in constructing structured flowcharts of business processes. Summary of the Invention

[0004] This application provides a method, apparatus, and device for implementing intelligent customer service. It not only improves the efficiency of constructing structured flowcharts but also significantly enhances the accuracy and response speed of intelligent customer service based on the knowledge gained from structured flowcharts, thereby increasing user satisfaction with intelligent customer service. The technical solution is as follows:

[0005] On the one hand, a method for implementing intelligent customer service is provided, the method including:

[0006] For any business in the field of intelligent customer service, the business process data of the business is preprocessed to obtain the business process text and business process table, and the business process data is used to represent the processing flow of the business.

[0007] The multimodal large model processes the business process text and business process table according to the prompt information to obtain a structured flowchart of the business. The prompt information is used to guide the multimodal large model to generate the structured flowchart of the business based on the business process text and the business process table. The structured flowchart is used to represent multiple entities in the business and the relationships between the multiple entities.

[0008] When the intelligent customer service receives a user's question text, it uses a large language model and a structured flowchart of multiple business processes in the domain to which the intelligent customer service belongs to respond to the question text, obtains the response text, and then sends the response text back to the user.

[0009] On the other hand, an intelligent customer service implementation device is provided, the device comprising:

[0010] The processing module is used to preprocess the business process data of any business in the field of intelligent customer service to obtain the business process text and business process table, and the business process data is used to represent the handling process of the business.

[0011] The construction module is used to process the business process text and business process table according to the prompt information through the multimodal large model to obtain the structured flowchart of the business. The prompt information is used to guide the multimodal large model to generate the structured flowchart of the business based on the business process text and the business process table. The structured flowchart is used to represent multiple entities in the business and the relationships between the multiple entities.

[0012] The generation module is used to, when the intelligent customer service receives a user's question text, respond to the question text using a large language model based on a structured flowchart of multiple businesses in the domain to which the intelligent customer service belongs, obtain the response text of the question text, and then feed the response text back to the user.

[0013] In some embodiments, the business process data is stored in document form. The processing module is configured to, for any business within the domain of intelligent customer service, segment the business process data into multiple pages, each page displaying a portion of the business process data; perform page recognition on the multiple pages to extract business process text and business process tables from the multiple pages; for any page, if a business process table exists on both the current page and the next page, determine whether the business process table on the current page is a complete table based on the bottom pixel position of the business process table on the current page and the top pixel position of the business process table on the next page; if the business process table on the current page is a complete table, determine that the business process table on the current page is the business process table of the business; if the business process table on the current page is not a complete table, concatenate the business process table on the current page and the business process table on the next page to obtain the business process table of the business.

[0014] In some embodiments, the processing module is configured to, for any page, determine that the business process table of the page is a complete table if the bottom pixel position of the business process table on the page is less than a first threshold; determine that the business process table of the page is a complete table if the bottom pixel position of the business process table of the page is not less than the first threshold and the top pixel position of the business process table of the next page is greater than a second threshold; and determine that the business process table of the page is not a complete table if the bottom pixel position of the business process table of the page is not less than the first threshold and the top pixel position of the business process table of the next page is not greater than the second threshold.

[0015] In some embodiments, the prompt information includes multiple prompt words, and the construction module includes:

[0016] The first construction unit is used to process the business process text and the business process table through a multimodal large model according to a first prompt word to obtain multiple business process steps of the business. The first prompt word is used to guide the multimodal large model to extract the business process steps of the business from the business process text and the business process table.

[0017] The second construction unit is used to process the multiple business process steps, the business process table, and the business process text according to the second prompt word through the multimodal big model to obtain a business processing description. The second prompt word is used to guide the multimodal big model to generate a business processing description based on the multiple business process steps, the business process table, and the business process text. The processing description includes the dependency relationships between the multiple business process steps and the detailed content of each business process step.

[0018] The third construction unit is used to process the processing procedure description based on the third prompt word through the multimodal large model to obtain a structured flowchart of the business. The third prompt word is used to guide the multimodal large model to generate the structured flowchart of the business based on the processing procedure description.

[0019] In some embodiments, the third construction unit is configured to extract multiple entities of the business and the relationships between the multiple entities from the processing procedure description using the multimodal large model and based on the third prompt word; and to generate a structured flowchart of the business using the multimodal large model and based on the third prompt word, with the multiple entities as nodes and the relationships between the multiple entities as edges.

[0020] In some embodiments, the generation module includes:

[0021] The recognition unit is used to perform intent recognition on the question text through the large language model when the intelligent customer service receives the user's question text, so as to obtain the user's question intent, which is used to indicate the content that the user wants to know or operate.

[0022] The query unit is used to query at least one structured flowchart related to the question intent from multiple business structured flowcharts in the domain, based on the question intent, using the large language model.

[0023] The generation unit is used to generate the response text to the question text based on at least one structured flowchart related to the question intent, using the large language model.

[0024] In some embodiments, the generation unit is configured to, for any structured flowchart, extract flowchart information related to the question intent from the structured flowchart, the flowchart information including at least one of entities and relationships between entities in the structured flowchart; integrate the flowchart information of the at least one structured flowchart based on the type of the entity; and generate a response text to the question text based on the integrated flowchart information.

[0025] In some embodiments, the apparatus further includes:

[0026] The storage module is used to bind any entity in the structured flowchart to the business file or business link that the entity needs to access; to merge the structured flowcharts of at least one business that is related to the business logic of multiple businesses in the domain; and to store the merged structured flowcharts in the form of graph data.

[0027] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the intelligent customer service implementation method as described above.

[0028] On the other hand, a computer-readable storage medium is provided, wherein at least one computer program is stored therein, the at least one computer program being loaded and executed by a processor to implement the intelligent customer service implementation method as described above.

[0029] On the other hand, a computer program product is provided, including a computer program that is loaded and executed by a processor to implement the intelligent customer service implementation method as described above.

[0030] This application provides a method for implementing intelligent customer service. By designing prompts for a multimodal large model, it can guide the model to quickly understand the specific content of business process data for multiple businesses within the domain of intelligent customer service. This allows for the correct inference of entities and relationships within the business process data, enabling the rapid generation of structured flowcharts and improving the efficiency of flowchart construction. By combining structured flowcharts with a large language model to implement intelligent customer service, and leveraging the knowledge gained from the structured flowcharts, the accuracy and response speed of intelligent customer service questions are significantly enhanced, thereby increasing user satisfaction with intelligent customer service. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0033] Figure 2 is a flowchart of an intelligent customer service implementation method provided in an embodiment of this application;

[0034] Figure 3 is a flowchart of another intelligent customer service implementation method provided in an embodiment of this application;

[0035] Figure 4 is a flowchart of a data preprocessing stage provided in an embodiment of this application;

[0036] Figure 5 is a flowchart of the steps of an "online account opening" service provided in an embodiment of this application;

[0037] Figure 6 is a structured flowchart of an "online account opening" service provided in an embodiment of this application;

[0038] Figure 7 is a structured flowchart of another "online account opening" service provided in an embodiment of this application;

[0039] Figure 8 is a flowchart of a flowchart generation stage provided in an embodiment of this application;

[0040] Figure 9 is a structural schematic diagram of an intelligent customer service implementation device provided in an embodiment of this application;

[0041] Figure 10 is a schematic diagram of another intelligent customer service implementation device provided in an embodiment of this application;

[0042] Figure 11 is a schematic diagram of the structure of a terminal provided in an embodiment of this application;

[0043] Figure 12 is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0045] It is understood that the terms "first," "second," etc., used in this application may be used to describe various concepts herein, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of this application, a first prompt may be referred to as a second prompt, and similarly, a second prompt may be referred to as a first prompt.

[0046] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the business process data involved in this application was obtained with full authorization.

[0047] The implementation environment of the embodiments of this application is described below.

[0048] Figure 1 is a schematic diagram of an implementation environment provided in an embodiment of this application. Referring to Figure 1, the implementation environment includes a terminal 101 and a server 102. The terminal 101 can be connected to the server 102 via a wireless network or a wired network.

[0049] Optionally, terminal 101 can be at least one of a smartphone, desktop computer, laptop, or foldable computer. Terminal 101 has an application installed or running, which provides intelligent customer service to the user. Upon receiving a user's question text, terminal 101 can extract the question text and send it to server 102. Server 102 is deployed with a large language model. It queries the database of server 102 for a structured flowchart of the business related to the question text, and then, based on the structured flowchart, responds to the question text using the large language model, obtaining the answer text, which is then fed back to the user. The application is associated with server 102, which provides backend services. Server 102 also deploys a multimodal large model, which processes the business process data uploaded by the technical personnel developing the intelligent customer service, generates a structured flowchart of the corresponding business, and stores it in the database.

[0050] Optionally, server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some embodiments, server 102 undertakes the main computing work, and terminal 101 undertakes the secondary computing work; or, server 102 undertakes the secondary computing work, and terminal 101 undertakes the main computing work; or, server 102 and terminal 101 collaborate on computing using a distributed computing architecture.

[0051] Terminal 101 can refer to one of a plurality of terminals; this embodiment uses terminal 101 as an example. Those skilled in the art will understand that the number of terminals can be more or less. For example, there may be several terminals, or dozens or hundreds of terminals, or even more. This application embodiment does not limit the number of terminals or the type of device.

[0052] Figure 2 is a flowchart of an intelligent customer service implementation method provided in an embodiment of this application. This embodiment is executed by a computer device, and referring to Figure 2, the method includes:

[0053] 201. For any business in the field of intelligent customer service, the computer equipment preprocesses the business process data to obtain the business process text and business process table. The business process data is used to represent the business handling process.

[0054] In this embodiment, the intelligent customer service can be customer service from a mobile communication company, a natural gas sales company, a game company, etc., and the type of intelligent customer service in this embodiment is not limited. Correspondingly, the field to which the intelligent customer service belongs can be the mobile communication field, the natural gas sales field, the game field, etc. Taking the intelligent customer service of a natural gas sales company as an example, the business in the natural gas sales field includes natural gas account opening, customer suspension and dismantling services, natural gas payment, and natural gas repair services, etc.

[0055] Developers of intelligent customer service systems can upload business process data from multiple business areas within the intelligent customer service ecosystem to computer devices. The business process data for each business represents its processing flow. This data includes both text and tabular data. Accordingly, for any given business, in order to obtain key information related to that business, the computer device can preprocess the business process data to produce both the text and tabular versions of the business process.

[0056] 202. The computer equipment processes the business process text and business process table according to the prompt information through the multimodal large model to obtain the structured flowchart of the business. The prompt information is used to guide the multimodal large model to generate the structured flowchart of the business based on the business process text and business process table. The structured flowchart is used to represent multiple entities in the business and the relationships between multiple entities.

[0057] In this embodiment, the multimodal large model is a deep learning model capable of processing various media data (such as text, images, audio, video, etc.). In processing various media data, the multimodal large model can integrate media data from different modalities and uncover the correlations and complementarities between them. The prompt information essentially poses specific questions to the multimodal large model, guiding it to generate a structured flowchart of the business process based on the business process text and tables. Accordingly, when a multimodal model is deployed on a computer device, the computer device can process the preprocessed business process text and tables according to the input prompt information to obtain a structured flowchart of the business process.

[0058] 203. When the intelligent customer service receives a user's question text, the computer device uses a large language model and a structured flowchart of multiple business processes in the domain of intelligent customer service to respond to the question text, obtain the response text, and then feeds the response text back to the user.

[0059] In this embodiment, the computer device is equipped with a large language model. A large language model is a deep learning model used for processing text data. During text data processing, the large language model can quickly understand the meaning of the text data and generate natural language text related to the text data. Users can ask questions to intelligent customer service representatives through an application that provides intelligent customer service. When the intelligent customer service representative in the application receives the user's question text, it can send the question text to the computer device. Accordingly, the computer device can respond to the question text using the large language model, based on a structured flowchart of multiple business processes within the domain of the intelligent customer service representative, obtaining a response text. This response text is then fed back to the intelligent customer service representative, who then replies to the user.

[0060] This application provides a method for implementing intelligent customer service. By designing prompts for a multimodal large model, it can guide the model to quickly understand the specific content of business process data for multiple businesses within the domain of intelligent customer service. This allows for accurate inference of entities and relationships within the business process data, enabling rapid generation of structured flowcharts and improving the efficiency of flowchart construction. Combining structured flowcharts with a large language model to implement intelligent customer service significantly enhances the accuracy and response speed of question answering, thereby increasing user satisfaction.

[0061] Figure 2 above exemplarily illustrates the main flow of the intelligent customer service implementation method provided in the embodiments of this application. The intelligent customer service implementation scheme will be described in detail below. Figure 3 is a flowchart of another intelligent customer service implementation method provided in the embodiments of this application. This method is executed by a computer device. Referring to Figure 3, the method includes:

[0062] 301. For any business in the field of intelligent customer service, the computer equipment preprocesses the business process data to obtain the business process text and business process table. The business process data is used to represent the business handling process.

[0063] In this embodiment, the computer device can acquire business process data of a service in various ways. For example, the computer device can acquire business process data from a local database. This local database stores business process data from multiple services within the domain of intelligent customer service. Alternatively, the computer device can acquire business process data uploaded by multiple terminals. These terminals can be those used by the developers of the intelligent customer service system.

[0064] Intelligent customer service can be provided by mobile communication companies, natural gas sales companies, game companies, etc., and the type of intelligent customer service is not limited in this application embodiment. Accordingly, the field to which intelligent customer service belongs can be the mobile communication field, the natural gas sales field, the game field, etc. Taking intelligent customer service as the customer service of a natural gas sales company as an example, the business in the natural gas sales field includes natural gas account opening, customer suspension, dismantling, natural gas payment, and natural gas repair services.

[0065] The business process data for each service represents the processing procedure for that service. This data includes both text and tabular data. Accordingly, for any given service, in order to obtain key information related to that service, the computer equipment can preprocess the business process data to obtain the text and tabular versions of the business process.

[0066] In some embodiments, the business process data is stored in document form. The process of preprocessing the business process data by a computer device may include the following steps:

[0067] (1) For any business within the field of intelligent customer service, the computer device segments the business process data into multiple pages, each displaying a portion of the business process data. Since the document comprises multiple pages, each displaying different business process data, the computer device can segment the business process data according to the pages. During the segmentation process, the computer device can convert each page into an image of a preset size. The preset size can be 2552×3579 pixels.

[0068] (2) The computer device performs page recognition on multiple pages to extract business process text and business process tables from the multiple pages. Specifically, by recognizing the content of each page, the computer device can understand the content displayed on each page, thereby extracting business process text and business process tables related to the business from the page.

[0069] (3) For any page, if both the current page and the next page contain business process tables, the computer device determines whether the business process table on the current page is a complete table based on the bottom pixel position of the business process table on the current page and the top pixel position of the business process table on the next page. Since tables are displayed in paginated form on a page, the computer device needs to determine whether the business process table extracted from the page is a complete table. If the next page does not contain a business process table, the computer device can determine that the business process table on the current page is a complete table. If the next page contains a business process table, the computer device can determine whether the two business process tables are the same table based on the bottom pixel position of the business process table on the current page and the top pixel position of the business process table on the next page.

[0070] In some embodiments, for any page, if the bottom pixel position of the business process table on the page is less than a first threshold, it indicates that the bottom edge of the business process table is far from the bottom boundary of the page, that is, it indicates that the business process table has ended on the page, and therefore the computer device determines that the business process table on the page is a complete table; if the bottom pixel position of the business process table on the page is not less than the first threshold, it indicates that the bottom edge of the business process table on the current page is close to the bottom boundary of the page, that is, the business process table may not have ended on the page. However, if the top pixel position of the business process table on the next page is greater than the second threshold, it indicates that the top edge of the business process table on the next page is far from the top boundary of the next page. In other words, the business process table on the next page is not the same table as the business process table on the current page. Therefore, the computer device can determine that the business process table on the current page is a complete table. Conversely, if the bottom pixel position of the business process table on the current page is not less than the first threshold, and the top pixel position of the business process table on the next page is not greater than the second threshold, it indicates that the top edge of the business process table on the next page is close to the top boundary of the next page. In other words, the business process table on the next page is the same table as the business process table on the current page. Therefore, the computer device determines that the business process table on the current page is not a complete table.

[0071] By comparing the pixel positions of the top and bottom edges of a table with a threshold value, it is possible to determine whether a table is complete. This method can efficiently and accurately determine whether the business process table on each page is complete, thereby improving the processing speed of business process data on computer equipment while ensuring the accuracy of the extracted business process table.

[0072] For example, for any page, the computer can establish a coordinate system with the top-left corner of the page as the origin, downwards as the y-axis, and to the right as the x-axis. The computer identifies the bottom pixel of the business process table on the current page as y_bottom. If y_bottom is less than a first threshold a_bottom, the table is considered to have ended on this page. If it is greater than or equal to a_bottom, the table may not have ended. If there is no table on an adjacent page, the table is considered to have ended on this page. If there is a table on an adjacent page, the top pixel of the adjacent page's table is identified as y_top. If the top pixel of the adjacent page's table is less than or equal to a second threshold a_top, the table is considered to have not ended on this page, and the adjacent page's table is grouped with this table. If the top pixel is greater than a_top, the table is considered to have ended on this page. The first threshold a_bottom can be 3200, and the second threshold a_top can be 530.

[0073] (4) If the business process table on the page is a complete table, the computer device determines that the business process table on the page is the business process table of the business.

[0074] (5) When the business process table on the page is not a complete table, the computer device splices the business process table on the page and the business process table on the next page to obtain the business process table. Among them, the computer device can vertically splice the business process table on the page and the business process table on the next page, and store the spliced ​​table in JPEG (Joint Photographic Experts Group) format.

[0075] For example, Figure 4 is a flowchart of a data preprocessing stage provided in an embodiment of this application. As shown in Figure 4, the computer device can first paginate the document of business process data, and then perform page recognition on each page. Based on the type of page elements, it extracts the business process text and business process tables from the page. For the business process table, the computer device needs to further determine whether the business process table is a complete table. If it is, it indicates that the business process table is a correct table. If not, it needs to concatenate business process tables belonging to the same table to obtain a complete table.

[0076] 302. The computer equipment uses a multimodal large model to process the business process text and business process table according to the first prompt word to obtain multiple business process steps. The first prompt word is used to guide the multimodal large model to extract the business process steps from the business process text and business process table.

[0077] In this embodiment, the multimodal big data model deployed on the computer device is a deep learning model capable of processing various media data (such as text, images, audio, video, etc.). During the processing of various media data, the multimodal big data model can integrate media data from different modalities and explore the correlations and complementarities between them. The first prompt word essentially poses a specific question to the multimodal big data model, guiding it to extract the business process steps from the business process text and business process tables. Accordingly, the computer device can process the business process text and business process tables based on the input first prompt word to obtain multiple business process steps.

[0078] Among them, the multimodal large model can be Qwen-VL-Max (Qwen Vision Language Max, the largest version of the Tongyi Qianwen visual language model) or Qwen-VL-Chat (Qwen Vision Language Chat, the Tongyi Qianwen visual language chat model).

[0079] For example, taking the natural gas sales sector as an example, Table 1 is a business process table for the "online account opening" service in the natural gas sales sector.

[0080] Table 1: Online Account Opening Process Table

[0081]

[0082] Taking Qwen-VL-Max, a closed-source multimodal large model, as an example, a computer device can call the Qwen-VL-Max API (Application Programming Interface) to process the images corresponding to Table 1 based on the input first prompt word. The first prompt word can be:

[0083] You are now acting as a step extraction tool (representing the role of a multimodal model);

[0084] Extract the specific steps of the business process from the table shown in the image (representing the tasks and inputs of the multimodal model);

[0085] You can analyze and extract information from the table column named "Business Activities" (representing the steps of the multimodal model);

[0086] Finally, all process steps are presented in the form of a two-dimensional table (representing the output of the multimodal model).

[0087] The final business process steps obtained by the computer equipment can be shown in Table 2.

[0088] Table 2: Business Process Steps for "Online Account Opening"

[0089] Serial Number Business Process Steps 01 Read Account Opening Instructions 02 Submit Gas Usage Address 03 Verify Gas Usage Address 04 Upload Account Opening Documents 05 Sign Gas Supply Agreement 06 Open Account surface

[0090] 303. The computer equipment, through a multimodal large model, processes multiple business process steps, business process tables, and business process texts according to the second prompt word to obtain a business processing description. The second prompt word is used to guide the multimodal large model to generate a business processing description based on multiple business process steps, business process tables, and business process texts. The processing description includes the dependencies between multiple business process steps and the detailed content of each business process step.

[0091] In this embodiment, the second prompt essentially poses a specific question to the multimodal large model, guiding it to generate a business processing description based on multiple business process steps, business process tables, and business process text. Correspondingly, the computer device can process the business process table, business process text, and multiple business process steps extracted from the first prompt based on the continuously input second prompt, obtaining a business processing description. The dependency relationships between the multiple business process steps included in the processing description indicate the sequential relationship between them. For example, step B can only be executed after step A is completed; or, if the result of step A is case 1, step B is executed; if the result of step A is case 2, step C is executed.

[0092] For example, the second prompt word could be:

[0093] Based on the content of this business process table and the step identification results obtained in the previous step (representing the input of the multimodal model);

[0094] Provide a detailed description of the entire business process (representing the task of the multimodal model);

[0095] The description should include all steps and clearly explain the dependencies between steps (representing the output of the multimodal model).

[0096] Based on the second prompt mentioned above, the computer equipment processes the content in Table 1 and the step recognition results in Table 2, resulting in the following business process description: "This business process mainly involves account opening services. First, the customer needs to read the account opening instructions to understand the relevant conditions and precautions. Next, the customer submits their gas usage address, which the system verifies to ensure it is within the company's business scope. After approval, residential customers need to upload photos of the homeowner's ID card and property certificate, while non-residential customers need to upload photos of their business license, company certificate, and legal representative's ID card. Next, the customer completes the online signing of the gas supply agreement. Finally, the customer enters their payment number and completes the payment, thus completing the entire account opening process."

[0097] In some embodiments, the computer device can also obtain flowchart drawing code representing the dependencies between business process steps through a multimodal model. Based on this flowchart drawing code, the computer device can generate a flowchart representing the dependencies between multiple business process steps. For example, Figure 5 is a flowchart of the steps of an "online account opening" service provided in an embodiment of this application. As shown in Figure 5, the "online account opening" service includes six steps, which are executed in the following order: reading the account opening instructions, submitting the gas address, verifying the gas address, uploading the account opening materials, signing the gas supply agreement, and opening the account.

[0098] 304. The computer equipment processes the process description based on the third prompt word through the multimodal large model to obtain a structured flowchart of the business. The third prompt word is used to guide the multimodal large model to generate a structured flowchart of the business based on the process description. The structured flowchart is used to represent multiple entities in the business and the relationships between multiple entities.

[0099] In this embodiment, the third prompt essentially poses a specific question to the multimodal large model, guiding it to generate a structured flowchart of the business process based on the processing procedure description. Correspondingly, the computer device can process the processing procedure description obtained from the second prompt based on the continuously outputting third prompt, thus obtaining a structured flowchart of the business process. This structured flowchart is graph data composed of multiple nodes and edges. Nodes represent entities, and edges represent relationships between entities.

[0100] In some embodiments, the computer device, using a multimodal large model and based on a third prompt, can first extract multiple entities and relationships between them from the process description. Then, using the multimodal large model and based on the third prompt, it generates a structured flowchart of the business, with the entities as nodes and the relationships between them as edges. Specifically, the computer device can first generate drawing code for the structured flowchart using the multimodal large model and based on the third prompt. Then, by executing the drawing code, it obtains a structured flowchart composed of multiple entities as nodes and the relationships between them as edges.

[0101] For example, the third cue word could be:

[0102] You are a knowledge graph entity and relation extractor (representing a multimodal model role);

[0103] Construct a corresponding knowledge graph (representing the tasks and inputs of the multimodal model) based on the description of the "online account opening" business process;

[0104] First, you must extract the entities and relationships, and then use "Online Account Opening Service" as the root node. All core nodes need to start from the root node (representing the steps of the multimodal model).

[0105] Finally, the constructed knowledge graph will be visualized, generating Mermaid (an open-source library for generating diagrams and flowcharts) drawing code (representing the output of the multimodal model).

[0106] Based on the aforementioned third prompt word, the computer equipment extracted multiple entities from the processing procedure description, as shown in Table 3.

[0107] Table 3: List of entities providing "online account opening" service

[0108] Serial Number Entity Name Serial Number Entity Name 1 Online Account Opening Service 9 Property Certificate Photo 2 Customer 10 Non-Residential Customer 3 Account Opening Instructions 11 Business License Photo 4 Gas Usage Address 12 Company Certificate Photo 5 Address Verification System 13 Legal Representative's ID Card Photo 6 Company Business Scope 14 Gas Supply Agreement 7 Residential Customer 15 Payment Number 8 Household Owner's ID Card Photo 16 Payment Process surface

[0109] The computer device processes multiple entities and the relationships between them based on third-party prompts, and the resulting Mermaid drawing code can be:

[0110] 1graph LR (declaring the flow chart direction from left to right)

[0111] 2. Online Account Opening Service --> Customer

[0112] 3. Customer --> Account Opening Instructions (Read the Account Opening Instructions)

[0113] 4. Customer --> | Submit | Gas Address (Gas Address)

[0114] 5. Gas usage address --> | Verification | Address verification system (address range confirmation)

[0115] 6. Address Verification System --> | Verification Approved | Resident Customer (Resident Customer)

[0116] 7. Address Verification System --> | Verification Approved | Non-Resident Customer (Non-Resident Customer)

[0117] 8. Residential Customers --> | Upload | Photo of the Head of Household's ID Card (ID Card Photo)

[0118] 9. Resident customers --> | Upload | Property certificate photo (Property certificate photo)

[0119] 10 Non-resident customers --> | Upload | Business license photo (Business license photo)

[0120] 11. Non-resident customers --> | Upload | Photo of employer's certificate (Employer's certificate photo)

[0121] 12 Non-resident customers --> | Upload | Legal representative's ID photo (Legal representative's ID photo)

[0122] 13 Customers --> | Sign | Gas Supply Agreement (Sign Gas Supply Agreement Online)

[0123] 14. Customer --> | Enter | Payment Number (Enter Payment Number)

[0124] 15 Customers --> | Complete | Payment Process (Payment Completed)

[0125] Based on the code described above, the computer equipment can generate the structured flowchart of the "Online Account Opening" service shown in Figure 6. As shown in Figure 7, "Online Account Opening Service" is the root node, and entities such as "Customer," "Gas Address," and "Residential Customer" are all nodes originating from the root node. The text on the lines connecting the nodes reflects the relationships between them. For example, the relationship between "Customer" and "Gas Address" is "Submit."

[0126] In some embodiments, the computer device can also optimize the structured flowchart shown in Figure 6 to obtain the structured flowchart of the "online account opening" business shown in Figure 7. As shown in Figure 7, this structured flowchart can arrange multiple entities according to the execution order of multiple business process steps of the "online account opening" business, enhancing the orderliness of the process nodes in the structured process, enabling intelligent customers to quickly locate the process node where the customer is and provide accurate services.

[0127] For example, Figure 8 is a flowchart of a flowchart generation stage provided in an embodiment of this application. As shown in Figure 8, the computer device can process the text and complete table of the business through a multimodal large model to obtain the business process steps. Then, through the multimodal large model, the business process steps, the text of the business, and the complete table are processed to obtain the dependency relationships between multiple business process steps and the specific content of each business process step. Finally, the computer device processes the dependency relationships between multiple business process steps and the specific content of each business process step through the multimodal large model to obtain the drawing code for the structured flowchart of the business, and then obtains the structured flowchart by executing the drawing code.

[0128] In some embodiments, after generating a structured flowchart, for any entity in the structured flowchart, the computer device can also bind the entity to a business file or business link that the entity needs to access; and merge the structured flowcharts of at least one business whose business logic is related among multiple businesses in the domain; finally, the merged multiple structured flowcharts are stored in the form of graph data. Specifically, for multiple entities in the structured flowchart, the computer device can bind backend business files or backend business links to entities that need to access backend services. In the example above, the computer device needs to add backend service links for entities such as the gas address verification system, account opening information verification system, gas supply agreement signing system, and payment system in the "online account opening" business to the entity's attributes, so that when the large language model calls these entities subsequently, it can directly access the relevant data bound to the entity. Business logic relatedness refers to the existence of the same entity in different business processes. For example, both the "online account opening" and "online account closing" business processes include the customer entity. Computer equipment can merge the structured flowcharts of these two business processes, that is, use the customer entity as the related node to integrate these two structured processes into a single structured flowchart. This not only saves storage space for the structured flowcharts but also improves the query efficiency of large language models in retrieving relevant information from the structured flowcharts.

[0129] 305. When the intelligent customer service receives a user's question text, the computer device uses a large language model to perform intent recognition on the question text to obtain the user's question intent, which is used to indicate what the user wants to know or operate.

[0130] In this embodiment, the large language model deployed on the computer device is a deep learning model for processing text data. During text data processing, the large language model can quickly understand the meaning of the text data and generate natural language text related to the text data. Users can ask questions to intelligent customer service representatives through an application that provides intelligent customer service. When the intelligent customer service representative in the application receives the user's question text, it can send the question text to the computer device. Accordingly, the computer device can preprocess the question text using the large language model to understand the user's question intent. This preprocessing includes operations such as word segmentation, stop word removal, and part-of-speech tagging.

[0131] The large language model can be a pre-trained ChatGLM3-6b (General Language Model for Chat), ChatGPT (Chat Generative Pre-trained Transformer), or LLaMA (Large Language Model MetaArtificial Intelligence), and this application does not limit it.

[0132] 306. The computer device uses a large language model to query at least one structured flowchart related to the question intent from multiple business structured flowcharts in the domain.

[0133] In this embodiment of the application, the computer device can query structured flowcharts related to the user's question intent from multiple business structured flowcharts within the domain of intelligent customer service. For example, when the user's question intent is to understand the operation process of "online account opening" and "online account cancellation," the computer device can query the structured flowcharts for "online account opening" and "online account cancellation" from multiple structured flowcharts.

[0134] 307. The computer device generates a response text to the question text based on at least one structured flowchart related to the question intent using a large language model, and then feeds the response text back to the user.

[0135] In this embodiment, the computer device can process at least one structured flowchart related to the user's question intent using a large language model to obtain a business processing flow represented by the structured flowchart. Based on this business processing flow, the device can then answer the user's question text, obtaining a response text. This response text is then fed back to the intelligent customer service system, which responds to the user.

[0136] In some embodiments, for any structured flowchart, the computer device extracts flowchart information related to the question intent from the structured flowchart. The flowchart information includes at least one of the entities in the structured flowchart and the relationships between entities. The computer device integrates the flowchart information of at least one structured flowchart based on the entity type. Based on the integrated flowchart information, it generates a response text to the question text. Specifically, after extracting multiple entities and their relationships from the structured flowchart, the computer device can integrate information related to the same entity based on the entity type to obtain comprehensive information about that entity. This comprehensive information includes all entities associated with that entity and the relationships between each entity. For example, the flowchart information extracted by the computer device from the structured flowchart of the "online account opening" business includes a customer entity, a property certificate photo entity, a homeowner's ID card photo entity, and a submission relationship between the customer entity and the property certificate photo entity, and between the customer entity and the homeowner's ID card photo entity. After integrating this flowchart information, the computer device can obtain the flowchart information as "customer submits property certificate photo and homeowner's ID card photo."

[0137] For example, when a user asks a question to the intelligent customer service, the specific process of the intelligent customer service responding to the user can be as follows:

[0138] Customer: Hello, the gas alarm in our house went off.

[0139] AI-powered customer service: Which neighborhood?

[0140] Client: XX Zone 1

[0141] Intelligent Customer Service: Does XX have any alarms for the newly repaired gas supply system?

[0142] Customer: I don't know about other people's cases, but we just started cooking when the police called.

[0143] AI Customer Service: Oh, you want to report a repair?

[0144] Customer: Could you send someone over to take a look at our house?

[0145] AI-powered customer service: Just give me your address.

[0146] Customer: Unit 101, Building 3, Zone 1, XX.

[0147] AI Customer Service: Okay, I'll contact an employee to come and check it out for you.

[0148] Customer: Okay, when can you come over?

[0149] AI-powered customer service: We'll get in touch with you right away.

[0150] Customer: Okay, as soon as possible, thank you.

[0151] AI customer service: You're welcome.

[0152] As described above, the computer equipment, through a large language model, can access the structured flowchart of the on-site repair service based on the customer's question text and obtain specific process information from it. After the question and answer session, the computer equipment can automatically input the obtained user address into the order dispatch process, eliminating the need for manual order dispatch by customer service. The repair technician can directly receive the work order and perform on-site repairs.

[0153] This application provides a method for implementing intelligent customer service. By designing prompts for a multimodal large model, it can guide the model to quickly understand the specific content of business process data for multiple businesses within the domain of intelligent customer service. This allows for the correct inference of entities and relationships within the business process data, enabling the rapid generation of structured flowcharts and improving the efficiency of flowchart construction. By combining structured flowcharts with a large language model to implement intelligent customer service, and leveraging the knowledge gained from the structured flowcharts, the accuracy and response speed of intelligent customer service questions are significantly enhanced, thereby increasing user satisfaction with intelligent customer service.

[0154] Figure 9 is a schematic diagram of an intelligent customer service implementation device provided in an embodiment of this application. Referring to Figure 9, the device includes: a processing module 901, a construction module 902, and a generation module 903.

[0155] The processing module 901 is used to preprocess the business process data of any business in the field of intelligent customer service to obtain the business process text and business process table. The business process data is used to represent the business handling process.

[0156] Module 902 is used to process business process text and business process tables through a multimodal large model based on prompt information to obtain a structured flowchart of the business. The prompt information is used to guide the multimodal large model to generate a structured flowchart of the business based on the business process text and business process tables. The structured flowchart is used to represent multiple entities in the business and the relationships between multiple entities.

[0157] The generation module 903 is used to respond to the user's question text when the intelligent customer service receives the question text, by using a large language model and based on the structured flowchart of multiple businesses in the domain to which the intelligent customer service belongs, to obtain the response text of the question text, and then to feed the response text back to the user.

[0158] In some embodiments, business process data is stored in document form. Processing module 901 is used to perform page segmentation on any business within the domain of intelligent customer service, resulting in multiple pages, each displaying a portion of the business process data; perform page recognition on the multiple pages to extract business process text and business process tables from the multiple pages; for any page, if a business process table exists on both the page and the next page, determine whether the page's business process table is a complete table based on the bottom pixel position of the page's business process table and the top pixel position of the next page's business process table; if the page's business process table is a complete table, determine that the page's business process table is the business process table of the business; if the page's business process table is not a complete table, concatenate the page's business process table and the next page's business process table to obtain the business process table of the business.

[0159] In some embodiments, the processing module 901 is configured to, for any page, determine that the business process table of the page is a complete table if the bottom pixel position of the business process table on the page is less than a first threshold; determine that the business process table of the page is a complete table if the bottom pixel position of the business process table of the page is not less than the first threshold and the top pixel position of the business process table of the next page is greater than a second threshold; and determine that the business process table of the page is not a complete table if the bottom pixel position of the business process table of the page is not less than the first threshold and the top pixel position of the business process table of the next page is not greater than the second threshold.

[0160] In some embodiments, the prompt information includes multiple prompt words. Figure 10 is a schematic diagram of another intelligent customer service implementation device provided in an embodiment of this application. Referring to Figure 10, the construction module 902 includes:

[0161] The first building unit 1001 is used to process the business process text and business process table according to the first prompt word through the multimodal large model to obtain multiple business process steps of the business. The first prompt word is used to guide the multimodal large model to extract the business process steps of the business from the business process text and business process table.

[0162] The second building unit 1002 is used to process multiple business process steps, business process tables and business process texts according to the second prompt word through the multimodal large model to obtain a business processing description. The second prompt word is used to guide the multimodal large model to generate a business processing description based on multiple business process steps, business process tables and business process texts. The processing description includes the dependency relationship between multiple business process steps and the detailed content of each business process step.

[0163] The third building unit 1003 is used to process the process description based on the third prompt word through the multimodal large model to obtain a structured flowchart of the business. The third prompt word is used to guide the multimodal large model to generate a structured flowchart of the business based on the process description.

[0164] In some embodiments, the third construction unit 1003 is used to extract multiple entities of the business and the relationships between the multiple entities from the process description through a multimodal large model based on the third prompt words; and to generate a structured flowchart of the business with multiple entities as nodes and the relationships between the multiple entities as edges through the multimodal large model based on the third prompt words.

[0165] In some embodiments, continuing to refer to FIG10, the generation module 903 includes:

[0166] The recognition unit 1004 is used to perform intent recognition on the question text through a large language model when the intelligent customer service receives the user's question text, so as to obtain the user's question intent. The question intent is used to indicate the content that the user wants to know or operate.

[0167] The query unit 1005 is used to query at least one structured flowchart related to the question intent from multiple business structured flowcharts in the domain based on the question intent, using a large language model.

[0168] The generation unit 1006 is used to generate the response text of the question text based on at least one structured flowchart related to the question intent using a large language model.

[0169] In some embodiments, the generation unit 1006 is configured to, for any structured flowchart, extract flowchart information related to the question intent from the structured flowchart, the flowchart information including at least one of the entities in the structured flowchart and the relationships between entities; integrate the flowchart information of at least one structured flowchart based on the type of the entity; and generate a response text to the question text based on the integrated flowchart information.

[0170] In some embodiments, continuing to refer to FIG10, the apparatus further includes:

[0171] Storage module 904 is used to bind the business files or business links that the entity needs to access to any entity in the structured flowchart; and to merge the structured flowcharts of at least one business whose business logic is related among multiple businesses in the domain. The merged structured flowcharts are then stored in the form of graph data.

[0172] This application provides an intelligent customer service implementation device. By designing prompts for a multimodal large model, it can guide the model to quickly understand the specific content of business process data for multiple businesses within the domain of intelligent customer service. This allows for accurate inference of entities and relationships within the business process data, enabling rapid generation of structured flowcharts and improving the efficiency of flowchart construction. By combining structured flowcharts with a large language model to implement intelligent customer service, and leveraging the knowledge gained from the structured flowcharts, the accuracy and response speed of intelligent customer service questions are significantly enhanced, thereby increasing user satisfaction with the intelligent customer service.

[0173] It should be noted that the intelligent customer service implementation device provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the intelligent customer service implementation device and the intelligent customer service implementation method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0174] In the embodiments of this application, the computer device can be configured as a terminal or a server. When the computer device is configured as a terminal, the terminal can act as the execution subject to implement the technical solutions provided in the embodiments of this application. When the computer device is configured as a server, the server can act as the execution subject to implement the technical solutions provided in the embodiments of this application. Alternatively, the technical solutions provided in this application can be implemented through the interaction between the terminal and the server. The embodiments of this application do not limit this.

[0175] Figure 11 is a schematic diagram of the structure of a terminal provided in an embodiment of this application.

[0176] Typically, terminal 1100 includes a processor 1101 and a memory 1102.

[0177] Processor 1101 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1101 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1101 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1101 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0178] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1102 are used to store at least one computer program, which is executed by the processor 1101 to implement the intelligent customer service method provided in the method embodiments of this application.

[0179] In some embodiments, the terminal 1100 may also optionally include a peripheral device interface 1103 and at least one peripheral device. The processor 1101, memory 1102, and peripheral device interface 1103 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1103 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, an audio circuit 1107, and a power supply 1108.

[0180] Peripheral device interface 1103 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1101 and memory 1102. In some embodiments, processor 1101, memory 1102 and peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1101, memory 1102 and peripheral device interface 1103 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0181] The radio frequency (RF) circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1104 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1104 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. In some embodiments, the RF circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1104 can communicate with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1104 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0182] Display screen 1105 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1105 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1101 for processing. In this case, display screen 1105 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1105, disposed on the front panel of terminal 1100; in other embodiments, there may be at least two display screens, disposed on different surfaces of terminal 1100 or in a folded design; in still other embodiments, display screen 1105 may be a flexible display screen, disposed on a curved or folded surface of terminal 1100. Furthermore, display screen 1105 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1105 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0183] The camera assembly 1106 is used to acquire images or videos. In some embodiments, the camera assembly 1106 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.

[0184] The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1101 for processing, or input to the radio frequency circuit 1104 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 1100. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1107 may also include a headphone jack.

[0185] Power supply 1108 is used to power the various components in terminal 1100. Power supply 1108 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1108 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0186] In some embodiments, the terminal 1100 further includes one or more sensors 1109. The one or more sensors 1109 include, but are not limited to: an acceleration sensor 1110, a gyroscope sensor 1111, a pressure sensor 1112, an optical sensor 1113, and a proximity sensor 1114.

[0187] Accelerometer 1110 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established with terminal 1100. For example, accelerometer 1110 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1101 can control display screen 1105 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1110. Accelerometer 1110 can also be used for games or for acquiring user motion data.

[0188] The gyroscope sensor 1111 can detect the orientation and rotation angle of the terminal 1100. The gyroscope sensor 1111 can work in conjunction with the accelerometer sensor 1110 to collect the user's 3D movements on the terminal 1100. Based on the data collected by the gyroscope sensor 1111, the processor 1111 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0189] The pressure sensor 1112 can be disposed on the side bezel of the terminal 1100 and / or on the lower layer of the display screen 1105. When the pressure sensor 1112 is disposed on the side bezel of the terminal 1100, it can detect the user's grip signal on the terminal 1100, and the processor 1101 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1112. When the pressure sensor 1112 is disposed on the lower layer of the display screen 1105, the processor 1101 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1105. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0190] An optical sensor 1113 is used to collect ambient light intensity. In one embodiment, the processor 1101 can control the display brightness of the display screen 1105 based on the ambient light intensity collected by the optical sensor 1113. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1105 is increased; when the ambient light intensity is low, the display brightness of the display screen 1105 is decreased. In another embodiment, the processor 1101 can also dynamically adjust the shooting parameters of the camera assembly 1106 based on the ambient light intensity collected by the optical sensor 1113.

[0191] The proximity sensor 1114, also known as a distance sensor, is typically located on the front panel of the terminal 1100. The proximity sensor 1114 is used to detect the distance between the user and the front of the terminal 1100. In one embodiment, when the proximity sensor 1114 detects that the distance between the user and the front of the terminal 1100 is gradually decreasing, the processor 1101 controls the display screen 1105 to switch from a screen-on state to a screen-off state; when the proximity sensor 1104 detects that the distance between the user and the front of the terminal 1100 is gradually increasing, the processor 1101 controls the display screen 1105 to switch from a screen-off state to a screen-on state.

[0192] Those skilled in the art will understand that the structure shown in FIG11 does not constitute a limitation on terminal 1100, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0193] Figure 12 is a schematic diagram of a server structure provided in an embodiment of this application. The server 1200 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 1201 and one or more memories 1202. The memory 1202 stores at least one computer program, which is loaded and executed by the processor 1201 to implement the intelligent customer service implementation method provided in the above-described method embodiments. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0194] This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the intelligent customer service method described in the above embodiments.

[0195] This application also provides a computer program product, including a computer program that is loaded and executed by a processor to implement the intelligent customer service method as described in the above embodiments.

[0196] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0197] The above are merely optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present application should be included within the protection scope of the present application.

Claims

1. A method for implementing intelligent customer service, characterized in that, The method includes: for any business within the domain of intelligent customer service, preprocessing the business process data of the business to obtain business process text and business process table, wherein the business process data represents the processing flow of the business; processing the business process text and business process table using a multimodal large model based on prompt information to obtain a structured flowchart of the business, wherein the prompt information guides the multimodal large model to generate the structured flowchart of the business based on the business process text and business process table, wherein the structured flowchart represents multiple entities in the business and the relationships between the multiple entities; and, when the intelligent customer service receives a user's question text, responding to the question text using a large language model based on the structured flowchart of multiple businesses within the domain of intelligent customer service to obtain a response text for the question text, and feeding back the response text to the user.

2. The intelligent customer service implementation method according to claim 1, characterized in that, The business process data is stored in document form. For any business within the intelligent customer service domain, the business process data is preprocessed to obtain business process text and business process tables. This includes: for any business within the intelligent customer service domain, the business process data is segmented into multiple pages, each displaying a portion of the business process data; page recognition is performed on the multiple pages to extract the business process text and business process tables; for any page, if a business process table exists on both the current page and the next page, it is determined whether the business process table on the current page is a complete table based on the bottom pixel position of the business process table on the current page and the top pixel position of the business process table on the next page; if the business process table on the current page is a complete table, it is determined to be the business process table for the business; if the business process table on the current page is not a complete table, the business process table on the current page and the business process table on the next page are concatenated to obtain the business process table for the business.

3. The intelligent customer service implementation method according to claim 2, characterized in that, For any page, if a business process table exists on both the current page and the next page, determining whether the business process table on the current page is a complete table based on the bottom pixel position of the business process table on the current page and the top pixel position of the business process table on the next page includes: for any page, if the bottom pixel position of the business process table on the current page is less than a first threshold, determining that the business process table on the current page is a complete table; if the bottom pixel position of the business process table on the current page is not less than the first threshold, and the top pixel position of the business process table on the next page is greater than a second threshold, determining that the business process table on the current page is a complete table; if the bottom pixel position of the business process table on the current page is not less than the first threshold, and the top pixel position of the business process table on the next page is not greater than the second threshold, determining that the business process table on the current page is not a complete table.

4. The intelligent customer service implementation method according to claim 1, characterized in that, The prompting information includes multiple prompt words. The step of processing the business process text and business process table using a multimodal large model based on the prompting information to obtain a structured flowchart of the business includes: processing the business process text and business process table using a multimodal large model based on a first prompt word to obtain multiple business process steps of the business; the first prompt word guides the multimodal large model to extract the business process steps of the business from the business process text and business process table; and processing the multiple business process steps, the business process table, and the... The business process text is processed to obtain a description of the business processing procedure. The second prompt word guides the multimodal model to generate the business processing procedure description based on the multiple business process steps, the business process table, and the business process text. The processing procedure description includes the dependencies between the multiple business process steps and the detailed content of each business process step. Through the multimodal model, the processing procedure description is processed according to the third prompt word to obtain a structured flowchart of the business. The third prompt word guides the multimodal model to generate the structured flowchart of the business based on the processing procedure description.

5. The intelligent customer service implementation method according to claim 4, characterized in that, The step of processing the processing procedure description using the multimodal large model and based on the third prompt word to obtain a structured flowchart of the business includes: extracting multiple entities of the business and the relationships between the multiple entities from the processing procedure description using the multimodal large model and based on the third prompt word; and generating a structured flowchart of the business using the multiple entities as nodes and the relationships between the multiple entities as edges using the multimodal large model and based on the third prompt word.

6. The intelligent customer service implementation method according to claim 1, characterized in that, The step of responding to a user's question text using a large language model based on structured flowcharts of multiple business processes within the domain of the intelligent customer service system, to obtain a response text, includes: first, using the large language model to identify the user's intent in the question text, indicating the content the user wants to know or operate; second, using the large language model to query at least one structured flowchart related to the intent from the structured flowcharts of multiple business processes within the domain; and third, using the large language model to generate a response text based on the at least one structured flowchart related to the intent.

7. The intelligent customer service implementation method according to claim 6, characterized in that, The step of generating a response text for the question text based on at least one structured flowchart related to the question intent includes: for any structured flowchart, extracting flowchart information related to the question intent from the structured flowchart, wherein the flowchart information includes at least one of the entities and relationships between entities in the structured flowchart; integrating the flowchart information of the at least one structured flowchart based on the type of the entity; and generating a response text for the question text based on the integrated flowchart information.

8. The intelligent customer service implementation method according to claim 1, characterized in that, The method further includes: for any entity in the structured flowchart, binding the entity to a business file or business link that the entity needs to access; merging the structured flowcharts of at least one business that is related to the business logic of multiple businesses in the domain; and storing the merged structured flowcharts in the form of graph data.

9. An intelligent customer service implementation device, characterized in that, The device includes: a processing module, configured to preprocess the business process data of any business within the domain of intelligent customer service to obtain business process text and a business process table, wherein the business process data represents the processing flow of the business; a construction module, configured to process the business process text and business process table using a multimodal large model based on prompting information to obtain a structured flowchart of the business, wherein the prompting information guides the multimodal large model to generate the structured flowchart of the business based on the business process text and the business process table, wherein the structured flowchart represents multiple entities in the business and the relationships between the multiple entities; and a generation module, configured to, when the intelligent customer service receives a user's question text, respond to the question text using a large language model based on the structured flowcharts of multiple businesses within the domain of intelligent customer service to obtain a response text for the question text, and then return the response text to the user.

10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to implement the intelligent customer service method as described in any one of claims 1 to 8.

Images