Information processing system and method

Abstract

Embodiments of the present specification provide an information processing system and method, wherein the information processing system comprises a server, a user terminal and a customer service terminal; the customer service terminal is configured to receive consultation information sent by the user terminal for a consultation event, and send the consultation information to the server; the server is configured to use a query model to screen target semantic structured data matching the consultation information in a customer service knowledge base, input target knowledge information corresponding to the consultation information and the target semantic structured data into an information processing model for processing to obtain customer service reference information, and send the customer service reference information to the customer service terminal; wherein the semantic structured data in the customer service knowledge base is generated by performing multi-dimensional semantic analysis on at least one type of multi-modal raw data respectively using a semantic analysis model; and the customer service terminal is configured to send target feedback information to the user terminal based on the customer service reference information.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to information processing systems and methods. Background Technology

[0002] With the rapid development of internet technology, customer service has become a crucial aspect for businesses to maintain customer relationships, enhance brand image, and strengthen market competitiveness. Currently, customer service relies entirely on manual processing. When receiving a user's inquiry, customer service personnel need to understand the user's intent, then search for relevant data. After obtaining the data, they manually filter, integrate, and edit it to produce a response for the user. However, manual searching and matching is often time-consuming, resulting in long waiting times for users and severely impacting user experience. Furthermore, it is highly susceptible to errors due to human negligence. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0003] In view of this, embodiments of this specification provide an information processing system. One or more embodiments of this specification also relate to information processing methods, another information processing system, an information processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, in order to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, an information processing system is provided, including a server, a user terminal, and a customer service terminal; The customer service terminal is used to receive inquiry information sent by the user terminal in response to an inquiry event, and to send the inquiry information to the server. The server is used to use a query model to filter target semantic structured data that matches the consultation information in the customer service knowledge base, input the consultation information and the target knowledge information corresponding to the target semantic structured data into an information processing model for processing to obtain customer service reference information, and send the customer service reference information to the customer service terminal; wherein, the semantic structured data in the customer service knowledge base is generated by using a semantic parsing model to perform multi-dimensional semantic parsing on at least one type of multimodal raw data respectively; The customer service terminal is used to send target feedback information to the user terminal based on the customer service reference information.

[0005] According to a second aspect of the embodiments of this specification, an information processing method is provided, applied to a server, comprising: Receive inquiry information sent from customer service terminals; The query model is used to filter target semantic structured data that matches the consultation information in the customer service knowledge base. The semantic structured data in the customer service knowledge base is generated by multi-dimensional semantic parsing of at least one type of multimodal raw data using a semantic parsing model. The consultation information and the target knowledge information corresponding to the target semantic structured data are input into the information processing model for processing to obtain customer service reference information; The customer service reference information is sent to the customer service terminal.

[0006] According to a third aspect of the embodiments of this specification, another information processing system is provided, including a terminal device and a server. The terminal device is used to send consultation information regarding the consultation event to the server; The server is used to use a query model to filter target semantic structured data that matches the consultation information in the target knowledge base, input the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing, obtain feedback information, and send the feedback information to the terminal device; wherein, the semantic structured data in the target knowledge base is generated by multi-dimensional semantic parsing of at least one type of multimodal raw data according to the semantic parsing model.

[0007] According to a fourth aspect of the embodiments of this specification, an information processing apparatus is provided, applied to a server, comprising: The receiving module is configured to receive inquiry information sent by the customer service terminal; The matching module is configured to use a query model to filter target semantic structured data that matches the consultation information in the customer service knowledge base, wherein the semantic structured data in the customer service knowledge base is generated by multi-dimensional semantic parsing of at least one type of multimodal raw data using a semantic parsing model. The generation module is configured to input the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing to obtain customer service reference information; The sending module is configured to send the customer service reference information to the customer service terminal.

[0008] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described information processing method.

[0009] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the information processing method described above.

[0010] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the information processing method described above.

[0011] The information processing system provided in this embodiment includes a server, a user terminal, and a customer service terminal. First, the customer service terminal receives consultation information sent by the user terminal regarding a consultation event, and then sends the consultation information to the server. The server, in response to the consultation information, uses a query model to filter target semantic structured data matching the consultation information from the customer service knowledge base. The server then inputs the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing, obtaining customer service reference information. This allows the client to provide feedback to the user based on the customer service reference information, improving the accuracy of the feedback. For the semantic structured data in the customer service knowledge base, a semantic parsing model can be used to perform multi-dimensional semantic parsing on various types of raw data, significantly improving the richness of the data included in the customer service knowledge base and resulting in higher quality customer service reference information generated based on the customer service knowledge base. Finally, the customer service terminal sends target feedback information to the user terminal based on the customer service reference information. Using the information processing system provided in this embodiment to process consultation information not only improves the efficiency of feedback information generation but also enhances the accuracy and comprehensiveness of the feedback information, thereby greatly improving the user's interactive experience. Attached Figure Description

[0012] Figure 1 This is a flowchart of an information processing system provided in one embodiment of this specification; Figure 2a This is a flowchart illustrating the processing procedure of an information processing system according to one embodiment of this specification; Figure 2b This is a flowchart illustrating an information processing method provided in one embodiment of this specification; Figure 3 This is a schematic diagram of the structure of another information processing system provided in one embodiment of this specification; Figure 4 This is a schematic diagram of the structure of an information processing device provided in one embodiment of this specification; Figure 5 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0013] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0014] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0015] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0016] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0017] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0018] Large Language Model (LLM) is a deep learning model in the field of artificial intelligence. It is mainly based on the Transformer architecture and is pre-trained on massive amounts of text data to enable it to understand, generate and process human language.

[0019] This specification provides an information processing system, and also relates to an information processing method, another information processing system, an information processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0020] See Figure 1 , Figure 1 A flowchart of an information processing system according to an embodiment of this specification is shown. The information processing system includes a server, a user terminal, and a customer service terminal, and specifically includes the following steps.

[0021] Step 102: The customer service terminal receives the consultation information sent by the user terminal in response to the consultation event, and sends the consultation information to the server.

[0022] The information processing system provided in this embodiment can be applied to any scenario requiring information consultation. For example, it can be applied to the e-commerce field for pre-sales, after-sales, or product detail information consultation scenarios, and also to the transportation field (e.g., inquiring about air tickets, train tickets, or flight rescheduling status), etc. In other words, the information processing system provided in this embodiment can be used for any scenario requiring customer service consultation. Relevant technical personnel can determine the application scenario based on the actual situation; this embodiment does not impose any limitations.

[0023] Specifically, an information processing system refers to a system capable of processing inquiry information. An information processing system includes multiple devices that can transmit data between each other, and may include server-side, user terminals, and customer service terminals. A server-side system refers to computer equipment or a cluster of computer equipment used to process inquiry information and obtain customer service reference information. A user terminal refers to the terminal device that initiates the inquiry request; a user terminal can be a smartphone, tablet, personal computer (PC), smart wearable device, etc. A customer service terminal refers to the work platform or device used by customer service personnel for question-and-answer interaction with users. An inquiry event refers to an event corresponding to a question requiring inquiry; for example, an inquiry event could be a product details inquiry, a pre-sales or after-sales inquiry, or a rights and benefits inquiry, etc. Inquiry information refers to the specific question a user needs to inquire about, such as asking about the price, color, etc., of a product.

[0024] Based on this, the user terminal can send inquiry information regarding the inquiry event to the customer service terminal. After receiving the inquiry information, the customer service terminal can forward it to the server. This allows the server to further process the inquiry information. Optionally, after receiving the inquiry information, the customer service terminal can also adjust it. For example, if there are typos or missing punctuation in the inquiry information, it can be corrected before being sent to the server. This processing flow can utilize a large language model; that is, the inquiry information received from the customer service terminal is input into the large language model, which performs semantic understanding to generate higher-quality inquiry information. This higher-quality inquiry information is then sent to the server.

[0025] Step 104: The server uses a query model to filter target semantic structured data that matches the consultation information in the customer service knowledge base.

[0026] Specifically, semantically structured data refers to semantic blocks obtained after processing various types of raw data; this data is composed of text. Target semantically structured data refers to the semantically structured data in the customer service knowledge base that matches the consultation information. The customer service knowledge base is a database composed of semantic blocks corresponding to various types of raw data, with each semantic block having a mapping relationship with the raw data. Although the data stored in the customer service knowledge base is in text form, it contains rich data content because it is generated based on semantic parsing of various types of data. The query model can be a large language model, used to match consultation information with the semantically structured data in the customer service knowledge base and filter to obtain the target semantically structured data. The semantically structured data in the customer service knowledge base is generated by using a semantic parsing model to perform multi-dimensional semantic parsing on at least one type of multimodal raw data.

[0027] Based on this, the server can pre-build a customer service knowledge base for matching with consultation information, serving as the basis for generating customer service reference information. Furthermore, the customer service knowledge base can be updated in real time. An update cycle can be preset, and upon reaching the update cycle, corresponding semantic structured data is generated based on newly acquired data of various types and stored in the customer service knowledge base, thus enabling the update operation of the customer service knowledge base.

[0028] After receiving an inquiry, the server can use a query model to search the customer service knowledge base for matching semantically structured data. The matching semantically structured data is then used as the target semantically structured data. Specifically, vector matching can be used. The query model generates a corresponding inquiry information vector based on the inquiry information, and then calculates the similarity between the inquiry information vector and the semantically structured data in the customer service knowledge base. Semantically structured data with a similarity exceeding a similarity threshold is used as the target semantically structured data. Alternatively, the query model can first extract keywords from the inquiry information, and then use these keywords to determine the target semantically structured data in the customer service knowledge base.

[0029] Furthermore, the data type can be determined, and corresponding semantic structured data can be generated based on the semantic parsing strategy corresponding to the data type to construct a customer service knowledge base. In this embodiment, the specific implementation is as follows: The server acquires multimodal raw data and determines the target type corresponding to the multimodal raw data; using the semantic parsing model, it performs multi-dimensional semantic parsing on the multimodal raw data according to the target semantic parsing strategy corresponding to the target type, generates semantic structured data corresponding to the multimodal raw data, and stores it in the customer service knowledge base.

[0030] Specifically, multimodal raw data refers to data of various types, such as tabular data, text data, image data, audio data, video data, etc. The target type refers to the data type corresponding to the multimodal raw data; for example, the target type can be text, image, audio, video, etc. The target semantic parsing strategy refers to the data processing strategy corresponding to the target type, used to generate corresponding semantically structured data based on the raw data. Multidimensional semantic parsing refers to the semantic parsing processing methods applied to multimodal raw data from different dimensions. For example, when the multimodal raw data is image data, text recognition, key visual feature extraction, and other dimension-based image processing can be performed. These various dimension-based processing methods constitute multidimensional semantic parsing.

[0031] Based on this, after the server obtains the multimodal raw data, it determines the data type corresponding to the raw data, which is the target type. Then, it determines the target semantic parsing strategy corresponding to the target type. Specifically, semantic parsing model prompts can be pre-set, specifying the semantic parsing strategy for each data type within these prompts. Using the semantic parsing model, the target semantic parsing strategy corresponding to the target type is determined within the semantic parsing prompt template. Then, multi-dimensional semantic parsing is performed on the multimodal raw data according to the target semantic parsing strategy for the target type, generating semantically structured data corresponding to the multimodal raw data. This semantically structured data is then stored in the customer service knowledge base. Furthermore, data address information corresponding to the multimodal raw data can be added to the semantically structured data to achieve the effect of tracing the source based on the semantically structured data.

[0032] For example, if the original multimodal data is image data, then the image data is processed based on the image semantic parsing strategy to obtain the semantic structured data of the image data. If the original multimodal data is table data, then the table data is processed based on the table semantic parsing strategy to obtain the corresponding semantic structured data of the table data.

[0033] In summary, by setting different semantic parsing strategies for different types of multimodal raw data, the customer service knowledge base can include information corresponding to various types of data, significantly improving the richness of the content in the customer service knowledge base, and thus making the customer service reference information generated based on the semantically structured data in the customer service knowledge base of higher quality.

[0034] Furthermore, when the original multimodal data is in the form of images or videos, semantically structured data with high information density is generated based on the corresponding semantic parsing strategy. In this embodiment, the specific implementation is as follows: When the multimodal raw data is an image or video, the server inputs the multimodal raw data into a semantic parsing model, which includes a text recognition module, a visual feature extraction module, and an image information fusion module. If the multimodal raw data is a video, it is determined by extracting keyframes from the video raw data. The text recognition module performs text recognition processing on the multimodal raw data to obtain image text information. The visual feature extraction module extracts visual dimension features from the multimodal raw data and performs visual dimension semantic parsing processing based on the visual dimension feature extraction results to obtain visual text information. The image information fusion module fuses the image text information and the visual text information to generate semantically structured data corresponding to the multimodal raw data and stores it in the customer service knowledge base.

[0035] Specifically, the text recognition module refers to the module that performs text recognition processing on images. The visual feature extraction module refers to the module that extracts visual features from images. The image information fusion module refers to the module that fuses image text information and visual text information determined based on visual features; that is, this module performs text fusion processing on multiple types of text information corresponding to an image. Raw video data refers to multimodal data of video type. Video keyframes refer to video frames in the raw video data that contain many features. These video keyframes can be obtained by extracting multiple video frames from the video based on a preset frequency, and then filtering from these multiple video frames to obtain video keyframes containing many features. Text recognition processing refers to the processing of recognizing text in image data. Image text information refers to the text content contained in the image obtained by text recognition processing. Visual dimension feature extraction refers to the processing method of extracting features from images based on visual dimensions. For example, it can identify prominent information in image data and then extract features from this prominent information, which may include product identification, product color, product selling points, etc. Visual dimension semantic parsing processing refers to the processing method of semantic parsing based on visual dimension features to generate corresponding text content. Visual text information refers to text content generated based on visual dimensional features. Fusion processing refers to the process of integrating image text information and visual text information. Fusion processing can be a simple physical overlay, or it can perform semantic understanding on image text information and visual text information to readjust the content and generate new text information containing key information.

[0036] Based on this, when the multimodal raw data is video, keyframes are extracted from the video raw data, and then these keyframes are processed subsequently, serving as the raw image data. To determine video keyframes, a sequence of video frames can be extracted from the raw video data at a preset frequency. Then, an image recognition model can be used to identify each video frame in the sequence, identifying those that meet the criteria for rich image information and high image clarity as keyframes, which are then used as the raw image data. When the multimodal raw data is image-based, the image raw data is processed directly. For ease of explanation, multimodal raw data of either image or video type will be referred to as image raw data.

[0037] The server inputs the raw image data into a semantic parsing model, which includes a text recognition module, a visual feature extraction module, and an image information fusion module. These three modules work together to obtain semantically structured data corresponding to the raw image data. The text recognition module in the semantic parsing model performs text recognition processing on the raw image data to obtain image text information, that is, extracting the text content from the raw image data. The visual feature extraction module performs visual dimension feature extraction on the raw image data, and performs visual dimension semantic parsing processing based on the results to obtain visual text information. In other words, key content of the raw image data can be analyzed from a visual dimension perspective. For example, when the raw image data is a product image, visual dimension feature extraction can determine the color features, shape features, and selling point features of the raw image data (selling points can be determined based on information that occupies a large proportion of the raw image data), etc. Finally, the image information fusion module is used to fuse the image text information and visual text information. This fusion process can be a simple physical overlay or a semantic understanding of the image text information and visual text information. The content is then readjusted to generate new text information containing key content. Based on the result of the fusion process, semantic structured data corresponding to the original image data is generated and stored in the customer service knowledge base.

[0038] For example, the original image data is a product image of a yoga mat, and the image text information includes information such as the price and thickness of the yoga mat. The visual text information is that the yoga mat is thicker than regular yoga mats on the market. The final generated semantic structured data is: yoga mat price, thickness, etc., and information about the yoga mat being thicker than regular yoga mats on the market.

[0039] In summary, multi-dimensional semantic analysis can be performed on raw image data, thereby enabling the semantically structured data corresponding to the raw image data to contain rich information content.

[0040] Furthermore, when the original image data consists of multiple related images, comparative analysis can be performed. In this embodiment, the specific implementation is as follows: When there are multiple sets of multimodal raw data, the semantic parsing model further includes a comparison analysis module. The server uses the comparison analysis module to compare the image text information corresponding to the multiple sets of multimodal raw data, and / or to compare the visual text information corresponding to the multiple sets of multimodal raw data, and generates attribute feature comparison information based on the comparison results. The image information fusion module uses the attribute feature comparison information to update the semantic structured data corresponding to the multiple sets of multimodal raw data, and stores the updated semantic structured data in the customer service knowledge base.

[0041] Specifically, the comparative analysis module refers to the module that performs comparative analysis on multiple multimodal raw data. Attribute feature comparison information refers to information obtained by comparing one or more attribute features, which can include the similarities, differences, advantages, or disadvantages of each multimodal raw data relative to one or more attributes.

[0042] Based on this, the multimodal raw data consists of multiple data sets, meaning there are multiple image raw data sets with related relationships. These relationships could be the same product produced by different vendors, or other relationships, which can be determined by relevant technical personnel according to the actual situation. This embodiment does not impose any limitations on this. In addition to the text recognition module, visual feature extraction module, and image information fusion module, the semantic parsing model may also include a comparison analysis module. On the server side, the comparison analysis module in the semantic parsing model compares the image text information corresponding to the multiple image raw data sets with related relationships, and / or compares the visual text information corresponding to the multiple image raw data sets, generating attribute feature comparison information based on the comparison results. Specifically, the image text information corresponding to each image raw data set generated by the text recognition module can be compared to obtain image text information comparison results, and the visual text information corresponding to each image raw data set generated by the visual feature extraction module can also be compared to obtain visual text information comparison results. Either of these two comparison results can be generated, or both can be generated. Relevant technical personnel can determine the processing method according to the actual situation; this embodiment does not impose any limitations on this. In other words, the attribute feature comparison information can be the image-text information comparison result, the visual-text information comparison result, or the result of fusing the image-text information comparison result and the visual-text information comparison result. Finally, the image information fusion module is used to update the semantic structured data corresponding to the original image data with the obtained attribute feature comparison information. That is, the attribute feature comparison information is added to the original semantic structured data to obtain the updated semantic structured data, and the updated semantic structured data is stored in the customer service knowledge base.

[0043] For example, the original data for multiple images consists of two images of yoga mats. The attribute feature comparison information can include comparison information related to color, pattern texture, size ratio, surface material gloss, etc.

[0044] In summary, the semantic structured data corresponding to the original image data also includes comparative analysis results of other related original image data, further enhancing the information richness of the semantic structured data corresponding to the original image data.

[0045] Furthermore, when the original multimodal data is in tabular form, semantically structured data with high information density is generated based on the corresponding semantic parsing strategy. In this embodiment, the specific implementation is as follows: When the multimodal raw data is in tabular form, the server inputs the multimodal raw data into the semantic parsing model, wherein the semantic parsing model includes a table processing module and a table information fusion module; using the table processing module, the multimodal raw data is standardized to obtain structured table data, and key features are extracted from the multimodal raw data to obtain table summary information; using the table information fusion module, semantic structured data corresponding to the multimodal raw data is generated based on the structured table data and the table summary information and stored in the customer service knowledge base.

[0046] Specifically, the table processing module refers to the module that processes the raw table data, including format standardization and table summary extraction. The table information fusion module integrates information from various dimensions of the table. Table format standardization refers to the processing of the raw table data to standardize its format. Key feature extraction refers to the extraction of key information from the table; this can be summarizing key column attributes or performing data calculations and summaries based on the table data. Structured table data refers to the data corresponding to the raw table data after format standardization. Table summary information refers to the information generated after key feature extraction from the raw table data; this can be the original text content of the table or information obtained after calculating and summarizing all or part of the data in the table content.

[0047] Based on this, the semantic parsing model can include a table processing module and a table information fusion module. When the multimodal raw data is in tabular form, the server inputs this raw data into the semantic parsing model. The table processing module within the model standardizes the table format to obtain structured table data and extracts key features to generate table summary information. This can involve summarizing key column attributes, performing data calculations and summaries based on the table data, and so on. Different key feature extraction methods can generate different table summaries; for example, they can be the original text content of the table or information obtained after calculating and summarizing all or part of the data in the table content. Finally, the table information fusion module physically integrates the structured table data and the table summary information to obtain the semantically structured data corresponding to the original table data, which is then stored in the customer service knowledge base.

[0048] For example, the original table data can be converted into a standard Markdown table to obtain structured table data. Three key data points can be extracted from the table data to obtain table summary information. The table summary information can then be added to the structured table data to form semantic structured data corresponding to the original table data.

[0049] In summary, the semantic structured data corresponding to the original table data also includes table summary information, thereby enhancing the richness of the semantic structured data corresponding to the original table data.

[0050] Furthermore, when the original multimodal data is text or audio, semantically structured data with high information density is generated based on the corresponding semantic parsing strategy. In this embodiment, the specific implementation is as follows: When the multimodal raw data is audio or text, the server uses the semantic parsing model to perform text format standardization processing on the multimodal raw data to obtain semantic structured data, and stores the semantic structured data in the customer service knowledge base; wherein, when the multimodal raw data is audio, the multimodal raw data is determined by audio recognition. Specifically, text formatting standard processing refers to the processing methods used to standardize the format of text.

[0051] Therefore, when the original multimodal data is audio, audio recognition is performed to obtain the corresponding text. The recognized text may be inaccurate at this point, so a large language model can be used to further refine it, resulting in more accurate text information. This more accurate text information is then used as the original text data. Alternatively, when the original multimodal data is text, it is processed directly. For ease of explanation, both text-type and audio-type multimodal data will be referred to as original text data.

[0052] On the server side, a semantic parsing model is used to standardize the original text data to obtain semantically structured data, which is then stored in the customer service knowledge base. Text format standardization may include one or more of the following: removing irrelevant characters (emoticons, special punctuation), correcting typos, standardizing terminology, time standardization, and structured reorganization (converting information into a computer-readable format such as JSON). Specific text format standardization procedures can be determined by relevant technical personnel based on actual circumstances; this embodiment does not impose any limitations.

[0053] For example, replace words describing a yoga mat, such as "mat" or "yoga blanket," in the original text data with the standard entity "yoga mat."

[0054] In summary, audio data can be identified to obtain corresponding text content. Furthermore, text format standardization processing of the original text data makes the generated semantic structured data more accurate.

[0055] Step 106: The server inputs the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing to obtain customer service reference information, and sends the customer service reference information to the customer service terminal.

[0056] Specifically, the information processing model can be a large language model, and the target knowledge model is used to integrate the target semantic structured data to obtain customer service reference information. Target knowledge information refers to the knowledge information corresponding to the target semantic structured data. Target knowledge information can be the target semantic structured data itself, or it can be the target semantic structured data and its corresponding raw data—that is, the data on which the semantic structured data was generated. Customer service reference information refers to information used to provide reference for customer service staff. Customer service reference information can include information stating objective facts, as well as suggestive information (for example, when the inquiry information is product details, a recommendation to purchase can be generated) to make the information richer and more comprehensive.

[0057] Based on this, after determining the target semantic structured data, the server can input the target semantic structured data and consultation information into the information processing model. The information processing model performs semantic understanding and integration processing on the target semantic structured data to obtain customer service reference information. It should be noted that multiple customer service reference messages can be generated; for example, they can be customer service reference messages that state facts, or customer service reference messages that provide suggestions, etc. Furthermore, the target semantic structured data may also include the address of the original data. In the event of missing data in the target semantic structured data, the corresponding original data can be retrieved based on the address. Then, semantic understanding and integration processing can be performed based on the target semantic structured data and the corresponding original data to obtain customer service reference information. In this case, the knowledge information corresponding to the target semantic structured data includes the target semantic structured data and the corresponding original data.

[0058] Furthermore, when the server generates customer service reference information using the information processing model, this reference information can include not only factual text information but also extended text information. In this embodiment, the specific implementation is as follows: The server uses an information processing model to integrate and expand the consultation information and the target knowledge information corresponding to the target semantic structured data based on information processing prompts. It determines factual text information and expanded text information based on the information integration results, and sends the factual text information and expanded text information to the customer service terminal as customer service reference information.

[0059] Specifically, information processing prompts refer to the prompts corresponding to the information processing model. These prompts may include specific information integration steps to guide the model in processing information. Information integration processing refers to the process of summarizing and analyzing target knowledge information corresponding to one or more target semantic structured data. Information expansion processing refers to the process of expanding upon the target knowledge information corresponding to the target semantic structured data; for example, opinions or suggestions can be generated based on target knowledge information. Factual text information refers to factual text content obtained by summarizing objective facts; for example, the various attributes and specifications of a product are factual text information. Extended text information refers to extended text content obtained by further expanding upon objective facts; for example, generating text content related to whether a product is recommended for purchase or suitable for a specific group based on its various attributes.

[0060] Based on this, the server utilizes an information processing model to integrate and expand the target knowledge information corresponding to the consultation information and target semantic structured data according to information processing prompts. Based on the information integration results, factual text information is generated, that is, text content with factual basis is generated by analyzing and summarizing one or more target knowledge information. Furthermore, extended text information is generated based on the information expansion results, that is, information is further expanded on the objective facts. For example, for a specific product, based on the product's various attributes, corresponding applicable population information is generated, or information such as whether to recommend purchasing it is generated. Finally, the factual and extended text information are sent to the customer service terminal as reference information.

[0061] For example, factual text information: The yoga mat is 6mm thick. Extended text information: Suitable for beginners or people with sensitive knees.

[0062] In summary, customer service reference information not only includes factual text information, but also provides related extended text information, thereby significantly improving the richness of customer service reference information.

[0063] Step 108: The customer service terminal sends target feedback information to the user terminal based on the customer service reference information.

[0064] Specifically, target feedback information is information generated by the customer service terminal based on customer service reference information and sent to the user terminal as a response to the inquiry.

[0065] Based on this, after receiving customer service reference information, the customer service terminal can evaluate it, for example, by assessing its format, sensitive information, and tone, to determine whether the format is standardized, whether it contains sensitive information that is inappropriate to send to the user, or whether the semantics are appropriate. Then, based on the evaluation results, the customer service reference information is adjusted to generate target feedback information and sent to the user terminal. If the evaluation results of all dimensions of the customer service reference information meet the feedback conditions, the customer service terminal can send the customer service reference information to the user terminal.

[0066] Furthermore, when the customer service reference information includes both factual and extended text information, the customer service representative can generate targeted feedback information based on these two types of information. In this embodiment, the specific implementation is as follows: The customer service terminal sends target feedback information to the user terminal based on the factual text information and the extended text information.

[0067] Based on this, the customer service terminal generates target feedback information based on factual and extended text information, and sends this target feedback information to the user terminal. Specifically, a large language model can be used to perform semantic understanding and integration processing on the factual and extended text information to obtain the target feedback information.

[0068] For example, the target feedback information is: Because this yoga mat has an ideal thickness of 6mm, it is especially suitable for beginners or people with sensitive knees. When performing kneeling poses (such as cat-cow pose, low lunge), it provides excellent cushioning and protection, effectively reducing joint stress.

[0069] In summary, this approach can further enhance the richness of target feedback information, thereby significantly improving the user's interactive experience.

[0070] Furthermore, after receiving the customer service reference information, the customer service terminal can determine whether the reference information contains sensitive user information, and obtain the target feedback information based on the determination result. In this embodiment, the specific implementation method is as follows: The customer service terminal determines whether the customer service reference information contains sensitive user information; if so, it performs desensitization processing on the customer service reference information according to the desensitization rules, and sends target feedback information to the user terminal based on the desensitization processing result; if not, it sends target feedback information to the user terminal based on the customer service reference information.

[0071] Specifically, sensitive user information refers to information that needs to be kept confidential from users. De-identification rules are predefined, standardized processing logic based on data type and security level policies. They specify how to irreversibly or partially reversibly mask, replace, or generalize different types of sensitive information to ensure that the original sensitive content cannot be directly reproduced when the information is displayed to the user, while retaining a certain degree of readability or business context. De-identification processing refers to the processing methods adopted according to the de-identification rules.

[0072] Based on this, the customer service terminal determines whether the customer service reference information contains sensitive user information. If so, it performs desensitization processing on the customer service reference information according to the desensitization rules and sends the target feedback information to the user terminal based on the desensitization processing result. If not, it sends the target feedback information to the user terminal based on the customer service reference information. Desensitization processing can include various implementation methods. For example, it can perform masking processing, replacing characters in a specific area with other specific symbols. Alternatively, it can use generalization processing, converting precise values ​​into range values ​​or category values ​​to reduce precision. In addition, it can also use replacement or direct deletion methods for desensitization processing, etc. In actual processing, relevant technical personnel can determine the corresponding desensitization processing method according to the actual situation. This embodiment does not impose any limitations here.

[0073] For example, the customer service reference information includes product pricing information. This product pricing information is deleted, and then the target feedback information is sent to the user's terminal based on the remaining content in the customer service reference information.

[0074] In summary, further examining the information provided by customer service representatives to determine if it contains sensitive user content and taking appropriate action is beneficial for ensuring data security.

[0075] The information processing system provided in this embodiment includes a server, a user terminal, and a customer service terminal. First, the customer service terminal receives consultation information sent by the user terminal regarding a consultation event, and then sends the consultation information to the server. The server, in response to the consultation information, uses a query model to filter target semantic structured data matching the consultation information from the customer service knowledge base. The server then inputs the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing, obtaining customer service reference information. This allows the client to provide feedback to the user based on the customer service reference information, improving the accuracy of the feedback. For the semantic structured data in the customer service knowledge base, a semantic parsing model can be used to perform multi-dimensional semantic parsing on various types of raw data, significantly improving the richness of the data included in the customer service knowledge base and resulting in higher quality customer service reference information generated based on the customer service knowledge base. Finally, the customer service terminal sends target feedback information to the user terminal based on the customer service reference information. Using the information processing system provided in this embodiment to process consultation information not only improves the efficiency of feedback information generation but also enhances the accuracy and comprehensiveness of the feedback information, thereby greatly improving the user's interactive experience.

[0076] The following is in conjunction with the appendix Figure 2a Taking the application of the information processing method provided in this specification in the e-commerce field as an example, the information processing method will be further explained. Figure 2a A flowchart illustrating the processing steps of an information processing method provided in one embodiment of this specification is shown, specifically including the following steps.

[0077] Step 212: The server obtains the multimodal raw data and determines the target type corresponding to the multimodal raw data.

[0078] Step 214: The server uses the semantic parsing model to perform multi-dimensional semantic parsing on the multimodal raw data according to the target semantic parsing strategy corresponding to the target type, generates semantic structured data corresponding to the multimodal raw data, and stores it in the customer service knowledge base.

[0079] Specifically, when the multimodal raw data is an image or video, the server inputs the multimodal raw data into a semantic parsing model. This semantic parsing model includes a text recognition module, a visual feature extraction module, and an image information fusion module. If the multimodal raw data is a video, it is determined by extracting keyframes from the video raw data. The text recognition module performs text recognition processing on the multimodal raw data to obtain image text information. The visual feature extraction module extracts visual dimension features from the multimodal raw data and performs visual dimension semantic parsing processing based on the visual dimension feature extraction results to obtain visual text information. The image information fusion module fuses the image text information and the visual text information to generate semantically structured data corresponding to the multimodal raw data and stores it in the customer service knowledge base.

[0080] When there are multiple sets of multimodal raw data, the semantic parsing model further includes a comparison analysis module. The server uses the comparison analysis module to compare the image text information corresponding to the multiple sets of multimodal raw data, and / or to compare the visual text information corresponding to the multiple sets of multimodal raw data, and generates attribute feature comparison information based on the comparison results. The image information fusion module uses the attribute feature comparison information to update the semantic structured data corresponding to the multiple sets of multimodal raw data, and stores the updated semantic structured data in the customer service knowledge base.

[0081] When the multimodal raw data is in tabular form, the server inputs the multimodal raw data into the semantic parsing model, wherein the semantic parsing model includes a table processing module and a table information fusion module; using the table processing module, the multimodal raw data is standardized to obtain structured table data, and key features are extracted from the multimodal raw data to obtain table summary information; using the table information fusion module, semantic structured data corresponding to the multimodal raw data is generated based on the structured table data and the table summary information and stored in the customer service knowledge base.

[0082] When the multimodal raw data is audio or text, the server uses the semantic parsing model to perform text format standardization processing on the multimodal raw data to obtain semantic structured data, and stores the semantic structured data in the customer service knowledge base; wherein, when the multimodal raw data is audio, the multimodal raw data is determined by audio recognition audio raw data.

[0083] Step 216: The customer service terminal receives the consultation information sent by the user terminal in response to the consultation event, and sends the consultation information to the server.

[0084] Step 218: The server uses a query model to filter target semantic structured data that matches the consultation information in the customer service knowledge base.

[0085] Step 220: The server inputs the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing to obtain customer service reference information, and sends the customer service reference information to the customer service terminal.

[0086] The server uses an information processing model to integrate and expand the consultation information and the target knowledge information corresponding to the target semantic structured data based on information processing prompts. It determines factual text information and expanded text information based on the information integration results, and sends the factual text information and expanded text information to the customer service terminal as customer service reference information.

[0087] Step 222: The customer service terminal determines whether the customer service reference information contains sensitive user information.

[0088] Step 224: If so, perform desensitization processing on the customer service reference information according to the desensitization rules, and send the target feedback information to the user terminal based on the desensitization processing result.

[0089] Step 226: If not, send target feedback information to the user terminal based on the customer service reference information.

[0090] The information processing system provided in this embodiment includes a server, a user terminal, and a customer service terminal. First, the customer service terminal receives consultation information sent by the user terminal regarding a consultation event, and then sends the consultation information to the server. The server, in response to the consultation information, uses a query model to filter target semantic structured data matching the consultation information from the customer service knowledge base. The server then inputs the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing, obtaining customer service reference information. This allows the client to provide feedback to the user based on the customer service reference information, improving the accuracy of the feedback. For the semantic structured data in the customer service knowledge base, a semantic parsing model can be used to perform multi-dimensional semantic parsing on various types of raw data, significantly improving the richness of the data included in the customer service knowledge base and resulting in higher quality customer service reference information generated based on the customer service knowledge base. Finally, the customer service terminal sends target feedback information to the user terminal based on the customer service reference information. Using the information processing system provided in this embodiment to process consultation information not only improves the efficiency of feedback information generation but also enhances the accuracy and comprehensiveness of the feedback information, thereby greatly improving the user's interactive experience.

[0091] Corresponding to the above system embodiments, this specification also provides an embodiment of an information processing method. Figure 2b A flowchart of an information processing method according to an embodiment of this specification is shown. This method is applied to a server. Specifically, it includes the following steps.

[0092] Step 202: Receive inquiry information sent by the customer service terminal.

[0093] Step 204: Use a query model to filter target semantic structured data that matches the consultation information in the customer service knowledge base.

[0094] The semantically structured data in the customer service knowledge base is generated by multi-dimensional semantic parsing of at least one type of multimodal raw data using a semantic parsing model.

[0095] Step 206: Input the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing to obtain customer service reference information.

[0096] Step 208: Send the customer service reference information to the customer service terminal.

[0097] In one optional embodiment, multimodal raw data is acquired, and the target type corresponding to the multimodal raw data is determined; Using the semantic parsing model, multi-dimensional semantic parsing is performed on the multimodal raw data according to the target semantic parsing strategy corresponding to the target type, generating semantic structured data corresponding to the multimodal raw data and storing it in the customer service knowledge base.

[0098] In an optional embodiment, if the multimodal raw data is an image or a video, the multimodal raw data is input into a semantic parsing model, wherein the semantic parsing model includes a text recognition module, a visual feature extraction module, and an image information fusion module; if the multimodal raw data is a video, the multimodal raw data is determined by extracting video keyframes from the video raw data. The text recognition module is used to perform text recognition processing on the multimodal raw data to obtain image text information; The visual feature extraction module is used to extract visual dimension features from the multimodal raw data, and visual dimension semantic parsing is performed based on the visual dimension feature extraction results to obtain visual text information. The image information fusion module is used to fuse the image text information and the visual text information to generate semantic structured data corresponding to the multimodal raw data and store it in the customer service knowledge base.

[0099] In an optional embodiment, when there are multiple multimodal raw data, the semantic parsing model further includes a comparison analysis module, which compares the image text information corresponding to the multiple multimodal raw data respectively, and / or compares the visual text information corresponding to the multiple multimodal raw data respectively, and generates attribute feature comparison information based on the comparison results; Using the image information fusion module, the semantic structured data corresponding to the multiple multimodal raw data are updated according to the attribute feature comparison information, and the updated semantic structured data is stored in the customer service knowledge base.

[0100] In an optional embodiment, if the multimodal raw data is in the form of a table, the multimodal raw data is input into the semantic parsing model, wherein the semantic parsing model includes a table processing module and a table information fusion module; Using the table processing module, the multimodal raw data is standardized to obtain structured table data, and key features are extracted from the multimodal raw data to obtain table summary information; Using the table information fusion module, semantic structured data corresponding to the multimodal raw data is generated based on the structured table data and the table summary information, and then stored in the customer service knowledge base.

[0101] In an optional embodiment, if the multimodal raw data is of audio or text type, the semantic parsing model is used to perform text format standardization processing on the multimodal raw data to obtain semantic structured data, and the semantic structured data is stored in the customer service knowledge base; In the case where the multimodal raw data is of audio type, the multimodal raw data is determined by audio recognition audio raw data.

[0102] In one optional embodiment, an information processing model is used to integrate and expand the consultation information and the target knowledge information corresponding to the target semantic structured data based on information processing prompts. Factual text information is determined based on the information integration results, and extended text information is determined based on the information expansion results. The factual text information and the extended text information are then sent to the customer service terminal as customer service reference information.

[0103] The information processing method provided in this embodiment is applied to the server side. For inquiry information, a query model is used to filter target semantic structured data matching the inquiry information from the customer service knowledge base. The inquiry information and the target knowledge information corresponding to the target semantic structured data are then input into the information processing model for processing to obtain customer service reference information. This allows the client to provide feedback to the user based on the customer service reference information, improving the accuracy of the feedback. For the semantic structured data in the customer service knowledge base, a semantic parsing model can be used to perform multi-dimensional semantic parsing on various types of raw data, significantly improving the richness of the data included in the customer service knowledge base and resulting in higher quality customer service reference information generated based on the customer service knowledge base. Using the information processing method provided in this embodiment to process inquiry information not only improves the efficiency of feedback information generation but also enhances the accuracy and comprehensiveness of the feedback information, thereby greatly improving the user's interactive experience.

[0104] Figure 3 A schematic diagram of another information processing system according to an embodiment of this specification is shown. The information processing system 300 includes a terminal device 310 and a server 320. The terminal device 310 is used to send consultation information for a consultation event to the server. The server 320 is used to use a query model to filter target semantic structured data matching the consultation information in a target knowledge base, input the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing, obtain feedback information, and send the feedback information to the terminal device 310. The semantic structured data in the target knowledge base is generated by multi-dimensional semantic parsing of at least one type of multimodal raw data according to a semantic parsing model.

[0105] The server 320 is configured to: when the consultation information is sent by a customer service terminal, use a query model to filter target semantic structured data matching the consultation information in the customer service knowledge base; when the consultation information is sent by a user terminal, use a query model to filter target semantic structured data matching the consultation information in the user knowledge base; wherein the semantic structured data in the customer service knowledge base and the user knowledge base are isolated from each other.

[0106] Based on this, another information processing system provided in this embodiment may include a terminal device and a server, and the terminal device and the server can transmit data. The terminal device can be a customer service terminal or a user terminal. It should be noted that the knowledge base accessed by the server is also different depending on the terminal device. When the terminal device is a user terminal, the knowledge base accessed by the server is the user knowledge base. When the terminal device is a customer service terminal, the knowledge base accessed by the server is the customer service knowledge base. The semantically structured data in the customer service knowledge base and the user knowledge base are isolated from each other. This ensures that the confidentiality of information obtained by different personnel is different, further maintaining data security. The construction methods of the user knowledge base and the customer service knowledge base can refer to the construction method of the customer service knowledge base described above, and the method of the server generating feedback information can refer to the method of the server generating customer service reference information described above. This embodiment will not repeat it here.

[0107] The information processing system provided in this embodiment includes a server and a terminal device. First, the terminal device sends consultation information to the server. The server, in response to the consultation information, uses a query model to filter target semantic structured data matching the consultation information from the corresponding target knowledge base. The server then inputs the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing, obtaining feedback information, and sending the feedback information back to the terminal device. By using the information processing system provided in this embodiment to process consultation information, the server can access knowledge bases with different security levels based on the source of the consultation information, ensuring data security. Furthermore, it can improve the efficiency, accuracy, and comprehensiveness of feedback information generation, thereby greatly enhancing the user's interactive experience.

[0108] Corresponding to the above method embodiments, this specification also provides embodiments of an information processing device applied to a server. Figure 4 A schematic diagram of the structure of an information processing apparatus according to one embodiment of this specification is shown. Figure 4 As shown, the device includes: The receiving module 402 is configured to receive inquiry information sent by the customer service terminal; The matching module 404 is configured to use a query model to filter target semantic structured data that matches the consultation information in the customer service knowledge base, wherein the semantic structured data in the customer service knowledge base is generated by multi-dimensional semantic parsing of at least one type of multimodal raw data using a semantic parsing model. The generation module 406 is configured to input the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing to obtain customer service reference information; The sending module 408 is configured to send the customer service reference information to the customer service terminal.

[0109] In one optional embodiment, multimodal raw data is acquired, and the target type corresponding to the multimodal raw data is determined; Using the semantic parsing model, multi-dimensional semantic parsing is performed on the multimodal raw data according to the target semantic parsing strategy corresponding to the target type, generating semantic structured data corresponding to the multimodal raw data and storing it in the customer service knowledge base.

[0110] In an optional embodiment, if the multimodal raw data is an image or a video, the multimodal raw data is input into a semantic parsing model, wherein the semantic parsing model includes a text recognition module, a visual feature extraction module, and an image information fusion module; if the multimodal raw data is a video, the multimodal raw data is determined by extracting video keyframes from the video raw data. The text recognition module is used to perform text recognition processing on the multimodal raw data to obtain image text information; The visual feature extraction module is used to extract visual dimension features from the multimodal raw data, and visual dimension semantic parsing is performed based on the visual dimension feature extraction results to obtain visual text information. The image information fusion module is used to fuse the image text information and the visual text information to generate semantic structured data corresponding to the multimodal raw data and store it in the customer service knowledge base.

[0111] In an optional embodiment, when there are multiple multimodal raw data, the semantic parsing model further includes a comparison analysis module, which compares the image text information corresponding to the multiple multimodal raw data respectively, and / or compares the visual text information corresponding to the multiple multimodal raw data respectively, and generates attribute feature comparison information based on the comparison results; Using the image information fusion module, the semantic structured data corresponding to the multiple multimodal raw data are updated according to the attribute feature comparison information, and the updated semantic structured data is stored in the customer service knowledge base.

[0112] In an optional embodiment, if the multimodal raw data is in the form of a table, the multimodal raw data is input into the semantic parsing model, wherein the semantic parsing model includes a table processing module and a table information fusion module; Using the table processing module, the multimodal raw data is standardized to obtain structured table data, and key features are extracted from the multimodal raw data to obtain table summary information; Using the table information fusion module, semantic structured data corresponding to the multimodal raw data is generated based on the structured table data and the table summary information, and then stored in the customer service knowledge base.

[0113] In an optional embodiment, if the multimodal raw data is of audio or text type, the semantic parsing model is used to perform text format standardization processing on the multimodal raw data to obtain semantic structured data, and the semantic structured data is stored in the customer service knowledge base; In the case where the multimodal raw data is of audio type, the multimodal raw data is determined by audio recognition audio raw data.

[0114] In one optional embodiment, an information processing model is used to integrate and expand the consultation information and the target knowledge information corresponding to the target semantic structured data based on information processing prompts. Factual text information is determined based on the information integration results, and extended text information is determined based on the information expansion results. The factual text information and the extended text information are then sent to the customer service terminal as customer service reference information.

[0115] This specification provides an information processing apparatus according to one embodiment, applied to a server. For inquiry information, a query model is used to filter target semantic structured data matching the inquiry information from a customer service knowledge base. The inquiry information and the target knowledge information corresponding to the target semantic structured data are then input into an information processing model for processing to obtain customer service reference information. This allows the client to provide feedback to the user based on the customer service reference information, improving the accuracy of the feedback. For the semantic structured data in the customer service knowledge base, a semantic parsing model can be used to perform multi-dimensional semantic parsing on various types of raw data, significantly improving the richness of the data included in the customer service knowledge base and resulting in higher quality customer service reference information generated based on the customer service knowledge base. Using the information processing apparatus provided in this embodiment to process inquiry information not only improves the efficiency of feedback information generation but also enhances the accuracy and comprehensiveness of the feedback information, thereby greatly improving the user's interactive experience.

[0116] The above is an illustrative scheme of an information processing device according to this embodiment. It should be noted that the technical solution of this information processing device and the technical solution of the information processing method described above belong to the same concept. For details not described in detail in the technical solution of the information processing device, please refer to the description of the technical solution of the information processing method described above.

[0117] Figure 5 A structural block diagram of a computing device 500 according to one embodiment of this specification is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0118] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0119] In one embodiment of this specification, the above-described components of the computing device 500 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0120] The computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 500 can also be a mobile or stationary server.

[0121] The processor 520 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described information processing method.

[0122] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the information processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the information processing method described above.

[0123] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described information processing method.

[0124] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the information processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the information processing method described above.

[0125] An embodiment of this specification also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described information processing method.

[0126] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the information processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the information processing method described above.

[0127] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0128] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0129] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0130] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0131] The preferred embodiments disclosed above are merely illustrative of this specification. Optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described in this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification.

Claims

1. An information processing system, characterized in that, This includes server-side components, user terminals, and customer service terminals. The customer service terminal is used to receive inquiry information sent by the user terminal in response to an inquiry event, and to send the inquiry information to the server. The server is used to use a query model to filter target semantic structured data that matches the consultation information in the customer service knowledge base, input the consultation information and the target knowledge information corresponding to the target semantic structured data into an information processing model for processing to obtain customer service reference information, and send the customer service reference information to the customer service terminal; wherein, the semantic structured data in the customer service knowledge base is generated by using a semantic parsing model to perform multi-dimensional semantic parsing on at least one type of multimodal raw data respectively; The customer service terminal is used to send target feedback information to the user terminal based on the customer service reference information.

2. The information processing system according to claim 1, characterized in that, The server is also used for: Acquire multimodal raw data and determine the target type corresponding to the multimodal raw data; Using the semantic parsing model, multi-dimensional semantic parsing is performed on the multimodal raw data according to the target semantic parsing strategy corresponding to the target type, generating semantic structured data corresponding to the multimodal raw data and storing it in the customer service knowledge base.

3. The information processing system according to claim 2, characterized in that, When the multimodal raw data is an image or video, the server is further configured to: The multimodal raw data is input into a semantic parsing model, wherein the semantic parsing model includes a text recognition module, a visual feature extraction module, and an image information fusion module. When the multimodal raw data is a video, the multimodal raw data is determined by extracting key frames from the video raw data. The text recognition module is used to perform text recognition processing on the multimodal raw data to obtain image text information; The visual feature extraction module is used to extract visual dimension features from the multimodal raw data, and visual dimension semantic parsing is performed based on the visual dimension feature extraction results to obtain visual text information. The image information fusion module is used to fuse the image text information and the visual text information to generate semantic structured data corresponding to the multimodal raw data and store it in the customer service knowledge base.

4. The information processing system according to claim 3, characterized in that, When there are multiple sets of multimodal raw data, the semantic parsing model further includes a comparative analysis module, and the server is also used for: Using the comparison analysis module, image text information corresponding to multiple multimodal raw data is compared, and / or visual text information corresponding to multiple multimodal raw data is compared, and attribute feature comparison information is generated based on the comparison results; Using the image information fusion module, the semantic structured data corresponding to the multiple multimodal raw data are updated according to the attribute feature comparison information, and the updated semantic structured data is stored in the customer service knowledge base.

5. The information processing system according to claim 2, characterized in that, When the multimodal raw data is of tabular type, the server is further configured to: The multimodal raw data is input into the semantic parsing model, wherein the semantic parsing model includes a table processing module and a table information fusion module; Using the table processing module, the multimodal raw data is standardized to obtain structured table data, and key features are extracted from the multimodal raw data to obtain table summary information; Using the table information fusion module, semantic structured data corresponding to the multimodal raw data is generated based on the structured table data and the table summary information, and then stored in the customer service knowledge base.

6. The information processing system according to claim 2, characterized in that, When the multimodal raw data is audio or text, the server is further configured to: Using the semantic parsing model, the multimodal raw data is processed by text format standardization to obtain semantic structured data, and the semantic structured data is stored in the customer service knowledge base; In the case where the multimodal raw data is of audio type, the multimodal raw data is determined by audio recognition audio raw data.

7. The information processing system according to claim 1, characterized in that, The server is used for: Using an information processing model, the consultation information and the target knowledge information corresponding to the target semantic structured data are integrated and expanded based on information processing prompts. Factual text information is determined based on the information integration results, and extended text information is determined based on the information expansion results. The factual text information and the extended text information are then sent to the customer service terminal as customer service reference information. The customer service terminal is used for: Based on the factual text information and the extended text information, target feedback information is sent to the user terminal.

8. The information processing system according to claim 1, characterized in that, The customer service terminal is used for: Determine whether the customer service reference information contains sensitive user information; If so, the customer service reference information is desensitized according to the desensitization rules, and the target feedback information is sent to the user terminal based on the desensitization result; If not, send the target feedback information to the user terminal based on the customer service reference information.

9. An information processing method, characterized in that, Applied to the server side, including: Receive inquiry information sent from customer service terminals; The query model is used to filter target semantic structured data that matches the consultation information in the customer service knowledge base. The semantic structured data in the customer service knowledge base is generated by multi-dimensional semantic parsing of at least one type of multimodal raw data using a semantic parsing model. The consultation information and the target knowledge information corresponding to the target semantic structured data are input into the information processing model for processing to obtain customer service reference information; The customer service reference information is sent to the customer service terminal.

10. An information processing system, characterized in that, Including terminal devices and server-side components; The terminal device is used to send consultation information regarding the consultation event to the server; The server is used to use a query model to filter target semantic structured data that matches the consultation information in the target knowledge base, input the consultation information and the target knowledge information corresponding to the target semantic structured data into the information processing model for processing, obtain feedback information, and send the feedback information to the terminal device; wherein, the semantic structured data in the target knowledge base is generated by multi-dimensional semantic parsing of at least one type of multimodal raw data according to the semantic parsing model.

11. The information processing system according to claim 10, characterized in that, The server is used for: When the consultation information is sent by a customer service terminal, a query model is used to filter target semantic structured data that matches the consultation information in the customer service knowledge base. When the consultation information is sent by a user terminal, a query model is used to filter target semantic structured data that matches the consultation information from the user knowledge base; The semantically structured data in the customer service knowledge base and the user knowledge base are isolated from each other.

12. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method of claim 9.

13. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method of claim 9.

14. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the steps of the method of claim 9.

Images