Large model data processing method and device, equipment and storage medium

Abstract

The invention provides a large model data processing method and device, equipment and a storage medium, and relates to the technical field of large language models and data processing in artificial intelligence. According to the specific implementation scheme, a prompt word set is determined based on a preset knowledge base in response to a query request of a user, whether a first query result corresponding to any prompt word in the prompt word set exists or not is queried in a single-point Redis cache according to the prompt word set, and the prompt words without the query result in the single-point Redis cache are queried in the single-point Redis cache according to the query result. And sending the first query result to the large model service through the shared memory channel, generating a corresponding second query result, and feeding back the first query result and the second query result to the user. Through the method, the query request of the user is efficiently and comprehensively responded, the response time of the large model is shortened, the processing pressure of the large model service is relieved, unnecessary computing resource consumption is avoided, and the processing efficiency of the whole system is improved. Decoupling between the components is achieved through the shared memory channel.

Description

Technical Field

[0001] The present disclosure relates to the field of large language models and data processing technology in artificial intelligence, and in particular to a large model data processing method, device, equipment and storage medium. Background Art

[0002] The big model is a large language model that is trained based on a deep learning algorithm and large amounts of training data. It has excellent generalization and expression capabilities and can be widely used in image recognition, natural language processing, speech recognition, and other fields. However, as big models are increasingly used in various fields, their efficiency in processing query requests has gradually become prominent, becoming one of the key factors affecting user experience.

[0003] In the prior art, the response speed to user requests is improved by optimizing the parameters of the large model and increasing service resources.

[0004] However, the above method requires a lot of machine resources, and it is very difficult to optimize the parameters of large models, and it is not feasible for large-scale promotion. Therefore, how to reduce the response time of large model services and improve user experience is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present disclosure provides a method, apparatus, device and storage medium for large model data processing.

[0006] According to a first aspect of the present disclosure, a large model data processing method is provided, comprising:

[0007] In response to a user's query request, determining a prompt word set based on a preset knowledge base, wherein the prompt word set includes at least two prompt words, each prompt word is used to represent the user's query requirement, and the knowledge base includes a plurality of prompt words;

[0008] According to the prompt word set, querying in a single-point Redis cache whether there is a first query result corresponding to any prompt word in the prompt word set;

[0009] Sending the prompt word for the query result not existing in the single-point Redis cache to the large model service through the shared memory channel to generate a corresponding second query result;

[0010] The first query result and the second query result are fed back to the user.

[0011] According to a second aspect of the present disclosure, there is provided a large model data processing device, comprising:

[0012] a determination unit, configured to determine, in response to a user's query request, a prompt word set based on a preset knowledge base, wherein the prompt word set includes at least two prompt words, each prompt word is used to represent the user's query requirement, and the knowledge base includes a plurality of prompt words;

[0013] A query unit, configured to query, according to the prompt word set, whether there is a first query result corresponding to any prompt word in the prompt word set in a single-point Redis cache;

[0014] A generating unit, configured to send a prompt word indicating that there is no query result in the single-point Redis cache to the large model service through a shared memory channel to generate a corresponding second query result;

[0015] A feedback unit is used to feed back the first query result and the second query result to the user.

[0016] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in any one aspect.

[0020] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in any one of the aspects.

[0021] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in any one aspect.

[0022] The technology disclosed herein solves the problem of how to reduce the response time of large model services and improve user experience.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0025] Figure 1 is a system architecture diagram that can implement the large model data processing method of the embodiment of the present disclosure;

[0026] Figure 2 A schematic flow chart of a large model data processing method provided in the first embodiment of the present disclosure;

[0027] Figure 3 A schematic diagram of a flow chart of a large model data processing method provided in the second embodiment of the present disclosure;

[0028] Figure 4 A schematic diagram of a flow chart of a large model data processing method provided in the third embodiment of the present disclosure;

[0029] Figure 5 A schematic diagram of a flow chart of a large model data processing method provided in a fourth embodiment of the present disclosure;

[0030] Figure 6 A flowchart of a large model data processing method provided in the fifth embodiment of the present disclosure;

[0031] Figure 7 A flowchart of a large model data processing method provided in a sixth embodiment of the present disclosure;

[0032] Figure 8 A structural block diagram of a large model data processing device provided in an embodiment of the present disclosure;

[0033] Fig. 9 A schematic block diagram of an electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0035] In the related art, a big model is a large language model, which is based on a deep learning algorithm and is obtained through large-scale training data training. It has excellent generalization and expression capabilities and can be widely used in image recognition, natural language processing, speech recognition and other fields. However, with the increasing application of big models in various fields, the efficiency problem of processing query requests has gradually become prominent, becoming one of the key factors affecting user experience. In the prior art, the response speed to user requests is improved by optimizing the parameters of the big model and increasing service resources. However, the above method requires a lot of machine resources, and it is very difficult to optimize the parameters of the big model, and it is not feasible for large-scale promotion. Therefore, how to reduce the response time of the big model service and improve the user experience is a technical problem that needs to be solved urgently.

[0036] In response to this existing problem, the technical conception process of the present invention is as follows: In the process of studying the large model, the inventor found that the large model is inefficient in processing user requests, so it takes a long response time to give the user a response result. If the service resources are increased, although the response speed of the large model can be improved, it takes a lot of machine resources and the cost is high. If the parameters of the large model are optimized to improve the response speed, the complexity of the large model is high, and it is difficult to optimize the parameters, which has limitations for large-scale promotion. Therefore, the inventor considers whether a central cache service can be configured and implemented by a single-point Redis, so that the historical response data processed by the large model can be stored in the single-point Redis cache, and then when the user requests, the corresponding response information stored in the cache can be directly fed back to the user, which not only improves the response speed, but also improves the user experience. At the same time, the self-developed disk drop service is adopted, that is, each data written to the single-point Redis cache will be written to another kafka service, so that the data is not lost.

[0037] Figure 1 It is a system architecture diagram that can implement the big model data processing method of the embodiment of the present disclosure. As shown in Question 1, the system architecture 100 includes a cache center service 101, a big model service 102, a kafka data disk service 103, a data preprocessing service 104 and a knowledge base 105, wherein the cache center service 101 and the big model service 102 are connected via shared memory communication, the cache center service 101 is connected via communication with the kafka data disk service 103, the cache center service 101 is connected via communication with the data preprocessing service 104, and the data preprocessing service 104 is connected via communication with the knowledge base 105.

[0038] The data preprocessing service 104 performs data conversion in response to the user's query request, converts the query request into a prompt word, and then expands the prompt word through the knowledge base 105 to obtain a prompt word set. Based on the prompt word set, the data preprocessing service 104 determines whether there is a corresponding query result in the cache center service 101, and sends the prompt word that has no corresponding query result to the big model service 102 for reasoning to generate a query result. The big model service 102 stores the generated query result in the cache center service 101. The cache center service 101 sends the prompt word combined with all corresponding query results to the data preprocessing service 104. The data preprocessing service 104 feeds back the query result to the user. The cache center service 101 can also write the query results stored in the big model service 102 to the kafka data disk service 103, thereby ensuring that no data is lost in the event of an exception.

[0039] Optionally, the cache center service 101 and the large model service 102 are connected via shared memory communication. In this case, the cache center service 101 and the large model service 102 need to be deployed on the same physical machine or virtual machine. The Kafka data disk service 103, the data preprocessing service 104 and the knowledge base 105 can be independent devices or can be in the same device, without specific limitation.

[0040] The cache center service 101 and the Kafka data storage service 103 communicate through the Kafka message queue. The cache center service 101 sends data to Kafka in the form of messages, and the Kafka data storage service consumes and stores these messages.

[0041] Exemplarily, the cache center service 101 and the data preprocessing service 104 can achieve data communication through an API interface, such as HTTP / HTTPS REST, and can also achieve communication through gRPC or other RPC frameworks.

[0042] The data preprocessing service 104 and the knowledge base 105 can select a suitable communication method according to the data form in the knowledge base. For example, if the knowledge base 105 is stored in the form of a database (such as MySQL, MongoDB, etc.), SQL or NoSQL query statements can be used directly to query data. The data preprocessing service 104 interacts with the knowledge base through a database connection pool or an ORM (object-relational mapping) framework. If the knowledge base 105 exists in the form of a service, the data preprocessing service 104 can communicate with the knowledge base 105 through HTTP requests.

[0043] The present invention discloses a large model data processing method, device, equipment and storage medium, which are applied to the large language model and data processing technology fields in artificial intelligence. The method stores the historically processed data of the large model based on the central cache service, and then when the user requests, the response data in the cache can be directly fed back to the user, thereby reducing the response time of the large model service and improving the user experience.

[0044] The following specific embodiments are used to describe in detail the technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0045] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0046] The specific implementation scheme of the large model data processing method provided by the present invention is introduced below.

[0047] Figure 2 A flowchart of a large model data processing method provided in the first embodiment of the present disclosure is shown as follows: Figure 2 The large model data processing method shown can be configured as a large model data processing device, which can be located in an electronic device. The large model data processing method can include the following steps:

[0048] S201: In response to a user's query request, a prompt word set is determined based on a preset knowledge base.

[0049] In this step, in response to the user's query request, the data preprocessing service receives the user's query request, and then performs data conversion on the query request to convert it into at least one prompt word. In order to increase the diversity of the query and thus more accurately obtain the query result corresponding to the query request, at least one prompt word is expanded based on a preset knowledge base to obtain a prompt word set, wherein the prompt word set includes at least two prompt words, each prompt word is used to characterize the user's query requirement, and the knowledge base includes multiple prompt words.

[0050] For example, the user's query request is: What are the effective methods to prevent children from becoming addicted? Then the prompt word set can be determined as: {children's Internet addiction, youth protection, game time limit, strategy, technique, measure}.

[0051] S202: According to the prompt word set, query in a single-point Redis cache whether there is a first query result corresponding to any prompt word in the prompt word set.

[0052] In this step, after the prompt word set is determined, in order to reduce the response time of the large model, it is possible to query whether each prompt word in the prompt word set has a corresponding first query result in the single-point Redis cache.

[0053] If any prompt word in the prompt word set has a corresponding first query result in the single-point Redis cache, the prompt word will not be repeatedly inferred. If any prompt word in the prompt word set does not have a corresponding first query result in the single-point Redis cache, the second query result can be obtained by inference based on the large model, thereby reducing the response time of the large model.

[0054] Exemplarily, the prompt word set is {prompt word 1, prompt word 2, prompt word 3, prompt word 4}. After querying the single-point Redis cache, if the first query results of prompt word 1 and prompt word 2 exist in the single-point Redis cache: {result 1, result 2, result 3}, then there is no need to repeatedly infer the query results of prompt word 1 and prompt word 2.

[0055] S203: Send the prompt word for which no query result exists in the single-point Redis cache to the large model service through the shared memory channel to generate a corresponding second query result.

[0056] S204: Feedback the first query result and the second query result to the user.

[0057] If any prompt word in the prompt word set does not have a corresponding first query result in the single-point Redis cache, in order to meet user needs and reduce the response speed of the large model, the prompt word that does not have a corresponding first query result can be sent to the large model service through the shared memory channel, and then the large model service infers it to obtain the corresponding second query result. Then, the second query result is sent to the single-point Redis cache through the shared memory channel, and the single-point Redis cache feeds back the first query result and the second query result to the user.

[0058] Exemplarily, the prompt word set is {prompt word 1, prompt word 2, prompt word 3, prompt word 4}. After querying the single-point Redis cache, if the first query results of prompt word 1 and prompt word 2 exist in the single-point Redis cache: {result 1, result 2, result 3}, prompt word 3 and prompt word 4 are sent to the big model service, and the big model service generates the second query result {result 4, result 5, result 6}. In order to be able to respond to the user's query request more comprehensively, the first query result and the second query result need to be processed and fed back to the user.

[0059] Exemplarily, since different prompt words may query similar or identical information, it is necessary to perform deduplication processing on these results. At the same time, results with similar content but different expressions can be merged to reduce information redundancy. The results can also be sorted according to factors such as the relevance and timeliness of the query results. Usually, the most relevant and latest results to the user's query request are placed in front so that the user can find the required information faster. For longer query results, summaries or summaries can be generated so that users can quickly understand the main content of the results. This helps users quickly browse multiple results and find the most interesting parts. In addition to text forms, you can also consider using visualization methods such as charts, pictures, and videos to present query results. It helps users understand information more intuitively and improves the readability and attractiveness of information. When feeding back query results, user interaction functions such as search suggestions and related query recommendations can also be provided. It helps users further explore relevant information based on current results and improves query efficiency and user experience.

[0060] The large model data processing method provided in this embodiment responds to the user's query request, determines the prompt word set based on the pre-set knowledge base, queries whether there is a first query result corresponding to any prompt word in the prompt word set in the single-point Redis cache according to the prompt word set, sends the prompt word for which there is no query result in the single-point Redis cache to the large model service through the shared memory channel, generates the corresponding second query result, and feeds back the first query result and the second query result to the user. Combining the fast access capability of the single-point Redis cache and the deep processing capability of the large model service, the user's query request can be responded to efficiently and comprehensively, reducing the large model response time, alleviating the processing pressure of the large model service, avoiding unnecessary consumption of computing resources, and improving the processing efficiency of the overall system. Decoupling between components is achieved through the shared memory channel.

[0061] Figure 3 This is a flow chart of a large model data processing method provided in the second embodiment of the present disclosure, refer to Figure 3 The large model data processing method shown is different from the aforementioned embodiment in that, in this embodiment, the second query result and the corresponding prompt word can also be cached in a single-point Redis cache through a shared memory channel.

[0062] like Figure 3 As shown, the method comprises the following steps:

[0063] S301: In response to a user's query request, a prompt word set is determined based on a preset knowledge base.

[0064] S302: According to the prompt word set, query in a single-point Redis cache whether there is a first query result corresponding to any prompt word in the prompt word set.

[0065] S303: Send the prompt word for the query result that does not exist in the single-point Redis cache to the large model service through the shared memory channel to generate a corresponding second query result.

[0066] The specific implementation of step S301 to step S303 is the same as that in the above-mentioned embodiment, and will not be repeated here.

[0067] S304: Cache the second query result and the corresponding prompt word in a single-point Redis cache through a shared memory channel.

[0068] In this step, in order to reduce the response time of the large model when any user makes the same or similar query request after the user's current query, the second query result and the corresponding prompt word inferred by the large model can be stored in a single-point Redis cache.

[0069] The large model caches the second query result and the corresponding prompt word in a single-point Redis cache through a shared memory channel.

[0070] Exemplarily, before storing the second query result in a single-point Redis cache, the cache data needs to be prepared. This includes: the prompt word itself or some form of its hash value can be used as the key. Ensuring the uniqueness of the key is key to avoid cache conflicts. The value is the second query result itself. Since Redis is a key-value storage system, if the query result is a complex data structure (such as a list, set, hash table, etc.), it needs to be properly serialized or encoded so that Redis can store and retrieve it. Once the cached data is prepared, this data can also be sent from the large model service to the component responsible for the Redis cache through a shared memory channel (such as a message queue, shared memory block, etc.). This ensures the reliable transmission of data and allows asynchronous operations between the large model service and the single-point Redis cache, improving the overall performance and scalability of the system. After the component responsible for the Redis cache receives the data from the shared memory channel, it will perform the following operations:

[0071] Parse the data: First parse out the prompt word (key) and the second query result (value).

[0072] Storing data: Use Redis commands such as SET, HSET (if the value is a hash table), and SADD (if the value is a set) to store the parsed data in Redis.

[0073] Optionally, you can also set an expiration time. According to actual needs, you can set an expiration time for the cached data to prevent expired data from occupying Redis memory space.

[0074] S305: Feedback the first query result and the second query result to the user.

[0075] The specific implementation of step S305 is the same as that in the above-mentioned embodiment, and will not be repeated here.

[0076] The large model data processing method provided in this embodiment responds to the user's query request, determines a prompt word set based on a pre-set knowledge base, queries whether there is a first query result corresponding to any prompt word in the prompt word set in the single-point Redis cache according to the prompt word set, sends the prompt word for which there is no query result in the single-point Redis cache to the large model service through a shared memory channel, generates a corresponding second query result, caches the second query result and the corresponding prompt word in the single-point Redis cache through a shared memory channel, and feeds back the first query result and the second query result to the user. For query requests that do not exist in the single-point Redis cache, they are sent to the large model service for processing through a shared memory channel. This process utilizes the high efficiency of shared memory, avoids transmission delays in traditional network communications, further improves processing speed, and improves the consistency of numbers, thereby improving the overall scalability of the system.

[0077] Figure 4 This is a flow chart of a large model data processing method provided in the third embodiment of the present disclosure, refer to Figure 4 The large model data processing method shown is different from the aforementioned embodiment in that, in this embodiment, the second query result and the corresponding prompt word can also be written into the Kafka service.

[0078] like Figure 4 As shown, the method comprises the following steps:

[0079] S401: In response to a user's query request, a prompt word set is determined based on a preset knowledge base.

[0080] S402: According to the prompt word set, query in a single-point Redis cache whether there is a first query result corresponding to any prompt word in the prompt word set.

[0081] S403: Send the prompt word for the query result that does not exist in the single-point Redis cache to the large model service through the shared memory channel to generate a corresponding second query result.

[0082] S404: Cache the second query result and the corresponding prompt word in a single-point Redis cache through a shared memory channel.

[0083] The specific implementation of step S401 to step S404 is the same as that in the above-mentioned embodiment, and will not be repeated here.

[0084] S405: Write the second query result and the corresponding prompt word into the Kafka service.

[0085] In this step, after the single-point Redis cache caches the second query result and the corresponding prompt word, in order to avoid data loss, a self-developed disk service is used, that is, each data written to Redis will be written to another Kafka service.

[0086] S406: Feedback the first query result and the second query result to the user.

[0087] The specific implementation of step S406 is the same as that in the above-mentioned embodiment, and will not be repeated here.

[0088] Optionally, after the single-point Redis cache is restarted, the single-point Redis cache can re-cache the data stored before the single-point Redis cache is restarted by consuming the data stored in the Kafka service.

[0089] In the above, when writing the single-point Redis cache to the Kafka service, the cached data will be lost after an exception occurs in the single-point Redis cache. In order to ensure the efficiency and user experience of the large model service, after the single-point Redis cache is restarted, the data stored in the Kafka service can be used to re-cache the data stored before the single-point Redis cache is restarted.

[0090] It should be noted that Kafka's persistence mechanisms (such as log files and replication mechanisms) can ensure that data will not be lost even if the Kafka service itself fails.

[0091] For example, a dedicated data recovery service or script can be created, which runs automatically after Redis is restarted and successfully started. The main task of this service or script is to connect to Kafka and consume the data related to the Redis cache previously stored in Kafka.

[0092] The data recovery service uses the Kafka consumer API to connect to the Kafka cluster and subscribes to the topic previously used to store Redis data.

[0093] The consumer starts reading messages from a specified offset in Kafka, which can be the offset of the last successful write to Kafka before Redis was restarted (usually this needs to be tracked in Redis or another storage system).

[0094] Each message read contains data (such as key-value pairs) that needs to be cached in Redis.

[0095] For each message read from Kafka, the data recovery service parses the key and value and stores them back into the Redis cache using the Redis API.

[0096] If the amount of data is large, this process may need to be performed in batches to avoid excessive write pressure on Redis at one time.

[0097] There is also ample monitoring and logging throughout the data recovery process to track recovery progress, detect potential problems, and quickly locate and resolve problems when they occur.

[0098] After the data recovery is complete, you can also verify that all data that should be cached in Redis has been correctly restored.

[0099] The large model data processing method provided in this embodiment responds to the user's query request, determines the prompt word set based on the pre-set knowledge base, queries whether there is a first query result corresponding to any prompt word in the prompt word set in the single-point Redis cache according to the prompt word set, sends the prompt word that does not exist in the query result in the single-point Redis cache to the large model service through the shared memory channel, generates the corresponding second query result, caches the second query result and the corresponding prompt word in the single-point Redis cache through the shared memory channel, writes the second query result and the corresponding prompt word into the kafka service, and feeds back the first query result and the second query result to the user. Through the above method, the query efficiency and response speed are improved, and the scalability and flexibility of the viewing system are enhanced. The second query result and the corresponding prompt word are cached in the Redis cache and written to the kafka service at the same time, realizing double backup of data. When the Redis cache fails, data can be recovered from kafka to ensure the reliability and consistency of the data and improve the user experience.

[0100] Figure 5 This is a flow chart of a large model data processing method provided in the fourth embodiment of the present disclosure, refer to Figure 5 The large model data processing method shown is different from the aforementioned embodiment in that, in this embodiment, in response to the user's query request, a user's prompt word set is determined based on a pre-set knowledge base, which can be refined as follows: the query request is converted into at least one first prompt word, for each first prompt word, a second prompt word corresponding to the first prompt word is determined in the knowledge base, and all the first prompt words and all the second prompt words are summed up to obtain a prompt word set.

[0101] like Figure 5 As shown, the method comprises the following steps:

[0102] S501: Convert a query request into at least one first prompt word.

[0103] In this step, in order to accurately provide the user with a query result, after receiving the user's query request, the query request is converted into data, thereby obtaining at least one first prompt word.

[0104] Exemplarily, it can be implemented by natural language processing (NLP) technology, such as word segmentation, part-of-speech tagging, named entity recognition, etc., to identify key concepts and entities in the query. Based on understanding the intent, the keywords in the query request can be extracted as the first prompt word.

[0105] For example, keywords can be extracted by analyzing the frequency, position, co-occurrence and other statistical information of words in the query text. Or, a predefined dictionary or thesaurus can be used to match the query text with the entries in the dictionary and extract the matching entries as keywords. Alternatively, semantic analysis technology can be used to understand the semantic relationship in the query text and extract more representative keywords.

[0106] S502: For each first prompt word, determine a second prompt word corresponding to the first prompt word in the knowledge base.

[0107] S503: Sum up all the first prompt words and all the second prompt words to obtain a prompt word set.

[0108] After obtaining at least one first prompt word, in order to improve the accuracy and recall rate of the query so that the user can obtain more comprehensive and more relevant query results, each prompt word is expanded to obtain a second prompt word corresponding to each first prompt word.

[0109] Specifically, a knowledge base is pre-set, and a large number of keywords are stored in the knowledge base, so that the similarity between each first prompt word and the keyword can be calculated, so as to determine the second prompt word, and finally the first prompt word and the second prompt word are summed up to obtain a prompt word set.

[0110] S504: According to the prompt word set, query in the single-point Redis cache whether there is a first query result corresponding to any prompt word in the prompt word set.

[0111] S505: Send the prompt word for the query result that does not exist in the single-point Redis cache to the large model service through the shared memory channel to generate a corresponding second query result.

[0112] S506: Feedback the first query result and the second query result to the user.

[0113] The specific implementation of step S504 to step S506 is the same as that in the above-mentioned embodiment, and will not be repeated here.

[0114] The large model data processing method provided in this embodiment converts the query request into at least one first prompt word, and for each first prompt word, determines the second prompt word corresponding to the first prompt word in the knowledge base, and sums up all the first prompt words and all the second prompt words to obtain a prompt word set. According to the prompt word set, it is queried in the single-point Redis cache whether there is a first query result corresponding to any prompt word in the prompt word set, and the prompt word for which there is no query result in the single-point Redis cache is sent to the large model service through the shared memory channel to generate the corresponding second query result, and the first query result and the second query result are fed back to the user. Through the above method, not only the accuracy and comprehensiveness of the query are improved, but also the query efficiency is optimized, the user experience is improved, and the response time of the large model is reduced.

[0115] Figure 6 This is a flow chart of a large model data processing method provided in the fifth embodiment of the present disclosure, refer to Figure 6 The large model data processing method shown is different from the aforementioned embodiment in that, in this embodiment, determining the second prompt word corresponding to the first prompt word in the knowledge base can be refined as follows: for each first prompt word, calculating the similarity between the first prompt word and each prompt word in the knowledge base, and determining the prompt word with a similarity greater than a pre-set similarity threshold as the second prompt word.

[0116] like Figure 6 As shown, the method comprises the following steps:

[0117] S601: Convert a query request into at least one first prompt word.

[0118] The specific implementation of step S601 is the same as that in the above-mentioned embodiment, and will not be repeated here.

[0119] S602: For each first prompt word, calculate the similarity between the first prompt word and each prompt word in the knowledge base.

[0120] S603: Determine the prompt word whose similarity is greater than a preset similarity threshold as the second prompt word.

[0121] In this step, in order to meet the diverse needs of users and improve the query coverage, the similarity between each first prompt word and the prompt words in the knowledge base is calculated, and then the prompt word with a similarity greater than a preset similarity threshold is determined as the second prompt word.

[0122] For example, the similarity calculation may be performed by cosine similarity calculation or Jaccard similarity calculation. The similarity threshold may be set to 80%, 90% or 95%, etc., without specific limitation.

[0123] Optionally, a text matching model may be used to determine the second prompt word.

[0124] S604: Sum up all the first prompt words and all the second prompt words to obtain a prompt word set.

[0125] S605: According to the prompt word set, query in the single-point Redis cache whether there is a first query result corresponding to any prompt word in the prompt word set.

[0126] S606: Send the prompt word for the query result that does not exist in the single-point Redis cache to the large model service through the shared memory channel to generate a corresponding second query result.

[0127] S607: Feedback the first query result and the second query result to the user.

[0128] The specific implementation of step S604 to step S607 is the same as that in the above-mentioned embodiment, and will not be repeated here.

[0129] The large model data processing method provided in this embodiment converts the query request into at least one first prompt word, calculates the similarity between the first prompt word and each prompt word in the knowledge base for each first prompt word, determines the prompt word with a similarity greater than a preset similarity threshold as the second prompt word, sums up all the first prompt words and all the second prompt words to obtain a prompt word set, and queries whether there is a first query result corresponding to any prompt word in the prompt word set in the single-point Redis cache according to the prompt word set, sends the prompt word for which there is no query result in the single-point Redis cache to the large model service through a shared memory channel, generates the corresponding second query result, and feeds back the first query result and the second query result to the user. This method optimizes query efficiency, accuracy and resource utilization and improves user experience by introducing technical means such as prompt word similarity calculation, single-point Redis cache, shared memory channel and large model service.

[0130] Figure 7 This is a flow chart of a large model data processing method provided in the sixth embodiment of the present disclosure, refer to Figure 7 The big model data processing method shown is different from the aforementioned embodiment in that, in this embodiment, generating the corresponding second query result can be refined as follows: for each prompt word for which there is no query result in the single-point Redis cache, the big model service performs inference calculation on the prompt word to generate a second query result.

[0131] like Figure 7 As shown, the method comprises the following steps:

[0132] S701: In response to a user's query request, a prompt word set is determined based on a preset knowledge base.

[0133] S702: According to the prompt word set, query in a single-point Redis cache whether there is a first query result corresponding to any prompt word in the prompt word set.

[0134] S703: Send the prompt word for the query result not existing in the single-point Redis cache to the large model service through the shared memory channel.

[0135] The specific implementation of step S701 to step S703 is the same as that in the above-mentioned embodiment, and will not be repeated here.

[0136] S704: For each prompt word for which no query result exists in the single-point Redis cache, the large model service performs inference calculation on the prompt word to generate a second query result.

[0137] In this step, after the large model service receives the prompt word, it first pre-processes the prompt word.

[0138] Then, reasoning is performed, which may include text understanding, semantic analysis, and knowledge reasoning. Based on the results of the reasoning calculation, the big model service generates a second query result related to the prompt word. The second query result can be in various forms such as text description, classification label, and associated entity, depending on the requirements of the query request.

[0139] Optionally, the reasoning process can be implemented in the following ways:

[0140] Tokenization: Split the prompt word (if it is a long text) into a series of words or subword units (tokens), thereby converting continuous text data into discrete units that the model can process.

[0141] Vectorization: Convert each token into a fixed-length vector representation. These vectors contain the semantic information of the token and are the basis for the model to reason. This step is usually completed using a pre-trained word embedding model (such as Word2Vec, GloVe, or BERT's Embedding layer).

[0142] Load a pre-trained large model (such as GPT, BERT, etc.) into a computing device (such as a GPU or TPU). This may include loading the model's weights, architecture, and other necessary parameters.

[0143] Initialization: Initialize the model according to the requirements of the model, such as configuring the inference mode, setting the batch size, etc.

[0144] For large models based on the Transformer structure, the input data is first forward propagated through multiple layers of the encoder. Each layer contains a self-attention mechanism and a feed-forward neural network.

[0145] The self-attention mechanism allows the model to take into account the contextual information of the entire input sequence when processing each token, thereby capturing dependencies in the text.

[0146] The decoder gradually generates the tokens of the target sequence based on the output of the encoder and the generated sequence. Each newly generated token is inferred based on the previous output and input sequence.

[0147] For each newly generated token, the model outputs a probability distribution, indicating the probability of generating each word at that position. By selecting the word with the highest probability or adopting other sampling strategies (such as top-psampling, nucleus sampling, etc.), the complete sequence can be gradually generated.

[0148] During inference, the model computes self-attention multiple times to capture long-range dependencies in the input sequence. This is usually done by computing the correlation between each token and every other token in the input sequence.

[0149] Convert the vector output by the model into a readable text form. For classification tasks, it is also necessary to convert the probability distribution into the most likely category label.

[0150] For the generated text, some correction or post-processing operations can also be performed to improve the quality and readability of the text. Finally, the processed second query result is obtained.

[0151] S705: Feedback the first query result and the second query result to the user.

[0152] The specific implementation of step S305 is the same as that in the above-mentioned embodiment, and will not be repeated here.

[0153] The big model data processing method provided in this embodiment responds to the user's query request, determines a prompt word set based on a pre-set knowledge base, queries whether there is a first query result corresponding to any prompt word in the prompt word set in the single-point Redis cache according to the prompt word set, sends the prompt word that does not have a query result in the single-point Redis cache to the big model service through a shared memory channel, and for each prompt word that does not have a query result in the single-point Redis cache, the big model service performs inference calculation on the prompt word to generate a second query result, and feeds back the first query result and the second query result to the user. The method realizes an efficient, flexible, resource-optimized, and user-experience-enhanced data retrieval and response mechanism by combining technical means such as a single-point Redis cache, a shared memory channel, and a big model service.

[0154] Figure 8 A structural block diagram of a large model data processing device provided in an embodiment of the present disclosure, referring to Figure 8 The large model data processing device 800 shown includes the following units:

[0155] A determination unit 801 is used to determine a prompt word set in response to a user's query request based on a preset knowledge base, where the prompt word set includes at least two prompt words, each prompt word is used to represent the user's query requirement, and the knowledge base includes multiple prompt words;

[0156] A query unit 802 is used to query whether there is a first query result corresponding to any prompt word in the prompt word set in the single-point Redis cache according to the prompt word set;

[0157] A generating unit 803 is used to send the prompt word for the query result not existing in the single-point Redis cache to the large model service through the shared memory channel to generate a corresponding second query result;

[0158] The feedback unit 804 is used to feed back the first query result and the second query result to the user.

[0159] In a possible design, the large model data processing device 800 further includes:

[0160] The cache unit 805 is used to cache the second query result and the corresponding prompt word in a single-point Redis cache through a shared memory channel.

[0161] Optionally, the large model data processing device 800 further includes:

[0162] The writing unit 806 writes the second query result and the corresponding prompt word into the Kafka service.

[0163] In another possible design, the large model data processing device 800 further includes:

[0164] The consumption unit 807 is used for re-caching the data stored before the single-point Redis cache is restarted by consuming the data stored in the Kafka service after the single-point Redis cache is restarted.

[0165] In a possible implementation, the determining unit 801 includes:

[0166] A conversion module, used for converting the query request into at least one first prompt word;

[0167] A determination module, configured to determine, for each first prompt word, a second prompt word corresponding to the first prompt word in a knowledge base;

[0168] The summarizing module is used to sum up all the first prompt words and all the second prompt words to obtain a prompt word set.

[0169] In a possible implementation, determining a module includes:

[0170] A calculation submodule, for calculating, for each first prompt word, the similarity between the first prompt word and each prompt word in the knowledge base;

[0171] The determination submodule is used to determine the prompt word with a similarity greater than a preset similarity threshold as the second prompt word.

[0172] In a possible implementation, the generating unit 803 includes:

[0173] The generation module is used for the large model service to perform inference calculation on each prompt word for which no query result exists in the single-point Redis cache, and generate a second query result.

[0174] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0175] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0176] Fig. 9 A schematic block diagram of an electronic device implementing an embodiment of the present disclosure is shown in FIG. Fig. 9 As shown, Fig. 9A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0177] like Fig. 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0178] Multiple components in the electronic device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0179] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as the method large model data processing method. For example, in some embodiments, the method large model data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the method large model data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the method large model data processing method in any other appropriate manner (e.g., by means of firmware).

[0180] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0181] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0182] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0183] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0184] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0185] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("ViRtual PRivate SeRveR", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0186] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0187] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A large model data processing method, comprising: In response to a user's query request, determining a prompt word set based on a preset knowledge base, wherein the prompt word set includes at least two prompt words, each prompt word is used to represent the user's query requirement, and the knowledge base includes a plurality of prompt words; According to the prompt word set, querying in a single-point Redis cache whether there is a first query result corresponding to any prompt word in the prompt word set; Sending the prompt word for the query result not existing in the single-point Redis cache to the large model service through the shared memory channel to generate a corresponding second query result; The first query result and the second query result are fed back to the user.

2. The method according to claim 1, further comprising: The second query result and the corresponding prompt word are cached in the single-point Redis cache through the shared memory channel.

3. The method according to claim 2, further comprising: The second query result and the corresponding prompt word are written into the Kafka service.

4. The method according to claim 3, further comprising: After the single-point Redis cache is restarted, the single-point Redis cache re-caches the data stored before the single-point Redis cache is restarted by consuming the data stored in the kafka service.

5. The method according to any one of claims 1 to 4, wherein: The step of determining the user's prompt word set in response to the user's query request based on a preset knowledge base includes: Converting the query request into at least one first prompt word; For each first prompt word, determining a second prompt word corresponding to the first prompt word in the knowledge base; All the first prompt words and all the second prompt words are summed up to obtain the prompt word set.

6. The method according to claim 5, wherein: Determining a second prompt word corresponding to the first prompt word in the knowledge base includes: For each first prompt word, calculating the similarity between the first prompt word and each prompt word in the knowledge base; The prompt word whose similarity is greater than a preset similarity threshold is determined as the second prompt word.

7. The method according to any one of claims 1 to 4, wherein: The generating a corresponding second query result includes: For each prompt word for which no query result exists in the single-point Redis cache, the large model service performs inference calculation on the prompt word to generate the second query result.

8. A large model data processing device, comprising: a determination unit, configured to determine, in response to a query request of a user, a prompt word set based on a preset knowledge base, wherein the prompt word set includes at least two prompt words, each prompt word is used to represent the query requirement of the user, and the knowledge base includes a plurality of prompt words; A query unit, configured to query, according to the prompt word set, whether there is a first query result corresponding to any prompt word in the prompt word set in a single-point Redis cache; A generating unit, configured to send a prompt word indicating that there is no query result in the single-point Redis cache to the large model service through a shared memory channel to generate a corresponding second query result; A feedback unit is used to feed back the first query result and the second query result to the user.

9. The apparatus according to claim 8, further comprising: The cache unit is used to cache the second query result and the corresponding prompt word in the single-point Redis cache through the shared memory channel.

10. The apparatus according to claim 9, further comprising: The writing unit writes the second query result and the corresponding prompt word into the Kafka service.

11. The apparatus according to claim 8, further comprising: The consumption unit is used for, after the single-point Redis cache is restarted, the single-point Redis cache consumes the data stored in the kafka service to re-cache the data stored before the single-point Redis cache is restarted.

12. The device according to any one of claims 9 to 11, wherein: The determining unit comprises: A conversion module, used for converting the query request into at least one first prompt word; a determination module, configured to determine, for each first prompt word, a second prompt word corresponding to the first prompt word in the knowledge base; The summarizing module is used to sum up all the first prompt words and all the second prompt words to obtain the prompt word set.

13. The device according to claim 12, wherein: The determining module comprises: A calculation submodule, used for calculating, for each first prompt word, the similarity between the first prompt word and each prompt word in the knowledge base; The determination submodule is used to determine the prompt word whose similarity is greater than a preset similarity threshold as the second prompt word.

14. The device according to any one of claims 9 to 11, wherein: The generating unit comprises: A generation module is used for the large model service to perform inference calculation on each prompt word for which no query result exists in the single-point Redis cache, so as to generate the second query result.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

17. A computer program product, comprising a computer program, which implements the steps of the method according to any one of claims 1 to 7 when executed by a processor.

Images