Business processing method and related equipment
By receiving service requests, determining hash values and using multimodal service gated routing and Markov decision-making process, the problem of hardware matching difficulties for large models in heterogeneous resource computing pools is solved, inference consistency and resource collaborative efficient computing are achieved, and resource utilization and flexibility are improved.
Patent Information
- Application Number
- CN202510387074.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-28
- Publication Date
- 2025-08-08
AI Technical Summary
The hardware matching of large models is difficult to match in engineering applications, which makes it difficult to achieve inference consistency and efficient resource collaborative computing. Especially in heterogeneous resource computing pools, manual configuration rules are required under traditional architectures, and heterogeneous resource inference computing routing is not intelligent enough.
By receiving service requests, determining hash values, retrieving corresponding hardware resources, and using multimodal service gated routing mechanism and Markov decision-making process, service requests are distributed to target computing cards, achieving unified management and scheduling of different hardware resources, and using parallel processing and key-value caching technology to improve resource utilization and flexibility.
It ensures the consistency of inference computing in distributed heterogeneous resources, improves resource utilization and flexibility, realizes efficient management and scheduling of different hardware resources, and adapts to application needs in multiple industries and multiple scenarios.
Smart Images
Figure CN120448093A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a business processing method and related equipment. Background Art
[0002] With the rapid development of large models, the scale of model parameters has become increasingly large, making it difficult to match hardware for large models in engineering applications. Customers' demand for efficient inference and fast response is becoming increasingly urgent. Due to hardware resource limitations, there is an increasing demand for hybrid computing and inference using both foreign and domestic cards, using multiple machines and multiple cards.
[0003] How to ensure the consistency of reasoning and the collaborative and efficient computing of resources has become an urgent problem that needs to be solved.
[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention
[0005] The present disclosure provides a business processing method and related equipment, which can at least ensure the consistency of reasoning and the collaborative and efficient computing of resources.
[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.
[0007] In a first aspect, embodiments of the present disclosure provide a service processing method, the method comprising:
[0008] Receive business requests;
[0009] Determine a hash value based on the user identifier and request identifier of the service request;
[0010] Searching according to the hash value to determine a first hardware resource corresponding to the same historical hash value as the service request;
[0011] The service request is distributed to the first hardware resource, and a key-value cache is obtained to process the service request.
[0012] In a possible embodiment, the method further includes:
[0013] Through the multimodal service gating routing mechanism, the service request is distributed to the target computing card of the corresponding type according to the service type of the service request.
[0014] In a possible embodiment, distributing the service request to a target computing card of a corresponding type according to the service type of the service request through a multimodal service gating routing mechanism includes:
[0015] Determining a service type of the service request;
[0016] If the service request is of a single service type, determining the target computing card of the service request through a multimodal service gating routing mechanism according to the unique valid code of the computing card and the service request;
[0017] If the service request is a multimodal service type, the target computing card for processing the service request is determined through the activation function and the multimodal service gating routing mechanism according to the data vector and the corresponding weight value corresponding to each service type.
[0018] In a possible embodiment, the method further includes:
[0019] The service request is distributed to the second hardware resource from the plurality of hardware resources according to a Markov decision process.
[0020] In a possible embodiment, distributing the service request to the second hardware resource from the multiple hardware resources according to a Markov decision process includes:
[0021] Determining a defined hardware resource space, a defined action space for distributing the service request to the hardware resources, and a defined reward space for different distribution actions;
[0022] The service request is distributed to the second hardware resource among multiple hardware resources according to a Markov decision process, the hardware resource space, the action space, and the reward space.
[0023] In a possible embodiment, the method further includes:
[0024] If the business request is a long text;
[0025] Segmenting the long text and numbering them in sequence to obtain multiple segmented texts;
[0026] The multiple segmented texts are processed in parallel to obtain and save a key-value cache.
[0027] In a second aspect, embodiments of the present disclosure provide a service processing device, including:
[0028] A receiving unit, configured to receive a service request;
[0029] a determining unit, configured to determine a hash value according to a user identifier and a request identifier of the service request;
[0030] a retrieval unit, configured to perform a search based on the hash value to determine a first hardware resource corresponding to a historical hash value identical to the service request;
[0031] A distribution unit is used to distribute the service request to the first hardware resource and obtain a key-value cache to process the service request.
[0032] In a possible embodiment, the apparatus further includes: a gated routing distribution unit configured to:
[0033] Through the multimodal service gating routing mechanism, the service request is distributed to the target computing card of the corresponding type according to the service type of the service request.
[0034] In a possible embodiment, the apparatus further includes a decision unit configured to:
[0035] The service request is distributed to the second hardware resource from the plurality of hardware resources according to a Markov decision process.
[0036] In a possible embodiment, the device further includes: a long text processing unit, configured to:
[0037] If the business request is a long text;
[0038] Segmenting the long text and numbering them in sequence to obtain multiple segmented texts;
[0039] The multiple segmented texts are processed in parallel to obtain and save a key-value cache.
[0040] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method described in the first aspect above by executing the executable instructions.
[0041] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method described in the first aspect when executed by a processor.
[0042] In a fifth aspect, according to another aspect of the present disclosure, a computer program product or computer program is further provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the methods described above.
[0043] Embodiments of the present disclosure provide a business processing method and related equipment, comprising: receiving a business request, determining a hash value based on a user identifier and a request identifier of the business request, searching based on the hash value, determining a first hardware resource corresponding to a historical hash value identical to the business request, distributing the business request to the first hardware resource, and obtaining a key-value cache to process the business request. This ensures consistency in inference computations from business requests to distributed heterogeneous resources, enabling unified management and scheduling of different hardware resources for computing resources, thereby improving resource utilization and flexibility.
[0044] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.
[0046] Figure 1 A flowchart showing a business processing method in an embodiment of the present disclosure is shown;
[0047] Figure 2 A schematic diagram showing a hash processing distribution service request according to an embodiment of the present disclosure is shown;
[0048] Figure 3 A flowchart showing a routing process according to an embodiment of the present disclosure is shown;
[0049] Figure 4 A schematic diagram illustrating a business processing process in an embodiment of the present disclosure;
[0050] Figure 5 A flowchart showing a parallel processing process according to an embodiment of the present disclosure is shown;
[0051] Figure 6 A flowchart showing a long text processing process according to an embodiment of the present disclosure is shown;
[0052] Figure 7 A schematic diagram showing a Markov decision process in an embodiment of the present disclosure;
[0053] Figure 8 A schematic diagram showing the structure of a service processing device according to an embodiment of the present disclosure is shown;
[0054] Figure 9 A schematic structural diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0055] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0056] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0057] Currently, large models are rapidly expanding from natural language understanding to multiple industries and scenarios, including multimedia, audiovisual, and multimodal computing. The scale of data and model capacity continues to climb. Model parameters have leapt from tens of billions to trillions, but the iteration speed of physical hardware, especially graphics cards, has been relatively slow, and the two have grown out of sync.
[0058] This explosive growth in parameters makes it difficult to match hardware with large models for engineering applications. Customers are increasingly demanding efficient inference and rapid response. Due to hardware resource constraints, there is an increasing demand for hybrid inference computing using both foreign and domestic graphics cards, across multiple machines and multiple cards. To ensure consistent inference and efficient resource coordination, it is crucial to employ reliable scheduling algorithms and systems. This not only improves inference efficiency but also ensures the widespread application and importance of models across multiple industries and scenarios.
[0059] In related technologies, the perception and matching of heterogeneous resource computing pools by large model reasoning under traditional architecture is based on static business rules, which means that adding and reducing related rules requires manual configuration.
[0060] Domestic and foreign cards have different processing performance for text, image, and video service requests, and traditional distributed heterogeneous resource reasoning and computing routing are blind.
[0061] The disclosed embodiments provide a business processing method and related equipment, comprising: receiving a business request, determining a hash value based on a user identifier and a request identifier of the business request, searching based on the hash value, determining a first hardware resource corresponding to a historical hash value identical to the business request, distributing the business request to the first hardware resource, and obtaining a key-value cache to process the business request. This ensures consistency in inference computations from business requests to distributed heterogeneous resources, enabling unified management and scheduling of different hardware resources for computing resources, thereby improving resource utilization and flexibility.
[0062] This exemplary implementation is described in detail below with reference to the accompanying drawings and examples.
[0063] First, an embodiment of the present disclosure provides a business processing method, which can be executed by any electronic device with computing and processing capabilities. In the following process, the electronic device is taken as an example of a business processing system.
[0064] Figure 1 A flowchart of a business processing method according to an embodiment of the present disclosure is shown. Figure 1 As shown, the service processing method provided in the embodiment of the present disclosure includes the following steps:
[0065] S102: Receive a service request.
[0066] The service requests may include service requests from a large number of users.
[0067] S104: Determine a hash value according to the user ID and request ID of the service request.
[0068] In a possible embodiment, a hash value of each service request is determined according to a request identifier of each service request and a user identifier of the user.
[0069] S106: Search according to the hash value to determine a first hardware resource corresponding to the same historical hash value as the service request.
[0070] S108: Distribute the service request to the first hardware resource and obtain the key-value cache to process the service request.
[0071] In one possible embodiment, different card computing resource pools are used together in a complex and potential field. The inconsistency of the underlying computing framework causes large models to encounter various incompatibility problems during training and reasoning. In the embodiment of the present disclosure, cloud computing platforms and virtualization technologies can be used for computing resources to achieve unified management and calling of different cards, thereby improving resource utilization and flexibility.
[0072] In cloud computing platforms, computing resources are divided into different hardware resources based on different physical hardware.
[0073] For example, foreign server clusters, domestic hardware resources, etc. belong to different hardware resources.
[0074] Through the method in the embodiment of the present disclosure, multiple rounds of business requests for the same user can be distributed to the same hardware resources (hardware cluster) for computational reasoning, so as to use the key-value (KV) cache pool data of the same hardware resources to achieve the purpose of improving the reasoning speed.
[0075] After obtaining the business request, the user identifier (user_id) and request identifier (resid) of the business request can be determined, and a hash operation can be performed to obtain a hash value. Business requests of the same user are distributed to the same computing resources in the cloud service cluster, that is, to the same hardware resources.
[0076] The first hardware resource is a hardware resource to which the same service request sent by the same user as that sending the service request is distributed.
[0077] The business request can also be distributed to the first hardware resource this time, so the key-value cache can be directly obtained, ensuring that the KV cache in the prefill stage can be reused by the same user, which can save a lot of computing resources.
[0078] The hash processing formula can be shown as formula (1).
[0079] Hash(user_id+resid)(1)
[0080] In a possible embodiment, the hardware resources in the resource pool are identified to obtain a hardware resource identifier (srvid), and the srvid and large model capabilities are stored in a cache database. The hash value can be searched from the resource cache library for matching and distribution, and the historical hash value that is the same as the hash value of the business request is retrieved from the resource cache library to determine the corresponding hardware resource identifier and then distribute it.
[0081] Cache consistency comparison and distribution are performed on user IDs, request IDs, and hardware resources to ensure business requests, reuse the KV cache for data calculations, and improve the hit rate and accuracy of the KV cache for inference calculations.
[0082] Figure 2 A schematic diagram of hash processing and distribution of service requests is shown, Figure 2After receiving a service request, it first passes through the service mesh proxy, and then is further processed to determine the user ID and request ID, hash it to obtain a hash value, and then retrieve it and distribute it to the corresponding hardware resources.
[0083] There may be a situation where the corresponding hardware resources cannot be retrieved after the hash value is determined. In this case, other methods can be used to distribute the service request to the hardware resources.
[0084] In a possible embodiment, a multimodal service gating routing mechanism may be used to distribute service requests to target computing cards of corresponding types according to the service types of the service requests.
[0085] Figure 3 A flow chart of a routing process in an embodiment of the present disclosure is shown. Figure 3 As shown, the following steps are included:
[0086] S302: Determine the service type of the service request.
[0087] The service types of service requests include multiple modalities, that is, multiple data types, such as web pages, pictures, voice, video, etc.
[0088] In the related art, traditional information interaction is that users select the business data type to make requests, which makes the user experience cumbersome.
[0089] The disclosed embodiment adopts a multimodal business expert mixture (Mixture of Experts, MoE) gated routing mechanism, which can intelligently identify and divert user requested data.
[0090] S304: If the service request is of a single service type, the target computing card of the service request is determined through a multimodal service gating routing mechanism according to the unique valid code of the computing card and the service request.
[0091] In one possible embodiment, each business type's big model will have a unique one-hot encoding that identifies the big model's capabilities. Business requests of a single business type can be identified and matched with the corresponding big model capabilities. After prefill encoding, the business request data is distributed to the corresponding big model capability for inference and computation via the multimodal business gating routing mechanism. This means that the data is distributed to the corresponding target compute card for inference and computation.
[0092] S306: If the service request is a multimodal service type, the target computing card for processing the service request is determined through the activation function and the multimodal service gating routing mechanism according to the data vector and weight value corresponding to each service type.
[0093] In a possible embodiment, gated routing distribution is performed for multimodal service types. The probability of distribution to a large model of the corresponding type can be determined by the data vector and the corresponding weight value, and distributed to the target computing card through a gating mechanism.
[0094] The distribution probability can be determined using the following formula (2):
[0095] G j (X i )=Softmax(X i * W b ) (2)
[0096] Among them, G j (X i ) represents the probability value determined by the activation function, which is used to determine the distribution to the target computing card. i The data vector representing the business request; W b Indicates the corresponding weight value.
[0097] Through the above method, multimodal business gating routing mechanism is used to distribute multimodal data to different heterogeneous resources for calculation, thereby improving the inference speed of large models and the consistency of business calculations.
[0098] Figure 4 A schematic diagram of a business processing process in an embodiment of the present disclosure is shown. Figure 4 shown.
[0099] Among them, through the multimodal business gating routing mechanism, different multimodal MoE gating strategies can be specified, and then the received business requests are routed to the inference nodes in different hardware resource pools using routing.
[0100] The cloud computing platform also includes cloud management and control task orchestration for task orchestration, task orchestration scheduling for task scheduling, and a model warehouse for storing large models.
[0101] Furthermore, under the traditional architecture, large model inference is divided into two stages: prefill and decoding. The calculations in these two stages are performed sequentially in a single-threaded blocking manner, resulting in slow inference speed and long response time.
[0102] In the embodiment of the present disclosure, multiple service requests may be processed in a parallel processing manner.
[0103] After receiving a large number of business requests, different business requests can be hashed separately and distributed to the corresponding hardware resource pool for parallel calculation. The key-value cache calculated for different business requests is stored back in the response memory.
[0104] Process different business requests in parallel. Figure 5 A flow chart of a parallel processing process is shown in FIG. Figure 5 As shown in the figure, different business requests can be distributed to the corresponding hardware resources and then prefill calculations can be performed in parallel.
[0105] There are many parallel processing methods, such as pipeline parallel processing, data parallelism, task parallelism, etc. The embodiments of the present disclosure are only examples and do not limit the specific parallel processing methods.
[0106] In a possible embodiment, the service request may include a long text or a short text, and both the long text and the short text may be processed in parallel.
[0107] When processing inference calculations for long texts on heterogeneous resources, ensuring consistency in hardware resource matrix calculations is paramount. Large model inference involves two phases: prefill and decoding. The prefill phase is computationally expensive, so improving the parallelization efficiency of this phase is key to accelerating large model inference.
[0108] KVM and other technologies are used to virtualize heterogeneous resources, and virtual CPU and memory resources are allocated accordingly for the same type of computing resources to ensure that the same type of hardware resources use consistent CPU and memory resources.
[0109] When processing business requests for long text in heterogeneous resources, Figure 6 A flowchart of a long text processing process in an embodiment of the present disclosure is shown. Figure 6 As shown, the following steps are included:
[0110] S602: If the service request is a long text.
[0111] S604: Segment the long text into segments and number them in sequence to obtain multiple segmented texts.
[0112] S606: Process the multiple segmented texts in parallel to obtain and save the key-value cache.
[0113] In a possible embodiment, a long text service request can be segmented into 512 characters and numbered sequentially to obtain multiple segmented texts. Resource consistency is ensured according to the hash value, and the service request is distributed to the same hardware resource requested by the same user. Prefill calculations are performed in parallel, and the calculated KV cache is stored back in the corresponding CPU memory.
[0114] Through the above method, the long text data of the business request is divided into chunks for processing, and the computing and storage resources are squeezed to the extreme in a parallel manner, thereby improving the utilization rate of heterogeneous physical hardware resources.
[0115] The disclosed embodiment also shows a solution for Markov heterogeneous resource matching routing.
[0116] In a possible embodiment, after receiving a service request from a user, the service request is distributed to a second hardware resource from a plurality of hardware resources according to relevant data using a Markov Decision Process (MDP).
[0117] Among them, the inference computing heterogeneous resource space, that is, the hardware resource space, can be set, which is defined as: S = {S1, S2, S3, S4, S5...}, where S represents different hardware resource spaces.
[0118] The action space A for distributing service requests to hardware resources is defined as A = {A1, A2, A3, A4, A5...}, where A represents the action of distributing service requests to hardware resources, i.e., the decision action of the Markov decision process.
[0119] Reward the decision-making process and define the reward space, R = {R1, R2, R3, R4, R5...}, where R represents the reward for different actions.
[0120] In one possible embodiment, a defined hardware resource space, a defined action space for distributing business requests to hardware resources, and a defined reward space for different distribution actions are obtained, and based on the Markov decision process, the hardware resource space, the action space, and the reward space, the business request is distributed to a second hardware resource among multiple hardware resources.
[0121] Through MDP forwarding, the traffic is diverted to the corresponding hardware resources, and then the return results of the computing resources, time, user comments and other information are analyzed and summarized, and rewards are given for the relevant decision-making process.
[0122] Figure 7 A schematic diagram of a Markov decision process in an embodiment of the present disclosure is shown. Figure 7As shown in the figure, the Markov heterogeneous resource matching routing solution makes a decision through MDP after receiving the service request. The actions distributed to different hardware resources correspond to different rewards, which are: 0.2, 0.3, 0.5, etc.
[0123] Among them, the hardware resources corresponding to the decision-making business request through MDP can be selected to be executed after the hash value retrieval fails, or the business request can be distributed to the hardware resources only using MDP.
[0124] Through the method in the embodiments of the present invention, a Markov decision algorithm is used to analyze, summarize and count business requests, computing resource processing status, processing time, etc., to ensure the matching of heterogeneous computing resources for multimodal business, and to realize intelligent diversion of business requests based on data type, long and short text, etc., ultimately achieving the purpose of efficient scheduling and management of heterogeneous distributed resources.
[0125] Through the business processing method in the embodiment of the present disclosure, by adopting multiple algorithms and strategies to schedule and manage tasks on distributed heterogeneous computing resources, it is possible to utilize the current multiple computing resources, save physical hardware costs, accelerate the industry application of large models, and use a strategy based on the Markov decision process to efficiently match heterogeneous resources. It can aggregate different multimodal business systems, divert different business data according to the inconsistency of computing card types, and realize the full and flexible application of heterogeneous resource pools.
[0126] Based on the same inventive concept, the present disclosure also provides a service processing device, such as the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.
[0127] Figure 8 A schematic diagram of the structure of a business processing device in an embodiment of the present disclosure is shown. Figure 8 As shown, the business processing device 80 includes: a receiving unit 801 for receiving a business request; a determining unit 802 for determining a hash value based on a user identifier and a request identifier of the business request; a retrieval unit 803 for searching based on the hash value to determine a first hardware resource corresponding to a historical hash value identical to the business request; a distribution unit 804 for distributing the business request to the first hardware resource and obtaining a key value cache to process the business request.
[0128] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods, or program products. Therefore, various aspects of the present disclosure may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."
[0129] Refer to the following Figure 9 1 and 2 to describe the electronic device 900 according to this embodiment of the present disclosure. Figure 9 The electronic device 900 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.
[0130] like Figure 9 As shown, electronic device 900 is implemented as a general-purpose computing device. Components of electronic device 900 may include, but are not limited to, at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting various system components (including storage unit 920 and processing unit 910).
[0131] The storage unit stores program code, which can be executed by the processing unit 910, so that the processing unit 910 performs the steps described in the "Exemplary Method" section above according to various exemplary embodiments of the present disclosure. For example, the processing unit 910 can perform the steps of any of the above method embodiments.
[0132] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 9201 and / or a cache memory unit 9202 , and may further include a read-only memory unit (ROM) 9203 .
[0133] The storage unit 920 may also include a program / utility 9204 having a set (at least one) of program modules 9205, such program modules 9205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0134] Bus 930 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0135] The electronic device 900 can also communicate with one or more external devices 940 (e.g., a keyboard, a pointing device, a Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 900, and / or any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., a router, a modem, etc.). Such communication can occur via an input / output (I / O) interface 950. Furthermore, the electronic device 900 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 960. As shown, the network adapter 960 communicates with other modules of the electronic device 900 via a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 900, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0136] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
[0137] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods of the above embodiments.
[0138] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided. The computer-readable storage medium may be a readable signal medium or a readable storage medium. A program product capable of implementing the above-mentioned method of the present disclosure is stored thereon. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Methods" section above of this specification.
[0139] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0140] In the present disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0141] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.
[0142] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, and the like, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a standalone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).
[0143] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.
[0144] Furthermore, although the steps of the method of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.
[0145] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
[0146] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.
Claims
1. A business processing method, characterized in that: The method comprises: Receive business requests; Determine a hash value based on the user identifier and request identifier of the service request; Searching according to the hash value to determine a first hardware resource corresponding to the same historical hash value as the service request; The service request is distributed to the first hardware resource, and a key-value cache is obtained to process the service request.
2. The method according to claim 1, characterized in that The method further comprises: Through the multimodal service gating routing mechanism, the service request is distributed to the target computing card of the corresponding type according to the service type of the service request.
3. The method according to claim 1, characterized in that The method of distributing the service request to a target computing card of a corresponding type according to the service type of the service request through a multimodal service gating routing mechanism includes: Determining a service type of the service request; If the service request is of a single service type, determining the target computing card of the service request through a multimodal service gating routing mechanism according to the unique valid code of the computing card and the service request; If the service request is a multimodal service type, the target computing card for processing the service request is determined through the activation function and the multimodal service gating routing mechanism according to the data vector and the corresponding weight value corresponding to each service type.
4. The method according to claim 1, wherein The method further comprises: The service request is distributed to the second hardware resource from the plurality of hardware resources according to a Markov decision process.
5. The method according to claim 4, characterized in that The distributing the service request to the second hardware resource from the plurality of hardware resources according to the Markov decision process includes: Determining a defined hardware resource space, a defined action space for distributing the service request to the hardware resources, and a defined reward space for different distribution actions; The service request is distributed to the second hardware resource among multiple hardware resources according to a Markov decision process, the hardware resource space, the action space, and the reward space.
6. The method according to claim 1, characterized in that The method further comprises: If the business request is a long text; Segmenting the long text and numbering them in sequence to obtain multiple segmented texts; The multiple segmented texts are processed in parallel to obtain and save a key-value cache.
7. A business processing device, characterized in that: include: A receiving unit, configured to receive a service request; A determining unit, configured to determine a hash value according to a user identifier and a request identifier of the service request; a retrieval unit, configured to perform a search based on the hash value to determine a first hardware resource corresponding to a historical hash value identical to the service request; A distribution unit is used to distribute the service request to the first hardware resource and obtain a key-value cache to process the service request.
8. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to perform the method according to any one of claims 1 to 6 by executing the executable instructions.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
10. A computer program product comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed by a processor, it implements the method according to any one of claims 1 to 6.