Task processing systems, methods, devices, storage media and chips

Abstract

This disclosure relates to a task processing system, method, device, storage medium, and chip. The system includes: middleware that, upon receiving a task request from a client, sends a request command to a database to request a target service matching the identifier and / or type of the task request; the database that responds to the request command by returning a corresponding response result to the middleware, the response result including information about the target service, which is a business service in a service cluster that is idle and matches the identifier and / or type of the task request, and at least one business service in the service cluster is registered in the database; the middleware further sends the task request to the target service based on the target service information; and the target service invokes a service engine to process the task request. This addresses the long-tail problem that can occur when there are many task requests in related technologies.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a task processing system, method, apparatus, storage medium, and chip. Background Technology

[0002] In the development of network system architecture, load balancers, as a key component of network architecture, have undergone many years of development, evolving from the initial hardware load balancers to software load balancers, and then to virtual load balancers. The technology has continuously evolved to meet different scenarios and needs.

[0003] For currently used network architectures, please refer to... Figure 1 As shown, a client-side (web-server) architecture is used to design the corresponding business service product. Taking speech synthesis service as an example, the speech synthesis service (TTS-server) is designed in the network architecture, and the specific implementation steps are as follows.

[0004] S1. Send the text to be synthesized or the result of the knowledge base to the client.

[0005] S2. After the client obtains the task request to be synthesized, it encapsulates and packages the task request and sends it to the gateway (apisix).

[0006] S3, after the gateway obtains the task request, it passes it through to the load balancing system (nginx) and the intermediate controller (Lua), which can be simply referred to as the central control load balancing system.

[0007] S4. The central control load balancing system reasonably allocates the above task requests and sends the task requests to the speech synthesis service by push.

[0008] S5. The speech synthesis service calls the speech synthesis engine to perform neural network inference, synthesizes high-quality speech from text, and streams it back to the load balancing system (nginx).

[0009] The S6 load balancing system (nginx) transmits the synthesized speech to the client through the gateway (apisix). The client then streams the audio.

[0010] In practice, it has been found that when there are many user task requests, the above solution cannot effectively distribute the task requests and fails to achieve the purpose of load balancing, resulting in a long tail problem for many task requests. Summary of the Invention

[0011] To overcome the problems existing in the related technologies, this disclosure provides a task processing system, method, device, storage medium and chip to solve the long tail problem that may occur when there are a large number of task requests in the aforementioned related technologies.

[0012] According to a first aspect of the present disclosure, a task processing system is provided, the system including middleware, a database, a service cluster, and a service engine; wherein: The middleware is used to send a request command to the database after receiving a task request from the client. The request command is used to request the acquisition of a target service that matches the identifier and / or type of the task request. The task request is used to request the processing of the task to be processed. The database is configured to return a corresponding response result to the middleware in response to the request command after receiving the request command. The response result includes information about the target service. The target service is a business service in the service cluster that is in an idle state and matches the identifier and / or type of the task request. At least one of the business services in the service cluster is registered in the database. The middleware is also used to send the task request to the target service based on the information of the target service after receiving the response result; The target service is used to invoke the service engine to process the task request after receiving it.

[0013] In some embodiments, the service cluster supports adding corresponding business services, wherein: The business service is used to send a registration request to the database, and the registration request carries information about the business service. The database is also used to respond to the registration request by registering the information of the business service into the database; and to send a corresponding registration result to the business service, the registration result being used to indicate whether the business service has been successfully registered.

[0014] In some embodiments, the system further includes a first cache; The first cache is used to store the processing results corresponding to high-frequency requests; The middleware is further configured to, upon receiving the high-frequency request, retrieve the processing result corresponding to the high-frequency request from the first cache, so as to prevent the high-frequency request from being sent to the service cluster for processing; wherein, the high-frequency request is a business request whose sending frequency and / or sending frequency exceeds a corresponding threshold.

[0015] In some embodiments, the service engine includes a front-end module, a back-end module, and a second cache; The second cache is used to store the recording library, which includes at least one of the following: health sample information, demand sample information, recording sample information, and high-frequency sample information. The front-end module is used to perform front-end business processing on the task request when the task request and the sample information in the recording library do not match, and send the obtained front-end processing result to the back-end module. The backend module is used to perform backend business processing on the task request when the task request matches the sample information in the recording library, and obtain the processing result corresponding to the task request; or, after receiving the frontend processing result, perform backend business processing on the frontend processing result to obtain the processing result corresponding to the task request.

[0016] In some embodiments, the front-end module is further configured to perform front-end business processing on the task request according to the priority of the task request; and / or, The backend module is further configured to perform backend business processing on the task request according to the priority of the sample information matching the task request; or, to perform backend business processing on the frontend processing result according to the priority of the task request; wherein the priority of the health sample information, the demand sample information, the recording sample information and the high-frequency sample information decreases sequentially.

[0017] In some embodiments, the target information stored in the database is set with a corresponding expiration time, and the target information includes at least one of the following: the identifier of the task request, the type of the task request, the information of the business service, and the processing result corresponding to the task request. The database is used to determine the expiration time of the target information based on the key value of the target information after receiving the key value of the target information; The database is further configured to determine that the target information has expired when the expiration time of the target information is less than or equal to the current system time, and to delete the target information from the database; wherein the target information and the expiration time of the target information are stored in the database as key-value pairs.

[0018] In some embodiments, when there are multiple target information items... The database is further configured to, when the number of expired target information items exceeds a preset number, traverse all target information items in the database and repeatedly execute the step of determining the expiration time of the target information item based on the key value of the target information item after receiving the key value of the target information item.

[0019] In some embodiments, the database is further configured to update the status of the target service to a non-idle state; or to remove the information of the target service from the database.

[0020] In some embodiments, the front-end module is further configured to store the processing result corresponding to the front-end module in a preset result table; and / or, the back-end module is further configured to store the processing result corresponding to the back-end module in the preset result table.

[0021] According to a second aspect of the present disclosure, a task processing method is provided, applied to a task processing system, the system including middleware, a database, a service cluster, and a service engine, the method comprising: After receiving a task request from a client, the middleware sends a request command to the database. The request command is used to request the acquisition of a target service that matches the identifier and / or type of the task request. The task request is used to request the processing of the task to be processed. The database responds to the request command to determine the target service and returns a corresponding response result to the middleware. The response result includes information about the target service. The target service is a business service in the service cluster of the task processing system that is in an idle state and matches the identifier and / or type of the task request. At least one of the business services in the service cluster is registered in the database. After receiving the response result, the middleware sends the task request to the target service based on the information of the target service. After receiving the task request, the target service invokes the corresponding service engine to process the task request.

[0022] For any content not introduced or described in this disclosure, please refer to the relevant description of the system described in the first aspect above, which will not be repeated here.

[0023] According to a third aspect of the present disclosure, a task processing apparatus is provided, applied to a task processing system, the system including middleware, a database, a service cluster, and a service engine, the apparatus comprising: The transceiver module is configured to send a request command to the database after receiving a task request from a client through the middleware. The request command is used to request the acquisition of a target service that matches the identifier and / or type of the task request. The task request is used to request the processing of the task to be processed. The transceiver module is further configured to, after receiving the request command through the database, return a corresponding response result to the middleware in response to the request command. The response result includes information about the target service. The target service is a business service in the service cluster of the task processing system that is in an idle state and matches the identifier and / or type of the task request. At least one of the business services in the service cluster is registered in the database. The transceiver module is further configured to send the task request to the target service based on the information of the target service after receiving the response result through the middleware; The processing module is configured to, after receiving the task request, invoke the corresponding service engine to process the task request through the target service.

[0024] For any content not introduced or described in the embodiments of this disclosure, please refer to the relevant descriptions in the foregoing method embodiments or system embodiments. This disclosure does not limit the scope of the embodiments.

[0025] According to a fourth aspect of the present disclosure, a computer device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the steps of the task processing method described above.

[0026] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the task processing method provided in the second aspect of the present disclosure.

[0027] According to a sixth aspect of the present disclosure, a chip is provided, comprising: a processor and an interface; the processor is configured to read instructions to execute the steps of the task processing method described above.

[0028] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: This disclosure provides a task processing system, which includes middleware, a database, a service cluster, and a service engine; wherein, after receiving a task request from a client, the middleware sends a request command to the database, the request command being used to request the acquisition of a target service that matches the identifier and / or type of the task request, the task request being used to request processing of a task to be processed; the database, after receiving the request command, returns a corresponding response result to the middleware in response to the request command, the response result including information about the target service, the target service being a business service in the service cluster that is in an idle state and matches the identifier and / or type of the task request, and at least one of the business services in the service cluster is registered in the database; the middleware is also used to send the task request to the target service based on the information of the target service after receiving the response result; the target service, after receiving the task request, calls the service engine to process the task request. In this way, the task processing system can use a push-pull combination method to send the above-mentioned task request to the target service in an idle state for processing, avoiding technical problems such as long tails when there are many task requests. This helps improve the reliability and efficiency of task processing.

[0029] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0031] Figure 1 This is a schematic diagram of a network architecture provided by existing technology.

[0032] Figure 2 This is a schematic diagram of the structure of a task processing system according to an exemplary embodiment.

[0033] Figure 3 This is a schematic diagram illustrating the structure of another task processing system according to an exemplary embodiment.

[0034] Figure 4 This is a schematic diagram illustrating the structure of a service engine according to an exemplary embodiment.

[0035] Figure 5 This is a schematic diagram illustrating the structure of another service engine according to an exemplary embodiment.

[0036] Figure 6 This is a schematic diagram of the data structure of an expired dictionary according to an exemplary embodiment.

[0037] Figure 7 This is a schematic diagram illustrating a user request processing flow according to an exemplary embodiment.

[0038] Figure 8 This is a flowchart illustrating a task processing method according to an exemplary embodiment.

[0039] Figure 9 This is a schematic diagram of the structure of a task processing device according to an exemplary embodiment.

[0040] Figure 10 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment.

[0041] Figure 11 This is a schematic diagram of the structure of a chip according to an exemplary embodiment. Detailed Implementation

[0042] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0043] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are performed with the authorization of the respective device owner.

[0044] Existing load balancers not only intelligently distribute requests based on server load, but also possess high availability and fault tolerance capabilities, ensuring service continuity and stability. In large-scale speech synthesis model systems, load balancers are widely used in the front-end access layer to improve system response speed and concurrent processing capabilities by intelligently allocating computing resources. In similar systems, caching techniques reduce redundant calculations and improve system response speed by storing intermediate results or model parameters of frequently requested data in a cache.

[0045] Large-scale speech synthesis systems, a major breakthrough in modern technology, convert text into rich and lifelike speech, bringing a completely new interactive experience. This technology not only makes the conversion between text and speech more intuitive and efficient, but also makes speech the primary carrier of multimedia information. People's expectations for intelligent voice interaction systems are also rising. Large-scale speech synthesis services, as an important component of question-and-answer interaction systems, act as the machine's mouth, needing not only to respond quickly but also to synthesize accurate and realistic speech, providing users with a more natural and richer voice interaction experience. With continuous technological advancements, large-scale speech synthesis systems will play a vital role in more fields, bringing people more natural and realistic voice interaction experiences. With the improvement of computing power and the increase in data volume, the scale and performance of large models are also constantly improving. However, larger models often mean stronger expressive power and higher accuracy, but also bring problems / challenges such as underutilization of computing resources and difficulties in scaling. Therefore, how to ensure load balancing and scaling of large-scale model systems is one of the important directions of large-scale model research.

[0046] In practical applications, large-scale speech synthesis systems require stability, efficiency, scalability, and uninterrupted service. In other words, these systems must be able to expand their capacity by continuously increasing the number of speech synthesis services (or servers) without interruption. Simultaneously, they must meet two requirements: firstly, the expansion process must be transparent to users, meaning users are unaware of the system's actual capacity; secondly, after expansion, the various speech synthesis services should reasonably distribute user task requests to achieve load balancing. However, currently, large-scale speech synthesis systems typically employ the following methods... Figure 1 The network architecture shown has the following problems in practice: (1) During peak periods, users make a lot of task requests. The above solution cannot distribute the task requests well and cannot achieve the purpose of load balancing, resulting in many tasks having long tail problems.

[0047] (2) The existing large-scale speech synthesis model system has a long end-to-end, and updating speech synthesis services is cumbersome. When expanding the capacity by pushing, multiple speech synthesis services need to be launched at the same time, which is highly dependent and cannot be effectively expanded and scaled.

[0048] (3) Existing speech synthesis large model systems do not support multi-level caching strategies. Without retraining the model, the problems reported by each component in the model cannot be quickly and promptly fixed, and audio problems cannot be accurately corrected, affecting the user experience.

[0049] (4) Existing speech synthesis large model systems lack effective tools to view the waiting time and lifespan of task requests. When the number of user task requests surges suddenly, the number of task requests accumulated in the queue increases until it overwhelms the speech synthesis large model system, which can seriously affect the user's real experience.

[0050] (5) The system has poor general scalability and is highly dependent on business. Existing large-scale speech synthesis systems cannot be applied to computationally intensive neural network inference systems such as large-scale speech recognition models and large-scale text models. They cannot complete the task requests and processing results of the entire system chain through a single system. For example, they cannot obtain the task requests and processing results of the entire chain, such as wake-up, recognition, speech understanding and speech synthesis. Moreover, the chain debugging is relatively difficult.

[0051] To resolve the above issues, please refer to [link / reference]. Figure 2 This is a schematic diagram illustrating the structure of a task processing system according to an exemplary embodiment. For example... Figure 2The system shown may include a client 100 and a server 200. The server 200 may include a gateway 201 (apisix), a load balancer 202 (nginx), middleware 203 (outer-server), a database 204 (redis), a service cluster 205, and a service engine 206. Wherein: The client 100 can send task requests to the server 200. Correspondingly, the gateway 201 of the server 200 can receive these task requests. Specifically, a user or an upper-layer application can invoke the client 100 to send a task request to the server 200, which is used to request processing of a corresponding pending task. This disclosure does not limit the number of task requests; it can be determined according to the actual situation. The number of task requests can be one or more, and this disclosure does not limit this. For ease of description, this disclosure uses a single task request as an example to illustrate the relevant content, but this does not constitute a limitation.

[0052] After receiving a task request from client 100, gateway 201 can forward it to load balancer 202. Load balancer 202 can then distribute the task request to middleware 203 according to conventional load balancing strategies. These load balancing strategies include, but are not limited to, round-robin task distribution or other custom strategies, which will not be further limited or detailed in this disclosure. Correspondingly, middleware 203 can receive task requests from client 100.

[0053] After receiving a task request from client 100, the middleware 203 can send a request command to the database. This request command may include, but is not limited to, a combination of at least one or more of the following information: such as the task request identifier (request ID), the task type corresponding to the task request (also referred to as the task request type), or other custom information. This request command can be specifically used to actively request a specific business service, for example, to obtain a target service matching the task request identifier and / or type. Correspondingly, the database 204 can receive the request command. Specifically, after receiving the task request, the middleware 203 can send a query request command to the database 204. This query request command can be used to request a query for the target service used to process the task request. The query request may carry the task request identifier, and optionally may also carry, for example, the specific content of the task request, the task type of the task request, or other custom information. Correspondingly, the database 204 can receive the query request command and parse out the task request identifier and other information from it.

[0054] Upon receiving the aforementioned request command, the database 204 can respond by determining the target service information based on the identifier and / or type of the task request, and returning / sending a corresponding response result to the middleware 203. This response result may include at least the information of the target service. The target service may be an idle service in the service cluster 205, and the target service must match the identifier and / or type of the task request. The database registers all or some of the business services in the service cluster 205, all of which may be idle. The service cluster 205 may include one or more business services, which may include idle services or non-idle services, etc., without further limitation in this disclosure. Correspondingly, the middleware 203 can receive the response result and parse it to obtain the target service information, meaning that the middleware 203 can retrieve the target service for processing the task request from the database 204 via a pull method. The information of the aforementioned target service may refer to information used to describe the aforementioned target service, which may include, but is not limited to, the identifier of the target service, the Internet Protocol (IP) information of the target service, the port information of the target service, or other information used to route to the target service, etc. This disclosure does not impose further limitations or details on this.

[0055] This disclosure does not limit the implementation method for determining the target service. For example, this disclosure can first determine the type of the task request, specifically by parsing it directly from the request command, or by determining the corresponding task type based on the identifier of the task request, etc. This disclosure does not limit this. After determining the type of the task request, this disclosure selects the information of the corresponding idle service from the idle list that matches the type as the information of the target service. For example, the information of an idle service can be randomly / arbitrarily selected from the idle list as the information of the target service, or the information of an idle service with a small load (e.g., a load less than a preset load value) can be selected from the idle list as the information of the target service, etc. This disclosure does not limit or elaborate on this. The idle list is pre-maintained by the system. Each type can maintain a corresponding idle list. The idle list includes information on one or more business services in the service cluster 205 that support processing the task requests of the above type and are in an idle state, that is, the idle list includes information on one or more idle services that support processing the task requests of the above type. Among them, the business services in an idle state can also be simply referred to as idle services, and this disclosure does not limit or elaborate on this.

[0056] After receiving the response result, the middleware 203 can send the task request to the target service based on the information of the target service. Correspondingly, the target service can receive the task request and invoke the service engine 206 to process it. Specifically, after obtaining the information of the target service, the middleware 203 can push the task request to the target service for corresponding business processing. Taking a speech synthesis request as an example, the target service or each business service in the service cluster can be a speech synthesis service. After receiving the task request, the target service (e.g., the speech synthesis service) can perform speech synthesis on the text to be processed in the task request, etc. For details, please refer to the following description in this disclosure; it will not be elaborated here.

[0057] To better understand the embodiments of this disclosure, an example is provided below. Taking a speech synthesis service (tts-server) request as an example, after receiving the speech synthesis service request from client 100, middleware 203 can send a corresponding request command to database 204. This request command can include the identifier of the speech synthesis service request and its corresponding task type (which can be simply referred to as the type of speech synthesis service request). This request command can specifically be used to instruct the active retrieval / acquisition of a business service that matches the type of the speech synthesis service request. Accordingly, after receiving the request command, database 204 can respond to the request command by returning a corresponding response result to middleware 203. This response result can include information about the target service, such as the IP address and port of the target service. In a specific implementation, database 204 can respond to the request command by retrieving the information of an idle service from its own maintained idle list based on the task type corresponding to the speech synthesis service request, using this information as the target service information. This target service information is then carried in the response result and returned to middleware 203. This allows middleware 203 to obtain the target service information from database 204 by actively retrieving it. Then, the middleware 203 can actively push the above-mentioned speech synthesis service request to the above-mentioned target service for processing based on the information of the target service. This can be referred to in the relevant description in the previous embodiment, and will not be repeated here.

[0058] In an optional embodiment, after determining the target service, the database 204 and / or the middleware 203 can bind / associate the information of the target service with the task request. Specifically, for example, the IP information of the target service can be associated with the identifier ID of the task request to instruct the target service to process the task request.

[0059] In another optional embodiment, after determining the target service or the information of the target service, the database 204 can update / mark the status of the target service as non-idle to avoid assigning the target service to other task requests after receiving their identifiers, thus preventing load imbalance. Alternatively, after determining the target service, the database 204 can directly remove the information of the target service from the database 204 to remove the relevant information of the target service registered in the database 204, thus preventing the target service from being assigned to other task requests after receiving their identifiers, thus preventing load imbalance.

[0060] In another optional embodiment, after processing the task request, the target service can obtain the processing result corresponding to the task request. Optionally, the target service can encapsulate the processing result corresponding to the task request and sequentially transmit the encapsulated processing result to the client 100 through the middleware 203, the load balancer 202, and the gateway 201. After receiving the processing result corresponding to the task request, the client 100 can perform corresponding subsequent operations. For example, in a speech synthesis scenario, the processing result corresponding to the task request is speech audio. After receiving the processing result, the client 100 can use a player to play the speech audio so that the user can perceive the speech audio synthesized by the speech synthesis, etc. This disclosure does not impose further limitations or details on this.

[0061] By implementing embodiments of this disclosure, a task processing system is provided, comprising middleware, a database, a service cluster, and a service engine. The middleware, upon receiving a task request from a client, sends a request command to the database. The request command requests a target service matching the identifier and / or type of the task request, which requests processing of a task to be processed. The database, upon receiving the request command, returns a corresponding response to the middleware. The response includes information about the target service, which is an idle service in the service cluster that matches the identifier and / or type of the task request. At least one such service from the service cluster is registered in the database. The middleware, upon receiving the response, sends the task request to the target service based on the target service information. The target service, upon receiving the task request, invokes the service engine to process the task request. This allows the task processing system to use a push-pull approach to send the task request to an idle target service, avoiding long-tail problems when there are many task requests. This improves the reliability and efficiency of task processing.

[0062] In another alternative embodiment, each service in the service cluster 205 can be registered to the database 204. Preferably, idle services in the service cluster 205 can be registered to the database 204, while non-idle services cannot be registered to the database 204.

[0063] In another optional embodiment, the task processing system (specifically, service cluster 205 within the system) supports dynamic expansion, meaning that service cluster 205 supports expansion to add corresponding business services. Without changing or stopping any of the task processing system's front-end services, corresponding business services can be directly added and deployed within service cluster 205. Since the business service acts as a client relative to database 204, it can directly send its own information to database 204 for registration. Each added business service automatically registers its information, supporting unlimited expansion until a task request arrives, and then appropriately allocating idle business services for processing. For ease of description, this disclosure will use a business service as an example to illustrate the embodiments related to business service registration.

[0064] A service in service cluster 205 can send a registration request to database 204. This registration request is used to register the service with database 204. The registration request may carry information about the service, such as its identifier, IP address, port number, or other custom information. Correspondingly, database 204 can receive the registration request.

[0065] Upon receiving the registration request, database 204 can respond by registering the information of the aforementioned business service in database 204, that is, storing the information of the aforementioned business service in database 204. Optionally, database 204 can return / send a corresponding registration result to the aforementioned business service, which indicates whether the aforementioned business service has successfully registered in database 204. Accordingly, the aforementioned business service can receive the registration result to determine whether it has successfully registered in database 204.

[0066] In practical applications, when deploying and starting the aforementioned business services, this disclosure can integrate database 204 into the business services to obtain information about the locally deployed business services, such as their IP and port information. After integrating database 204, the business services can proactively send a registration request to database 204, encapsulating and packaging their information into the request, and sending it to database 204 in the format required by database 204 for storage. Upon completion of the operation, database 204 can reply with a success message, i.e., a registration result. This completes the registration of the aforementioned business services.

[0067] In another optional embodiment, to meet business needs, the task processing system supports a multi-level caching strategy. Please refer to [link to details]. Figure 3 This is a schematic diagram illustrating the structure of another task processing system according to an exemplary embodiment. For example... Figure 3 The task processing system may include Figure 2 All or some components of the system shown may also include a first cache 207. Wherein: The first cache 207 mentioned above can be used to store the processing results corresponding to high-frequency requests, wherein the high-frequency requests are business requests whose sending frequency and / or sending frequency exceed the corresponding threshold. After receiving the high-frequency request, the middleware can directly obtain the processing result corresponding to the high-frequency request from the first cache 207, so as to prevent / avoid sending the high-frequency request to the subsequent service cluster 205 and service engine 206 for processing.

[0068] In this disclosure, the aforementioned high-frequency requests can refer to task requests whose sending frequency and / or number of sending requests exceed a corresponding preset threshold. This preset threshold can be a threshold pre-defined by the system or the user based on actual circumstances. It can be an empirical value set based on user experience, or a statistical value calculated based on a series of experimental data, etc. This disclosure does not impose further limitations or details on this.

[0069] In practical applications, the first cache 207 is mainly used for caching on the middleware 203 side. If the same task request is repeatedly requested (i.e., a high-frequency request), the middleware 203 will not send the high-frequency request to the backend service cluster 205 for processing, but will directly return the corresponding processing result cached in the first cache 207 to the client 100. This operation is mainly to cache high-frequency requests, such as frequently used operations like high-frequency words, to reduce the inference calculations of the subsequent service engine 206 and reduce the cost of the server 200.

[0070] In another optional embodiment, this disclosure may also introduce a recording library into the aforementioned service engine 206 to accurately and promptly correct issues where the front-end and back-end components of the engine cannot be quickly repaired, thereby improving the user experience. Please see [link to relevant documentation]. Figure 4 This is a schematic diagram illustrating the internal structure of a service engine 206 according to an exemplary embodiment. Figure 4 The service engine 206 shown may include a front-end module 2061, a back-end module 2062, and a second cache 2063; wherein: The aforementioned second cache 2063 is mainly used to store the recording library, which may include, but is not limited to, any one or more of the following: health sample information, demand sample information, recording sample information, high-frequency sample information, or other custom-categorized sample information, etc.

[0071] The sample information mentioned above may include audio and its corresponding text. The purpose and priority of each type of sample information may differ, and they may be pre-defined by the system or the user. This disclosure can classify the sample information in the recording library into the following four categories based on its purpose and source, and configure corresponding priorities, as shown in Table 1 below: Table 1 name Uses and sources Update Notes Priority Health Sample Information (healthmrnt) Health Monitoring, an example of online monitoring, can be obtained by filtering from the original audio recording library. Once the online health monitoring function is developed, it will no longer be updated. Highest Requirement sample information (responses) Customized response samples based on product requirements, customized recordings, and high-quality end-to-end synthesis, etc. It's up to the product manager to decide. high Record sample information (recordings) The content includes business scripts and other materials from the original recording library. The recording library continues to expand to meet business needs. middle High-frequency sample information (Highfreqs) High-frequency responses, online high-frequency feedback examples, high-quality end-to-end synthesis, etc. Updated at a fixed interval (e.g., quarterly), with manual listening required to confirm quality before each update. Low As shown in Table 1 above, the priorities of the aforementioned health sample information, demand sample information, recording sample information, and high-frequency sample information decrease sequentially. Among these, the aforementioned health sample information aims to monitor abnormal conditions of the service engine 206 online in real time. For the service engine 206, this involves monitoring the correctness of the processing results and the real-time nature of the system response. Currently, the service engine 206 uses two response methods: cache hit and real-time response processing. The purpose of the aforementioned health sample information is to monitor the correctness of the cache hit response method. The principles for selecting such sample information (e.g., text and corresponding audio) can include pure Chinese, mixed Chinese and English, pure English samples, etc.; the selected text must be audio with normal speech; the selected text includes Chinese characters, English letters and simple words, Chinese and English commas and periods; and the speech quality must be ensured for recognition by the corresponding speech processing component / engine.

[0072] The above-mentioned sample requirements may include pre-defined high-quality or special processing results, such as voice responses, to facilitate the launch of new product features. Based on the needs of various product lines, when a task request from one of these product lines is met, a high-quality processing result, such as a voice response, can be returned.

[0073] The number of recorded sample information examples is limited, but it can enhance processing performance and shorten response time, such as improving voice quality. The number of high-frequency sample information examples is unlimited, and it can similarly enhance processing performance and shorten response time, such as improving voice quality.

[0074] It should be noted that this disclosure designs the aforementioned second cache 2063 within the service engine 206, rather than within the service cluster 205. The reasons for this include: the service engine 206 updates and iterates more rapidly, while the business services in the service cluster 205 update and iterate more slowly. When the service engine 206 is updated, the cached recording library on it is also updated synchronously, allowing for timely resolution of some issues. In the speech synthesis service, engineers are highly sensitive to the audio of the speech synthesis, making it easier to control audio quality and facilitating local operation and testing. The service engine 206 performs many subsequent processing operations. The audio format cached in the speech synthesis service can be PCM format, supporting various sampling rates such as 16kbps, 24kbps, and 48kbps. Depending on user needs, the combinations of these parameters are diverse and complex. Appropriate combinations within the service engine 206 can quickly meet different user requirements.

[0075] When the task request and the sample information in the recording library do not match, the front-end module 2061 can perform front-end business processing on the task request and send the obtained front-end processing result to the back-end module 2062 for processing.

[0076] The backend module 2062 can perform backend business processing on the task request when it matches the sample information in the recording library, thereby obtaining the processing result corresponding to the task request. Alternatively, the backend module 2062 can continue to perform backend business processing on the frontend processing result after receiving it, thereby obtaining the processing result corresponding to the task request.

[0077] After receiving the task request, the service engine 206 can use a preset method to match the task request with the sample information cached in the second cache 2063 to determine whether the sample information in the cache is matched. Taking the speech synthesis service as an example, after receiving the task request, the service engine 206 can match the text to be synthesized carried in the request with the sample text cached in the second cache 2063. If the match is successful, it is determined that the task request matches the sample information in the recording library; otherwise, it is determined that the task request does not match the sample information in the recording library.

[0078] If the task request matches the sample information in the recording library, it can be processed directly by the backend module 2063 without going through the frontend module 2062, thus obtaining the corresponding processing result. If the task request does not match the sample information in the recording library, it must first be processed by the frontend module 2062 and then by the backend module 2063 to obtain the corresponding processing result.

[0079] In another alternative embodiment, please refer to Figure 5 This is a schematic diagram illustrating the internal structure of another service engine 206 according to an exemplary embodiment. (See diagram below.) Figure 5 The service engine 206 shown may include Figure 4 All or some components of the engine shown may also include communication interface 2064, task scheduling queue 2065, inference library 2066, and other custom functional components. For example... Figure 5 In this process, the service engine 206 receives task requests passed through the business service via the communication interface 2064, and stores the task requests in the task scheduling queue 2065 for processing. For example, they can be processed according to their respective priorities, with higher-priority task requests being processed first. If task requests of the same level are to be processed on a first-come, first-served basis, etc., this disclosure does not impose further limitations or details. This disclosure does not limit the number of inference libraries 2066 mentioned above; they can be customized according to actual needs. The illustration shows one inference library 2066, but this does not constitute a limitation. This disclosure also does not limit the type of the communication interface 2064 mentioned above; it can include, but is not limited to, C Application Programming Interface (C API), general network interface, or other interfaces used for communication transmission.

[0080] In another optional embodiment, the front-end module 2062 can select the task request from the task scheduling queue 2065 according to the priority of the task request, and then perform corresponding front-end business processing on the task request. The priority of the task request can be pre-defined by the system or the user, for example, it can be set according to the type of the task request. For example, in the speech synthesis service, the task request carries text to be synthesized. This disclosure can directly determine the priority of the task request according to the type of the text to be synthesized. For example, if the type of the text to be synthesized can match the four types shown in Table 1 above, then the priority of the task request can be directly determined according to the type of the sample information in Table 1 to which the text to be synthesized belongs.

[0081] In another optional embodiment, the backend module 2063 may also continue to perform corresponding backend business processing on the frontend processing results output by the frontend module 2062 according to the priority of the task request. Alternatively, the backend module 2063 may select the task request from the task scheduling queue 2065 according to the priority of the target sample information matching the task request, and then perform corresponding backend business processing on the task request.

[0082] This disclosure does not limit the internal structure and business processing of the aforementioned front-end module 2062 and back-end module 2063. Specifically, taking a speech synthesis service as an example, the front-end module 2062 can parse task requests to extract attribute information of the text task, which may include, but is not limited to, attribute options such as volume, speaker, tone, speech rate, output corresponding audio sampling rate, audio format, and vertical domain information. Optionally, the front-end module 2062 can also perform pre-processing such as sentence segmentation, text normalization, word segmentation, part-of-speech tagging, punctuation restoration, and text-to-speech conversion. The back-end module 2063 can perform post-processing such as generating linguistic feature sequences, acoustic parameter prediction, vocoder, volume control, noise suppression, reverberation effects, and pitch shifting; this disclosure does not provide further limitations or details in these areas. After the back-end module 2063 completes processing and obtains the processing result (such as synthesized speech audio) corresponding to the task request, it can be passed through to the corresponding business service in the service cluster 205, such as the aforementioned target service; this disclosure does not provide further limitations or details in these areas.

[0083] In another optional embodiment, when the number of user task requests surges, the task processing system can dynamically and in real-time control the lifecycle of task requests to handle them reasonably. This is typically achieved by introducing expiration times and / or priorities to prevent a sudden surge in traffic from overwhelming the system. For an introduction to priorities, please refer to the relevant descriptions in the preceding embodiments. The following describes relevant embodiments related to expiration times. Specifically, an expiration time can be set for the target information stored in the database 204. This target information can include, but is not limited to, any one or more combinations of the following: the identifier of the task request, the type of the task request, information about the business service (e.g., IP information or port information of the business service), the processing result corresponding to the task request (e.g., audio data from speech synthesis), or other custom information. Optionally, the target information and its expiration time can be stored in the database 204 in the form of key-value pairs. In practical applications, the database 204 can store the expiration times of the target information in an expiration dictionary (expires dict), as detailed in [link to relevant documentation]. Figure 6 This is a schematic diagram illustrating the data structure of an expired dictionary according to an exemplary embodiment. For example... Figure 6As shown, the target information and its expiration time in the expiration dictionary can be stored in key-value pair format. The target information, such as the identifier of a task request or the IP information of the aforementioned business service, can be represented as a key. The expiration time of the target information can be represented as a value. This disclosure does not limit the representation of the keys and values; for example, a long integer type can be used to represent the expiration time of the target information. This disclosure does not impose further limitations or details on this aspect.

[0084] After receiving the key value of the target information, database 204 can look up and determine the expiration time (value) of the target information based on the key value. Furthermore, it can compare the expiration time of the target information with the current system time to determine whether the target information has expired. Specifically, if the expiration time of the target information is less than or equal to the current system time, it can be determined that the target information has expired, and optionally, the target information can be deleted from database 204. Conversely, if the expiration time of the target information is greater than the current system time, it can be determined that the target information has not expired, and subsequent processes can continue.

[0085] For example, when the target information includes the identifier of the task request, the database 204, upon receiving the identifier, can first determine the expiration time of the task request using a key-value pair method. Based on the expiration time, it can determine whether the task request has expired. If it is determined that the task request has expired, the process can be terminated directly. Conversely, if it is determined that the task request has not expired, the corresponding target service can be determined based on the identifier of the task request, and then the target service can be used to process the task request. This can be referred to in the relevant descriptions in the preceding embodiments, and will not be repeated here.

[0086] In another optional embodiment, when there are multiple target information items, the database 204 can determine the number of expired target information items according to the expiration determination principle. If the number of expired target information items exceeds a preset number, all remaining target information items in the database 204 can be traversed, and the expired target information items in the database 204 can be directly deleted and cleaned up according to the implementation principle of the previous optional embodiment. The preset number can be a number set by the system or by the user. For example, it can be an empirical value set based on user experience, or a statistical value calculated based on a series of experimental data, etc., and this disclosure does not limit it.

[0087] For example, suppose the number of target information items is 20. The task processing system can periodically and randomly select 20 key-value pairs (i.e., the target information items) from the expired field at regular intervals. It checks and determines whether these 20 key-value pairs are expired and deletes the expired ones. If the number of expired key-value pairs checked in this round exceeds 5 (5 / 20 = 25%), meaning that the proportion of expired key-value pairs in the sampled key-value pairs exceeds 25%, then all the above steps can be repeated to proactively screen for expired key-value pairs in the expired dictionary. Conversely, if the number of expired key-value pairs checked in this round does not exceed 5, meaning that the proportion of expired key-value pairs does not exceed 25%, then the system can wait for the next trigger before checking for expired key-value pairs.

[0088] In another optional embodiment, the aforementioned front-end module 2062 can also save / store the processing results corresponding to the front-end module 2062, specifically, the processing results corresponding to each internal component of the front-end module 2062, to a preset result table. This preset result table is pre-defined by the system or user and assigned to the task request; for example, one preset result table can be configured for each task request (or task request identifier). This preset result table can be used to store the input information and output processing results of each component (e.g., the aforementioned front-end module 2062 or its various internal components) for subsequent processing, such as data tracking, performance indicator analysis, or other custom processing. This disclosure does not impose further limitations or details on this aspect.

[0089] In another optional embodiment, the backend module 2063 may also store the processing results corresponding to itself, specifically, for example, the processing results corresponding to each internal component of the backend module 2063, in a preset result table for users to view, track data, or perform performance analysis, etc. For a description of the preset result table, please refer to the relevant descriptions in the foregoing embodiments; they will not be repeated here.

[0090] It should be noted that the task processing system disclosed herein can be applied to scenarios such as speech synthesis, speech recognition, text processing, or other custom scenarios. Accordingly, the aforementioned task processing system may include, but is not limited to, large-scale speech synthesis model systems, large-scale speech recognition model systems, large-scale voice interaction model systems, large-scale text model systems, or other computationally intensive, heavily computationally-intensive neural network inference systems. Combined with... Figure 2The task processing system shown here, taking a large-scale voice interaction model system as an example, provides a brief introduction to related content. This disclosure does not limit the internal structure of the aforementioned large-scale voice interaction model system. For example, the large-scale voice interaction model system is a highly integrated system encompassing multiple module components, from wake-up and speech recognition to semantic understanding and speech synthesis. Through the collaborative work of these modules, intelligent interaction between the user and the vehicle can be achieved, making driving safer. When a user requests the vehicle's large-scale voice interaction model system, corresponding input information is typically added to each module. When the voice request reaches the system, it sequentially undergoes reasoning through the wake-up, speech recognition, semantic understanding, and speech synthesis modules, storing the input information and reasoning results of each module in a pre-defined result table in database 204. Throughout the process, the system can automatically trace the entire process using database 204 based on the user request identifier (i.e., the task request identifier). Please refer to... Figure 7 This is a schematic diagram illustrating a user request processing flow according to an exemplary embodiment. For example... Figure 7 The process shown may include the following implementation steps: S71. Obtain the version information and software environment information of service engine 206 so as to determine whether it is compatible when updating service engine 206 next time. This disclosure will not make too many limitations or details in this regard.

[0091] S72. Start each module component in the system, so that each module component is in the on / start state.

[0092] S73. Create a corresponding preset result table based on the type of user request (i.e., the type of task request). Receive voice signals (i.e., voice requests) from the user, retain the test results corresponding to the voice signals, and store the test results in the aforementioned preset result table.

[0093] S74. The voice signal is processed by the wake-up module component, the processing result corresponding to the wake-up module component is retained, and the processing result is added to the above-mentioned preset result table.

[0094] S75. The voice information processed by the wake-up module component in step S74 is then processed by the voice recognition module component. The processing result corresponding to the voice recognition module component is retained and added to the preset result table mentioned above.

[0095] S76. The text information processed by the speech recognition module component in step S75 is then processed by the semantic understanding module component. The processing result corresponding to the semantic understanding module component is retained and added to the preset result table mentioned above.

[0096] S77. The text information processed by the semantic understanding module component in step S76 is then processed by the result judgment / satisfaction module component. The processing result corresponding to the result judgment module component is retained and added to the above-mentioned preset result table.

[0097] S78. The text information processed by the result determination module component in step S77 is then processed by the speech synthesis module component. The processing result corresponding to the speech synthesis module component is retained and added to the preset result table mentioned above.

[0098] S79. After the audio signal is processed by the speech synthesis module component in step S78, it is processed by the audio module component, and the processing result corresponding to the audio module component (i.e., the audio slice result) is retained and added to the above-mentioned preset result table.

[0099] In practical applications, intermediate results of each module component are stored in a pre-defined result table in database 204. When other modules obtain the processing results of their dependent components, they can directly retrieve them from database 204 without needing to transmit them through a series of network transmissions, thus saving network bandwidth. Furthermore, the pre-defined result table facilitates subsequent processing of the input and output information of each module component, such as analyzing performance metrics like time consumption, thereby effectively improving system performance.

[0100] As can be seen, this embodiment of the disclosure, by adjusting the architecture of the task processing system and designing a refined distribution mechanism, allocates task requests to multiple downstream nodes (such as middleware, service clusters, and service engines), achieving balanced utilization of computing resources. This effectively solves problems such as the long tail problem and uneven task distribution in task processing, improving the efficiency of system services. By offloading the complex business logic of task requests to the service cluster and adopting a push-pull combined sending strategy, not only can the system's responsiveness be improved, but also its high availability and scalability can be achieved. This allows the system to flexibly respond to various business scenarios, ensuring the continuity and stability of system services. Multi-level caching is supported. Without retraining the model system, by introducing a recording library and multi-level caching audio in the service engine, problems such as the inability of the service engine's front-end and back-end modules to quickly repair can be corrected in a timely and accurate manner, reducing user-perceived latency errors and ensuring the accuracy and fluency of speech synthesis, thereby improving the user experience. When the number of task requests surges suddenly, the lifecycle of task requests is dynamically controlled, and expiration time and task request priorities are introduced to reasonably handle task requests, solving the problem of sudden increases in task requests and effectively mitigating the impact of instantaneous high traffic on the system. The system can reasonably handle task requests, ensuring stable system operation. The task processing system disclosed herein exhibits strong scalability and can be rapidly applied to computationally intensive, heavy-computation neural network inference systems, such as large-scale speech recognition systems, large-scale text recognition systems, and multimodal large-scale systems. It can quickly collect end-to-end information, such as the input and output information of each module component within the system, facilitating subsequent aggregation and analysis. This improves the overall scalability of the system and enhances the user experience. This end-to-end analysis capability is of great significance for building intelligent and efficient task processing systems.

[0101] By implementing embodiments of this disclosure, a task processing system is provided, comprising middleware, a database, a service cluster, and a service engine. The middleware, upon receiving a task request from a client, sends a request command to the database. The request command requests a target service matching the identifier and / or type of the task request, which requests processing of a task to be processed. The database, upon receiving the request command, returns a corresponding response to the middleware. The response includes information about the target service, which is an idle service in the service cluster that matches the identifier and / or type of the task request. At least one such service from the service cluster is registered in the database. The middleware, upon receiving the response, sends the task request to the target service based on the target service information. The target service, upon receiving the task request, invokes the service engine to process the task request. This allows the task processing system to use a push-pull approach to send the task request to an idle target service, avoiding long-tail problems when there are many task requests. This improves the reliability and efficiency of task processing.

[0102] Based on the foregoing embodiments, please refer to Figure 8 This is a flowchart illustrating a task processing method according to an exemplary embodiment. Figure 8 The method shown can be applied to the aforementioned Figures 2-7 In the task processing system shown, the method may include the following implementation steps. The task processing system may include a client 100 and a server 200. The server 200 may include a gateway 201 (apisix), a load balancer 202 (nginx), middleware 203 (outer-server), a database 204 (redis), a service cluster 205, and a service engine 206.

[0103] S801. After receiving a task request from the client through the middleware 203, a request command is sent to the database 204. The request command is used to request the acquisition of a target service that matches the identifier and / or type of the task request. The task request is used to request the processing of the task to be processed.

[0104] Users or upper-layer applications can invoke client 100 to send task requests to server 200. These task requests are used to request processing of corresponding pending tasks. This disclosure does not limit the number of task requests; it can be determined based on actual circumstances. The number of task requests can be one or more, and this disclosure does not limit this. For ease of description, this disclosure uses a single task request as an example below, but this does not constitute a limitation. Correspondingly, gateway 201 of server 200 can receive the task request and pass it through to load balancer 202. Load balancer 202 can distribute the task request to middleware 203 according to traditional load balancing strategies. These load balancing strategies include, but are not limited to, round-robin task distribution or other custom strategies, which this disclosure does not limit or elaborate on. Correspondingly, middleware 203 can receive task requests from client 100.

[0105] After receiving a task request from client 100, the middleware 203 can send a request command to the database. This request command may include, but is not limited to, a combination of at least one or more of the following information: such as the task request identifier (request ID), the task type corresponding to the task request (also referred to as the task request type), or other custom information. This request command can be specifically used to actively request a specific business service, for example, to obtain a target service matching the task request identifier and / or type. Correspondingly, the database 204 can receive the request command. Specifically, after receiving the task request, the middleware 203 can send a query request command to the database 204. This query request command can be used to request a query for the target service used to process the task request. The query request command may carry the task request identifier, and optionally may also carry, for example, the specific content of the task request, the task type of the task request, or other custom information. Correspondingly, the database 204 can receive the query request command and parse out the task request identifier and other information from it.

[0106] S802. After receiving the request command, the database 204 returns a corresponding response result to the middleware 203 in response to the request command. The response result includes information about the target service. The target service is a business service in the service cluster of the task processing system that is in an idle state and matches the identifier and / or type of the task request. At least one of the business services in the service cluster is registered in the database.

[0107] Upon receiving the aforementioned request command, database 204 can respond by determining the target service information based on the identifier and / or type of the task request, and then returning / sending the target service information to middleware 203 in the response result. The target service can be an idle service in service cluster 205 that matches the identifier and / or type of the task request. Correspondingly, after receiving the response result, middleware 203 can parse the target service information from it; that is, middleware 203 can retrieve the target service for processing the task request from database 204 via a pull mechanism.

[0108] S803. After receiving the response result, the middleware 203 sends the task request to the target service based on the information of the target service.

[0109] S804. After receiving the task request, the target service calls the corresponding service engine to process the task request.

[0110] After obtaining the information of the target service, the middleware 203 can send the task request to the target service. Correspondingly, the target service can receive the task request and invoke the service engine 206 to process it. Specifically, after obtaining the information of the target service, the middleware 203 uses a push method to pass-through / send the task request to the target service for corresponding business processing.

[0111] For any content not covered or described in this disclosure, please refer to the foregoing. Figures 1-7 The relevant descriptions in the embodiments will not be repeated here.

[0112] By implementing the embodiments of this disclosure, after receiving a task request from a client, the task processing system, through the middleware, sends a request command to the database. The request command requests a target service matching the identifier and / or type of the task request, which requests processing of the task to be processed. Upon receiving the request command, the database returns a corresponding response to the middleware. The response includes information about the target service, which is an idle service in the service cluster of the task processing system that matches the identifier and / or type of the task request. At least one such service from the service cluster is registered in the database. After receiving the response, the middleware sends the task request to the target service based on the target service information. Upon receiving the task request, the target service invokes its corresponding service engine to process the task request. In this way, the task processing system can use a push-pull approach to send the task request to an idle target service for processing, avoiding technical problems such as long tails when there are many task requests. This improves the reliability and efficiency of task processing.

[0113] Please see Figure 9 This is a schematic diagram illustrating the structure of a task processing device according to an exemplary embodiment. Figure 9 The device described above can be applied to the task processing system shown in the figure above, and the device may include: The transceiver module 901 is configured to send a request command to the database after receiving a task request from a client through the middleware. The request command is used to request the acquisition of a target service that matches the identifier and / or type of the task request. The task request is used to request the processing of the task to be processed. The transceiver module 901 is further configured to, after receiving the request command through the database, return a corresponding response result to the middleware in response to the request command. The response result includes information about the target service. The target service is a business service in the service cluster of the task processing system that is in an idle state and matches the identifier and / or type of the task request. At least one of the business services in the service cluster is registered in the database. The transceiver module 901 is further configured to send the task request to the target service based on the information of the target service after receiving the response result through the middleware; The processing module 902 is configured to, after receiving the target service, call the corresponding service engine to process the task request through the target service.

[0114] For any content not covered or described in this disclosure, please refer to the foregoing. Figures 1-8 The relevant descriptions in the embodiments will not be repeated here.

[0115] By implementing the embodiments of this disclosure, the above-described apparatus can, after receiving a task request from a client, send a request command to the database via the middleware. The request command requests a target service matching the identifier and / or type of the task request, which requests processing of the task to be processed. Upon receiving the request command, the database returns a corresponding response to the middleware. The response includes information about the target service, which is an idle service in the service cluster of the task processing system that matches the identifier and / or type of the task request. At least one such service from the service cluster is registered in the database. Upon receiving the response, the middleware sends the task request to the target service based on the target service information. Upon receiving the task request, the target service invokes its corresponding service engine to process the task request. In this way, the task processing system can use a push-pull approach to send the task request to an idle target service for processing, avoiding technical problems such as long tails when there are many task requests. This improves the reliability and efficiency of task processing.

[0116] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the task processing method provided in this disclosure.

[0117] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the task processing method described above when executed by the programmable device.

[0118] Please see Figure 10 This is a schematic diagram illustrating the structure of a computer device according to an exemplary embodiment. For example, the computer device 1000 described above can be provided as a server. (Refer to...) Figure 10 The computer device 1000 includes a processing component 1022, which further includes one or more processors, and memory resources represented by memory 1032 for storing instructions, such as application programs, that can be executed by the processing component 1022. The application programs stored in memory 1032 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1022 is configured to execute instructions to perform the aforementioned task processing methods.

[0119] The computer device 1000 may also include a power supply component 1026 configured to perform power management of the computer device 1000, a wired or wireless network interface 1050 configured to connect the computer device 1000 to a network, and an input / output interface 1058. The computer device 1000 can operate on an operating system stored in memory 1032.

[0120] Please see Figure 11 This is a schematic diagram illustrating the structure of a chip according to an exemplary embodiment. For example... Figure 11 The chip 1100 shown includes a processor 1101 and an interface 1102. Optionally, it may also include a memory 1103. The number of processors 1101 can be one or more, and the number of interfaces 1102 can be multiple.

[0121] In one embodiment, for the case where the chip is used to implement the method embodiments described in this disclosure: The interface 1102 is used to receive or output signals; The processor 1101 is used to execute some or all of the contents of the task processing method embodiment.

[0122] Understandably, the processor in this embodiment of the disclosure can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiment can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0123] Understandably, the memory in the embodiments of this disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0124] It should be noted that the descriptions of the storage media, devices, and chip embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage media, storage media, and device embodiments of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0125] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0126] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A task processing system, characterized in that, The system includes middleware, a database, a service cluster, and a service engine; wherein: The middleware is used to send a request command to the database after receiving a task request from the client. The request command is used to request the acquisition of a target service that matches the identifier and / or type of the task request. The task request is used to request the processing of the task to be processed. The database is configured to return a corresponding response result to the middleware in response to the request command after receiving the request command. The response result includes information about the target service. The target service is a business service in the service cluster that is in an idle state and matches the identifier and / or type of the task request. At least one of the business services in the service cluster is registered in the database. The middleware is also used to send the task request to the target service based on the information of the target service after receiving the response result; The target service is used to invoke the service engine to process the task request after receiving it.

2. The system according to claim 1, characterized in that, The service cluster supports the addition of corresponding business services, wherein: The business service is used to send a registration request to the database, and the registration request carries information about the business service. The database is also used to respond to the registration request by registering the information of the business service into the database; and to send a corresponding registration result to the business service, the registration result being used to indicate whether the business service has been successfully registered.

3. The system according to claim 1, characterized in that, The system also includes a first cache; The first cache is used to store the processing results corresponding to high-frequency requests; The middleware is also used to obtain the processing result corresponding to the high-frequency request from the first cache after receiving the high-frequency request, so as to prevent the high-frequency request from being sent to the service cluster for processing; The high-frequency request is a service request whose sending frequency and / or number of times exceeds a corresponding threshold.

4. The system according to claim 1, characterized in that, The service engine includes a front-end module, a back-end module, and a second cache; The second cache is used to store the recording library, which includes at least one of the following: health sample information, demand sample information, recording sample information, and high-frequency sample information. The front-end module is used to perform front-end business processing on the task request when the task request and the sample information in the recording library do not match, and send the obtained front-end processing result to the back-end module. The backend module is used to perform backend business processing on the task request when the task request matches the sample information in the recording library, and obtain the processing result corresponding to the task request; or, after receiving the frontend processing result, perform backend business processing on the frontend processing result to obtain the processing result corresponding to the task request.

5. The system according to claim 4, characterized in that, The front-end module is further configured to perform front-end business processing on the task request according to the priority of the task request; and / or, The backend module is further configured to perform backend business processing on the task request according to the priority of the sample information matching the task request; or, to perform backend business processing on the frontend processing result according to the priority of the task request. The priority of the health sample information, the demand sample information, the recording sample information, and the high-frequency sample information decreases sequentially.

6. The system according to claim 1, characterized in that, The target information stored in the database is set with a corresponding expiration time. The target information includes at least one of the following: the identifier of the task request, the type of the task request, the information of the business service, and the processing result corresponding to the task request. The database is used to determine the expiration time of the target information based on the key value of the target information after receiving the key value of the target information; The database is also used to determine that the target information has expired when the expiration time of the target information is less than or equal to the current system time, and to delete the target information from the database. The target information and its expiration time are stored in the database as key-value pairs.

7. The system according to claim 6, characterized in that, When there are multiple target information items The database is further configured to, when the number of expired target information items exceeds a preset number, traverse all target information items in the database and repeatedly execute the step of determining the expiration time of the target information item based on the key value of the target information item after receiving the key value of the target information item.

8. The system according to any one of claims 1-7, characterized in that, The database is also used to update the status of the target service to a non-idle state; or to remove the information of the target service from the database.

9. The system according to any one of claims 1-7, characterized in that, The front-end module is further configured to store the processing results corresponding to the front-end module into a preset result table; and / or, The backend module is also used to store the processing results corresponding to the backend module into the preset result table.

10. A task processing method, characterized in that, Applied to a task processing system, the system including middleware, database, service cluster, and service engine, the method includes: After receiving a task request from a client, the middleware sends a request command to the database. The request command is used to request the acquisition of a target service that matches the identifier and / or type of the task request. The task request is used to request the processing of the task to be processed. After receiving the request command, the database returns a corresponding response result to the middleware in response to the request command. The response result includes information about the target service. The target service is a business service in the service cluster of the task processing system that is in an idle state and matches the identifier and / or type of the task request. At least one of the business services in the service cluster is registered in the database. After receiving the response result, the middleware sends the task request to the target service based on the information of the target service. After receiving the task request, the target service invokes the corresponding service engine to process the task request.

11. The method according to claim 10, characterized in that, The task processing system includes the system as described in any one of claims 1 to 9 above.

12. A computer device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions to implement the steps of the method of claim 10 or 11.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the steps of the method of claim 10 or 11.

14. A chip, characterized in that, It includes a processor and an interface; the processor is used to read instructions to execute the method of claim 10 or 11.

Images