Task processing method and apparatus, storage medium, and electronic device

Abstract

The present disclosure relates to a task processing method, a task processing device, a storage medium and an electronic device, and relates to the technical field of computers. The method comprises: receiving request data sent by a target interface, converting the request data into first data, and generating a to-be-processed task using the first data; obtaining a task type of the to-be-processed task, and putting the to-be-processed task into a task input queue corresponding to the task type; obtaining a task processing template, and according to a corresponding relationship between a task type and a processing model included in the task processing template, pushing the to-be-processed task included in the task input queue to the corresponding processing model for processing to obtain a processing result; and sending the processing result to a task output queue corresponding to the task type of the to-be-processed task, and pushing the processing result to the target interface through the task output queue to complete processing of the to-be-processed task. The present disclosure reduces the coupling between the interface type and the core basic database type.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a task processing method, a task processing apparatus, a storage medium, and an electronic device. Background Technology

[0002] In software project development, in order to process, aggregate, merge, and store task processing results, as well as receive various complex data requests sent by the front end, a complete data framework and specifications need to be built.

[0003] In existing technologies, different technologies can be selected for different functionalities when building a complete system framework. In actual development, one or two technologies are usually chosen to quickly implement the complete functionality of the system. While choosing one or two technologies to implement a specific function may not present problems in the proof-of-concept phase, a series of issues arise during actual development, including:

[0004] On the one hand, in order to improve development efficiency, the definition of basic data types is based on the data used by the interface, which results in strong coupling between the interface type and the core basic database type, reducing the code reusability. On the other hand, there may be complex call relationships between different tasks of the same type. When different framework mechanisms are coupled in, task tracing not only involves the relationship of the call context, but also the mutual connection between different interface engines. Furthermore, the task entry points of different frameworks are different, making task tracing more difficult.

[0005] Therefore, a new task processing method is needed.

[0006] It should be noted that the information in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to provide a task processing method, a task processing device, a storage medium, and an electronic device, thereby overcoming, to at least a certain extent, the problem of strong coupling between interface types and core basic database types caused by the limitations and defects of related technologies.

[0008] According to one aspect of this disclosure, a task processing method is provided, comprising:

[0009] Receive request data sent by the target interface, convert the request data into first data, and use the first data to generate a task to be processed;

[0010] Obtain the task type of the task to be processed, and place the task to be processed into the task input queue corresponding to the task type;

[0011] Obtain a task processing template, and based on the correspondence between the task types included in the task processing template and the processing model, push the tasks to be processed included in the task input queue to the corresponding processing model for processing to obtain the processing results;

[0012] The processing result is sent to the task output queue corresponding to the task type of the task to be processed, and the processing result is pushed to the target interface through the task output queue to complete the processing of the task to be processed.

[0013] In one exemplary embodiment of this disclosure, receiving request data sent by a target interface, converting the request data into first data, and generating a task to be processed using the first data includes:

[0014] Based on the data communication method of the target interface, determine the interface conversion function corresponding to the target interface;

[0015] The request data is converted into first data corresponding to a preset data structure according to the interface conversion function;

[0016] The first data is used to generate a task to be processed.

[0017] In one exemplary embodiment of this disclosure, obtaining the task type of the task to be processed and placing the task to be processed into a task input queue corresponding to the task type includes:

[0018] Based on the task to be processed, create a mutually exclusive resource corresponding to the task to be processed;

[0019] Obtain the task type of the task to be processed, and determine the task input queue corresponding to the task type;

[0020] The task to be processed is placed in the task input queue and waits for the mutex resource.

[0021] In one exemplary embodiment of this disclosure, before obtaining the task processing template, the task processing method further includes:

[0022] Determine the correspondence between task types and processing models for those task types; wherein the relationship between task types and processing models is many-to-many.

[0023] The correspondence is processed using the specialization syntax of class templates to generate the task processing template.

[0024] In one exemplary embodiment of this disclosure, the tasks to be processed included in the task input queue are pushed to the corresponding processing model for processing to obtain processing results, including:

[0025] Read the tasks to be processed from the task input queue, and push the tasks to be processed to the processing model corresponding to the task type according to the task type.

[0026] The task to be processed is processed by the task processing function included in the processing model to obtain the processing result.

[0027] In one exemplary embodiment of this disclosure, the processing result is sent to a task output queue corresponding to the task type of the task to be processed, and the processing result is pushed to the target interface through the task output queue, including:

[0028] Obtain the processing result of the task to be processed, and push the processing result to the target interface through the mutual exclusion resource.

[0029] In one exemplary embodiment of this disclosure, pushing the processing result to the target interface through the mutually exclusive resource includes:

[0030] Upon receiving notification of the mutual exclusion resource, the processing result of the task to be processed is obtained from the task output queue.

[0031] The processing result is converted into a target processing result corresponding to the target interface through the interface conversion function, and the target processing result is sent to the target interface.

[0032] According to one aspect of this disclosure, a task processing apparatus is provided, comprising:

[0033] The pending task generation module is used to receive request data sent by the target interface, convert the request data into first data, and generate pending tasks using the first data.

[0034] The pending task input module is used to obtain the task type of the pending task and put the pending task into the task input queue corresponding to the task type.

[0035] The pending task processing module is used to obtain a task processing template, and according to the correspondence between the task types included in the task processing template and the processing model, push the pending tasks included in the task input queue to the corresponding processing model for processing to obtain the processing result.

[0036] The processing result sending module is used to send the processing result to the task output queue corresponding to the task type of the task to be processed, and push the processing result to the target interface through the task output queue to complete the processing of the task to be processed.

[0037] According to one aspect of this disclosure, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the task processing method described in any of the preceding claims.

[0038] According to one aspect of this disclosure, an electronic device is provided, comprising:

[0039] Processor; and

[0040] Memory for storing the executable instructions of the processor;

[0041] The processor is configured to execute any of the above-described task processing methods by executing the executable instructions.

[0042] This disclosure provides a task processing method, which includes: receiving request data sent by a target interface, converting the request data into first data, and generating a task to be processed using the first data; obtaining the task type of the task to be processed and placing the task to be processed into a task input queue corresponding to the task type; obtaining a task processing template, and according to the correspondence between the task types included in the task processing template and the processing models, pushing the tasks to be processed included in the task input queue to the corresponding processing models for processing to obtain processing results; sending the processing results to a task output queue corresponding to the task type of the task to be processed, and pushing the processing results to the target interface through the task output queue to complete the processing of the task to be processed. The task processing method, by converting received request data into first data, generating tasks to be processed based on the first data, and then processing these tasks, solves the problem of strong coupling between interface types and core basic database types in existing technologies, thus improving code reusability. On the other hand, by placing tasks to be processed into corresponding task input queues and pushing processing results through task output queues, each task to be processed has a corresponding task queue to achieve a closed loop of task transmission, solving the problem of difficult task tracking in existing technologies. Furthermore, by determining the task processing model corresponding to the task to be processed through a task processing template and completing the processing of the task to be processed through the corresponding task processing model, the decoupling between task processing and task transmission is achieved, improving task processing efficiency.

[0043] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0045] Figure 1 This illustration schematically depicts a scenario of a service platform in the prior art according to an example embodiment of this disclosure.

[0046] Figure 2 A flowchart illustrating a task processing method according to an example embodiment of the present disclosure is shown schematically.

[0047] Figure 3 A block diagram of a task processing system according to an example embodiment of the present disclosure is shown schematically.

[0048] Figure 4 The flowchart illustrating a method for generating tasks to be processed according to an example embodiment of the present disclosure is shown schematically.

[0049] Figure 5 The flowchart illustrates a method for placing a task to be processed into a corresponding task input queue according to an example embodiment of the present disclosure.

[0050] Figure 6 The flowchart illustrates a task processing method prior to obtaining a task processing template according to an example embodiment of the present disclosure.

[0051] Figure 7 The flowchart illustrates a method for obtaining a processing result according to an example embodiment of the present disclosure.

[0052] Figure 8 This illustration shows a scenario of a task processing template processing a task to be processed according to an example embodiment of the present disclosure.

[0053] Figure 9 The flowchart illustrates a method for pushing processing results to a target interface according to an example embodiment of the present disclosure.

[0054] Figure 10 A flowchart illustrating a task processing method according to an example embodiment of the present disclosure is shown schematically.

[0055] Figure 11 A block diagram of a task processing apparatus according to an exemplary embodiment of the present disclosure is shown schematically.

[0056] Figure 12An electronic device for implementing the above-described task processing method is illustrated according to an example embodiment of the present disclosure. Detailed Implementation

[0057] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make the invention more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention may be practiced with one or more of these specific details omitted, or other methods, components, apparatus, steps, etc., may be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the invention.

[0058] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0059] The main purpose of a video structuring service platform is to process, aggregate, fuse, and store the processing results from the video structuring processing platform, as well as to handle various complex query requests from the front end. Its basic structure can be referenced from [reference needed]. Figure 1 As shown. In Figure 1Data communication methods in China can include RESTful (Representational State Transfer) APIs (Application Programming Interfaces), MQ (Message Queues), RPC (Remote Procedure Call), and WebSocket (a full-duplex communication protocol based on TCP). Different communication methods have different implementation frameworks and specifications. RESTful APIs have specifications such as Swagger (a standardized and complete framework for generating, describing, invoking, and visualizing RESTful web services) and OpenAPI. MQs have frameworks such as RabbitMQ (an open-source high-level message queue protocol), Kafka (a high-throughput distributed publish-subscribe messaging system), and RocketMQ (an open-source distributed messaging system). RPC implementations include gRPC (an open-source remote procedure call system) and bRPC (a remote procedure call network framework developed by Baidu). The database used for storage also varies depending on the implementation system, such as MySQL (a relational database management system), MongoDB (a database based on distributed file storage), Elasticsearch (a search server based on Lucene), and HBase (Hadoop). Database (column-oriented open-source databases), etc.

[0060] In real-world project development, when a specific function needs to be implemented, choosing one or two related technologies for development may not cause major problems during the proof-of-concept phase. However, this can lead to a series of issues during subsequent platform development, including:

[0061] 1) In actual development, to improve development efficiency, the definition of basic data types is often based on the data used by the interface. When the requirements change and multiple interfaces need to be supported, or when another interface is used, the inconsistency of field definitions between different interfaces leads to low code reusability. If compatibility is achieved by expanding the basic data, the data dictionary of the basic data will keep changing, which will affect the development of the project. Existing technologies include the SWIG solution and the Swagger Generator solution. SWIG starts from a unified interface description and can automatically generate code generators for multiple languages ​​and multiple interfaces. Swagger Generator is a Swagger-based interface solution that can generate code generators for RESTful interfaces based on code templates. However, the code structure of these two solutions is relatively fixed, the code volume is large, and the types of interfaces supported are limited, which is not flexible.

[0062] 2) Because the underlying data structure undergoes different serialization and deserialization operations in each interface, if the underlying data format for a specific function in the upstream structured processing module changes, all corresponding interfaces must be modified accordingly to adapt. Simultaneously, the subsequent data storage and processing parts must also be modified accordingly. These changes are distributed across different parts of the code, involving numerous modification points, causing considerable inconvenience for feature upgrades.

[0063] 3) Complex call relationships may exist between different tasks of the same type. When different framework mechanisms are also coupled in, task tracing will involve not only call context relationships but also the interrelationships between different interface engines. This is especially true when dealing with microservice-based middleware architectures, where a mechanism is needed to clearly and effectively trace the overall task flow. In traditional architectures, the different entry points for tasks from different frameworks make task tracing difficult to implement.

[0064] To address one or more of the aforementioned issues, this exemplary embodiment first provides a task processing method that can run on a server, server cluster, or cloud server, etc. Of course, those skilled in the art can also run the method of this invention on other platforms as needed, and this exemplary embodiment does not specifically limit this. References Figure 2 As shown, the task processing method may include the following steps:

[0065] Step S210. Receive request data sent by the target interface, convert the request data into first data, and use the first data to generate a task to be processed;

[0066] Step S220. Obtain the task type of the task to be processed, and put the task to be processed into the task input queue corresponding to the task type;

[0067] Step S230. Obtain the task processing template. Based on the correspondence between the task types included in the task processing template and the processing model, push the tasks to be processed included in the task input queue to the corresponding processing model for processing to obtain the processing result.

[0068] Step S240. Send the processing result to the task output queue corresponding to the task type of the task to be processed, and push the processing result to the target interface through the task output queue to complete the processing of the task to be processed.

[0069] The above task processing method, on the one hand, receives request data sent by the target interface, converts the request data into first data, and uses the first data to generate a task to be processed; obtains the task type of the task to be processed, and places the task to be processed into a task input queue corresponding to the task type; obtains a task processing template, and according to the correspondence between the task types included in the task processing template and the processing models, pushes the tasks to be processed included in the task input queue to the corresponding processing models for processing to obtain processing results; sends the processing results to the task output queue corresponding to the task type of the task to be processed, and pushes the processing results to the target interface through the task output queue to complete the processing of the task to be processed. By converting received request data into first data, generating tasks to be processed based on the first data, and then processing these tasks, the problem of strong coupling between interface types and core basic database types in existing technologies is solved, improving code reusability. Furthermore, by placing tasks to be processed into corresponding task input queues and pushing processing results through task output queues, each task to be processed has a corresponding task queue to achieve a closed loop of task transmission, solving the problem of difficult task tracking in existing technologies. Moreover, by determining the task processing model corresponding to the task to be processed through a task processing template and completing the processing of the task through the corresponding task processing model, the decoupling between task processing and task transmission is achieved, improving task processing efficiency.

[0070] The following provides a detailed explanation and description of each step involved in the task processing method of the example embodiments of this disclosure.

[0071] First, the application scenarios and inventive objectives of the exemplary embodiments of this disclosure will be explained and described.

[0072] Specifically, the exemplary embodiments disclosed herein can be used in various service platforms, mainly to realize the conversion of arbitrary data structures with different interface formats, reduce the coupling between task transmission and task processing, and realize end-to-end tracing processing of task processing.

[0073] This disclosure uses the received request data from the target interface as a basis, converts the request data into first data, generates tasks to be processed based on the first data, and then processes the tasks to obtain the processing results, thereby improving task processing efficiency. The specific reasons are as follows: Firstly, upon receiving request data from the target interface, to avoid inconsistencies in data structures between different interfaces, the request data can be converted into first data, and tasks to be processed can be generated using the first data, achieving flexible support for multiple interfaces and improving code reusability. Secondly, by using a task output queue corresponding to the type of task to be processed and by passing the task to be processed through the task output queue, full-link tracing of task processing is achieved. Thirdly, by using the correspondence between the task type and the processing model included in the task processing template definition, the tasks to be processed are pushed to the corresponding processing model for processing, improving task processing efficiency.

[0074] Secondly, the task processing system involved in the exemplary embodiments of this disclosure will be explained and described. (Reference) Figure 3As shown, the task processing system may include: a data conversion module 310, a task input queue 320, a task scheduling module 330, a task processing module 340, a task output queue 350, a result forwarding module 360, and a data insertion stream 370. The data conversion module 310 includes a core data structure 311, an interface conversion function 312, and task status information 313. It is used to convert the received request data from the target interface into data corresponding to the core data structure and generate tasks to be processed based on the converted data. The task input queue 320 is used to transmit tasks to be processed and may include multiple task input queues, such as task input queue 1, task input queue 2… task input queue N, where N is a positive integer. The task scheduling module 330 is used to generate task processing templates and route and schedule different tasks to be processed to the corresponding task processing models. The task scheduling module may include multiple task processing templates, such as task processing template 1, task processing template 2… task processing template M, where M is a positive integer. The task processing module 340 is used to determine the task processing model of the task to be processed and to process the task according to the task processing model. The task is processed to obtain the processing result. The task processing module can include multiple task processing models, such as processing model 1, processing model 2... processing model O, where O is a positive integer. The task output queue 350 can include multiple task output queues, such as task output queue 1, task output queue 2... task output queue N, where N is a positive integer. The number of task input queues is equal to the number of task output queues. It is used to transmit the processing result of the task to be processed. The result forwarding module 360 ​​can include multiple forwarders, such as forwarder 1, forwarder 2... forwarder N, where N is a positive integer. The number of forwarders is equal to the number of task output queues. It is used to obtain the processing result of the task to be processed from the task output queue 350 and forward the task processing result to the interface for sending request data. The data insertion stream 370 is connected to the data conversion module 310 via the network. It is used to organize the returned processing result into batch data insertion commands.

[0075] The following will combine Figure 3 Steps S210-S240 will be explained and described.

[0076] In step S210, request data sent by the target interface is received, the request data is converted into first data, and the first data is used to generate a task to be processed.

[0077] In this example embodiment, the first data is the data corresponding to the core data structure 311 included in the data conversion module 310, as referenced. Figure 4As shown, receiving request data sent by the target interface, converting the request data into first data, and generating a task to be processed using the first data may include steps S410-S430:

[0078] Step S410. Determine the interface conversion function corresponding to the target interface according to the data communication method of the target interface;

[0079] Step S420. Convert the request data into first data corresponding to a preset data structure according to the interface conversion function;

[0080] Step S430. Generate a task to be processed using the first data.

[0081] The following will explain and illustrate steps S410-S430. Specifically, firstly, the data communication method of the request data is determined, and a corresponding data conversion function is matched according to the data communication method of the request data. The interface request data can include both request data and response data. Therefore, the core data structure can also include the data structure corresponding to the request and the data structure corresponding to the response. When the interface request data is request data, the data conversion function corresponding to the request data needs to be determined in the interface conversion function 312. This data conversion function can include a request serialization function and a request deserialization function. For example, the request serialization function and the request deserialization function can be Void request_from_json(std::string&&json) and std::string request_to_json(), respectively. When the interface request data is response data, the data conversion function can include a response serialization function and a response deserialization function. For example, the response serialization function and the response deserialization function can be Void response_from_json(std::string&&json) and std::string response_to_json(), respectively. The data conversion module 310 can include multiple sets of data conversion functions, which can bidirectionally convert requests and responses and basic data from various different interfaces. This also makes the conversion between different interfaces easier.

[0082] Once the data conversion function corresponding to the task to be processed is determined, the request data of the interface can be converted into first data consistent with the core data structure through the data conversion function, and the task to be processed can be generated using the first data. The core data structure may include variables of different data types, or it may include methods, which are not specifically limited in this example embodiment. The task to be processed may include a unique task identifier, a task type, or it may include the task processing status and whether an error has occurred, which are not specifically limited in this example embodiment.

[0083] In step S220, the task type of the task to be processed is obtained, and the task to be processed is placed into the task input queue corresponding to the task type.

[0084] To ensure the speed of task processing and the processing capacity under varying pressure, a queue mechanism can be used to reduce the sensitivity of the backend processing module to I / O (Input / Output) pressure. Specifically, the task input queue and task output queue corresponding to the task to be processed can be uniquely represented by the task type.

[0085] In this example embodiment, reference Figure 5 As shown, obtaining the task type of the task to be processed and placing the task to be processed into the task input queue corresponding to the task type may include steps S510-S530:

[0086] Step S510. Based on the task to be processed, create a mutually exclusive resource corresponding to the task to be processed;

[0087] Step S520. Obtain the task type of the task to be processed, and determine the task input queue corresponding to the task type;

[0088] Step S530. Place the task to be processed into the task input queue and wait for the mutex resource.

[0089] The following will explain and illustrate steps S510-S530. Specifically, each task to be processed has a corresponding task input queue and task output queue. When processing a task, firstly, the thread responsible for receiving interface data requests creates a mutex resource within the task structure and waits for the mutex resource after placing the task into the task input queue. The task in the task input queue can be consumed and processed by the processing model. After the task is processed, the result is placed into the task output queue. The result forwarding module 360 ​​pushes the mutex resource included in the task structure to the thread waiting for the processing result. This thread is the one responsible for receiving interface data requests.

[0090] In step S230, a task processing template is obtained. Based on the correspondence between the task types included in the task processing template and the processing models, the tasks to be processed included in the task input queue are pushed to the corresponding processing models for processing to obtain the processing results.

[0091] The task scheduling module 330 may include multiple task processing templates. A task processing template can be represented as Task x Model, where Task represents the task type of the task to be processed, Model represents the processing model, and x is the Cartesian product in mathematics. Task x Model means that a task to be processed can be processed by multiple processing models, and a processing model can also process multiple tasks to be processed.

[0092] In this example embodiment, reference Figure 6 As shown, before obtaining the task processing template, the task processing method further includes steps S610 and S620:

[0093] Step S610. Determine the correspondence between task types and processing models for those task types; wherein the relationship between task types and processing models is many-to-many.

[0094] Step S620. Process the correspondence using the specialization syntax of class templates to generate the task processing template.

[0095] The following will explain and illustrate steps S610 and S620. Specifically, firstly, the processing model for the task type to be processed is determined; secondly, using the specialization syntax of C++ class templates, the task to be processed is converted into a processing function within the namespace that is jointly specialized by the task type of the task to be processed and the processing model for processing that task type. Here, specialization refers to defining some special cases for the elementary template. In this example embodiment, the task type and processing model of the task to be processed can be specialized. To ensure the timeliness of the service, a time constant for timeout requirements can also be defined in the specialized template. In this example embodiment, no special limitations are made on the specialization parameters.

[0096] Furthermore, the processing model may include common initialization components required to complete certain operations. For example, the processing model may include the creation of thread pools / connection pools, the initialization of ZooKeeper connections, and the loading of the processing model itself. This example embodiment does not specifically limit this. Extracting the initialization components allows the allocation and release of common resources to be performed within the same processing model, while also making the task processing logic clearer.

[0097] Furthermore, task processing templates facilitate the migration of tasks between processing models. Specifically, each task processing template has a corresponding independent thread that performs specific tasks, including: retrieving tasks from the task input queue and pushing them to the corresponding processing model based on their task type. For example, if a task is initially processed by processing model A, and its processing model needs to be changed to model B, the declared types in the task processing template can be modified, and the specialized code can be adjusted according to the changed processing model. This allows the task to be converted from model A to model B. Additionally, multiple routes for task models capable of handling the task can be opened within the task processing template, enabling different processing models to handle the task. The task processing template is a template class that can be declared in the main program or dynamically generated through configuration file parsing; this example embodiment does not impose any special limitations on this.

[0098] In this example embodiment, reference Figure 7 As shown, pushing the tasks to be processed in the task input queue to the corresponding processing model for processing to obtain the processing result may include steps S710 and S720:

[0099] Step S710. Read the tasks to be processed from the task input queue, and push the tasks to be processed to the processing model corresponding to the task type according to the task type of the tasks to be processed.

[0100] Step S720. Process the task to be processed using the task processing function included in the processing model to obtain the processing result.

[0101] The following will explain and illustrate steps S710 and S720. Specifically, firstly, a task to be processed is read from the task input queue through an independent thread corresponding to the task processing template, and a task processing template corresponding to the task to be processed is determined according to the task type of the task to be processed; secondly, the processing model of the task to be processed is determined through the task processing template, and the task to be processed is processed through the processing functions contained in the task processing model to obtain the processing result, and the processing result is pushed to the task output queue through the independent thread corresponding to the task processing template.

[0102] For example, refer to Figure 8The diagram illustrates the processing of tasks to be processed, where the tasks include face detection 801, human body detection 802, download 803, transaction 804, and product information 805; the task processing templates include: face x detection model 811, human body x detection model 812, download x local database 813, download x cloud 814, transaction x transaction database 815, and product information x resource management database 816; the processing models include: detection algorithm model 821, local database 822, object storage cloud service (OSS) 823, transaction record database 824, and product resource system 825. The specific processing steps may include: for different tasks such as face detection 801 and human body detection 802, a unified detection algorithm model 821 can be used for processing; for the same task of downloading, different processing models can be used to download from the local database 822 or from the object storage cloud service 823; for transactions 804 that require product information 805, the product information resource management database 816 can be directly called in the task processing template, or a query can be performed directly in the transaction record database. At the same time, product information processing 805 can be directly queried in the product resource system 825.

[0103] In step S240, the processing result is sent to the task output queue corresponding to the task type of the task to be processed, and the processing result is pushed to the target interface through the task output queue to complete the processing of the task to be processed.

[0104] In this example embodiment, the processing result is sent to the task output queue corresponding to the task type of the task to be processed, and the processing result is pushed to the target interface through the task output queue, including:

[0105] Obtain the processing result of the task to be processed, and push the processing result to the target interface through the mutual exclusion resource.

[0106] Specifically, after the independent thread corresponding to the task processing template pushes the processing result of the task to be processed to the task output queue, the result forwarding module 360 ​​can push the mutex resource corresponding to the task to be processed to the thread receiving the interface data request. When the thread receiving the interface data request is awakened, it retrieves the processing result of the task to be processed from the task output queue and returns the processing result to the target interface.

[0107] Further reference Figure 9 As shown, pushing the processing result to the target interface may include steps S910 and S920:

[0108] Step S910. Upon receiving the notification of the mutual exclusion resource, obtain the processing result of the task to be processed from the task output queue;

[0109] Step S920. Convert the processing result into a target processing result corresponding to the target interface through the interface conversion function, and send the target processing result to the target interface.

[0110] The following will explain and describe steps S910 and S920. Specifically, when the thread receiving the interface data request receives the mutex resource pushed by the result forwarding module 360, that is, when the thread receiving the interface data request is awakened, the thread receiving the interface data request obtains the processing result of the task to be processed from the task output queue, and converts the processing result of the task to be processed into a target processing result consistent with the data type supported by the target interface through the interface conversion function 312 included in the data conversion module 310, and pushes the target processing result to the target interface to complete the processing of the task to be processed.

[0111] The task processing method and system provided in this exemplary embodiment have at least the following advantages: Firstly, by converting the received interface request data and processing the request data based on the unified data structure obtained from the conversion, the conversion of data structures between different interfaces is realized, improving code reusability. Secondly, by determining the task input queue and task output queue according to the task type of the task to be processed, and by tracking the processing status of the task to be processed in real time through the unique identifier of the task to be processed in different processing models and logs during the task processing process, a closed loop of task transmission is realized. Thirdly, by determining the processing model of the task to be processed through the task processing template, and by decoupling the processing model from the processing of the task to be processed to the task transmission through an independent thread, the efficiency of task processing is improved.

[0112] The following, combined with Figure 10 The task processing method of the exemplary embodiments of this disclosure will be further explained and described. The task processing method may include the following steps:

[0113] Step S1002. Receive request data from the target interface;

[0114] Step S1004. Convert the received request data into first data according to the data conversion function, and generate a task to be processed based on the first data;

[0115] Step S1006. Create a mutual exclusion resource corresponding to the task to be processed, and push the task to be processed to the corresponding task input queue according to the task type of the task to be processed.

[0116] Step S1008. The thread corresponding to the task processing template retrieves the task to be processed from the task input queue;

[0117] Step S1010. Determine the processing model corresponding to the task to be processed according to the task processing template, and process the task to be processed through the task processing function included in the processing model to obtain the processing result;

[0118] Step S1012. Push the processing result to the task output queue, and wake up the thread receiving interface data requests through the mutex resource corresponding to the task to be processed;

[0119] Step S1014. The thread receiving the interface data request obtains the processing result from the task output queue, transforms the processing result through the data transformation function, and pushes the transformed result to the target interface.

[0120] This disclosure also provides a task processing apparatus, with reference to exemplary embodiments thereof. Figure 11 As shown, the task processing device may include: a task generation module 1110, a task input module 1120, a task processing module 1130, and a processing result sending module 1140. Wherein:

[0121] The task generation module 1110 is used to receive request data sent by the target interface, convert the request data into first data, and generate a task to be processed using the first data.

[0122] The pending task input module 1120 is used to obtain the task type of the pending task and put the pending task into the task input queue corresponding to the task type.

[0123] The pending task processing module 1130 is used to obtain a task processing template, and according to the correspondence between the task types included in the task processing template and the processing model, push the pending tasks included in the task input queue to the corresponding processing model for processing to obtain the processing result.

[0124] The processing result sending module 1140 is used to send the processing result to the task output queue corresponding to the task type of the task to be processed, and push the processing result to the target interface through the task output queue to complete the processing of the task to be processed.

[0125] In one exemplary embodiment of this disclosure, receiving request data sent by a target interface, converting the request data into first data, and generating a task to be processed using the first data includes:

[0126] Based on the data communication method of the target interface, determine the interface conversion function corresponding to the target interface;

[0127] The request data is converted into first data corresponding to a preset data structure according to the interface conversion function;

[0128] The first data is used to generate a task to be processed.

[0129] In one exemplary embodiment of this disclosure, obtaining the task type of the task to be processed and placing the task to be processed into a task input queue corresponding to the task type includes:

[0130] Based on the task to be processed, create a mutually exclusive resource corresponding to the task to be processed;

[0131] Obtain the task type of the task to be processed, and determine the task input queue corresponding to the task type;

[0132] The task to be processed is placed in the task input queue and waits for the mutex resource.

[0133] In one exemplary embodiment of this disclosure, before obtaining the task processing template, the task processing method further includes:

[0134] Determine the correspondence between task types and processing models for those task types; wherein the relationship between task types and processing models is many-to-many.

[0135] The correspondence is processed using the specialization syntax of class templates to generate the task processing template.

[0136] In one exemplary embodiment of this disclosure, the tasks to be processed included in the task input queue are pushed to the corresponding processing model for processing to obtain processing results, including:

[0137] Read the tasks to be processed from the task input queue, and push the tasks to be processed to the processing model corresponding to the task type according to the task type.

[0138] The task to be processed is processed by the task processing function included in the processing model to obtain the processing result.

[0139] In one exemplary embodiment of this disclosure, the processing result is sent to a task output queue corresponding to the task type of the task to be processed, and the processing result is pushed to the target interface through the task output queue, including:

[0140] Obtain the processing result of the task to be processed, and push the processing result to the target interface through the mutual exclusion resource.

[0141] In one exemplary embodiment of this disclosure, pushing the processing result to the target interface through the mutually exclusive resource includes:

[0142] Upon receiving notification of the mutual exclusion resource, the processing result of the task to be processed is obtained from the task output queue.

[0143] The processing result is converted into a target processing result corresponding to the target interface through the interface conversion function, and the target processing result is sent to the target interface.

[0144] The specific details of each module in the above-mentioned task processing device have been described in detail in the corresponding task processing methods, so they will not be repeated here.

[0145] It should be noted that although several modules or units of the device for performing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0146] Furthermore, although the steps of the method in this invention are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0147] In an exemplary embodiment of the present invention, an electronic device capable of implementing the above-described data conversion method is also provided.

[0148] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0149] The following is for reference. Figure 12 To describe an electronic device 1200 according to this embodiment of the present invention. Figure 12 The electronic device 1200 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0150] like Figure 12As shown, the electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 1210, at least one storage unit 1220, a bus 1230 connecting different system components (including storage unit 1220 and processing unit 1210), and a display unit 1240.

[0151] The storage unit stores program code that can be executed by the processing unit 1210, causing the processing unit 1210 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 1210 can perform actions such as... Figure 2 The steps shown are as follows: Step S210: Receive request data sent by the target interface, convert the request data into first data, and generate a task to be processed using the first data; Step S220: Obtain the task type of the task to be processed, and place the task to be processed into the task input queue corresponding to the task type; Step S230: Obtain a task processing template, and according to the correspondence between the task types included in the task processing template and the processing model, push the tasks to be processed included in the task input queue to the corresponding processing model for processing to obtain the processing result; Step S240: Send the processing result to the task output queue corresponding to the task type of the task to be processed, and push the processing result to the target interface through the task output queue to complete the processing of the task to be processed.

[0152] Storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 12201 and / or cache memory 12202, and may further include a read-only memory (ROM) 12203.

[0153] Storage unit 1220 may also include a program / utility 12204 having a set (at least one) of program modules 12205, such program modules 12205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0154] Bus 1230 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0155] Electronic device 1200 can also communicate with one or more external devices 1300 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 1200, and / or with any device that enables electronic device 1200 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1250. Furthermore, electronic device 1200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1260. As shown, network adapter 1260 communicates with other modules of electronic device 1200 via bus 1230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0156] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the embodiments of the present invention.

[0157] In exemplary embodiments of the present invention, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the present invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the present invention described in the "Exemplary Methods" section above.

[0158] According to embodiments of the present invention, a program product for implementing the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0159] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0160] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0161] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0162] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0163] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0164] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention described herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not invented herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

Claims

1. A task processing method, characterized in that, include: The process involves receiving request data sent by a target interface, converting the request data into first data, and generating a task to be processed using the first data. The step of receiving request data sent by the target interface and converting the request data into first data includes: determining an interface conversion function corresponding to the target interface based on its data communication method; and converting the request data into first data corresponding to a preset data structure based on the interface conversion function. Obtain the task type of the task to be processed, and place the task to be processed into the task input queue corresponding to the task type; Determine the correspondence between task types and processing models for those task types; wherein the relationship between task types and processing models is many-to-many; process the correspondence using the specialization syntax of class templates to generate task processing templates; obtain the task processing templates; and based on the correspondence between task types and processing models included in the task processing templates, push the tasks to be processed in the task input queue to the corresponding processing models for processing to obtain processing results. The processing result is sent to the task output queue corresponding to the task type of the task to be processed, and the processing result is pushed to the target interface through the task output queue to complete the processing of the task to be processed.

2. The task processing method according to claim 1, characterized in that, Obtaining the task type of the task to be processed and placing the task to be processed into the task input queue corresponding to the task type includes: Based on the task to be processed, create a mutually exclusive resource corresponding to the task to be processed; Obtain the task type of the task to be processed, and determine the task input queue corresponding to the task type; The task to be processed is placed in the task input queue and waits for the mutex resource.

3. The task processing method according to claim 2, characterized in that, The tasks to be processed in the task input queue are pushed to the corresponding processing model for processing to obtain the processing results, including: Read the tasks to be processed from the task input queue, and push the tasks to be processed to the processing model corresponding to the task type according to the task type. The task to be processed is processed by the task processing function included in the processing model to obtain the processing result.

4. The task processing method according to claim 3, characterized in that, Sending the processing result to the task output queue corresponding to the task type of the task to be processed, and pushing the processing result to the target interface through the task output queue, including: Obtain the processing result of the task to be processed, and push the processing result to the target interface through the mutual exclusion resource.

5. The task processing method according to claim 4, characterized in that, Pushing the processing result to the target interface through the mutually exclusive resource includes: Upon receiving notification of the mutual exclusion resource, the processing result of the task to be processed is obtained from the task output queue. The processing result is converted into a target processing result corresponding to the target interface through the interface conversion function, and the target processing result is sent to the target interface.

6. A task processing device, characterized in that, include: A task generation module is used to receive request data sent by a target interface, convert the request data into first data, and generate a task to be processed using the first data; wherein, receiving the request data sent by the target interface and converting the request data into the first data includes: determining an interface conversion function corresponding to the target interface according to the data communication method of the target interface; and converting the request data into first data corresponding to a preset data structure according to the interface conversion function. The pending task input module is used to obtain the task type of the pending task and put the pending task into the task input queue corresponding to the task type. The pending task processing module is used to determine the correspondence between task types and processing models for processing those task types; wherein the relationship between the task types and the processing models is many-to-many; the correspondence is processed using the specialization syntax of class templates to generate a task processing template; the task processing template is obtained, and according to the correspondence between the task types and processing models included in the task processing template, the pending tasks included in the task input queue are pushed to the corresponding processing models for processing to obtain the processing results; The processing result sending module is used to send the processing result to the task output queue corresponding to the task type of the task to be processed, and push the processing result to the target interface through the task output queue to complete the processing of the task to be processed.

7. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the task processing method according to any one of claims 1-5.

8. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the task processing method of any one of claims 1-5 by executing the executable instructions.

Images