Star-connected multi-model scheduling system and data identification processing method thereof

Abstract

The invention discloses a star-connected multi-model scheduling system and a data identification processing method thereof, and belongs to the technical field of artificial intelligence. A star-connected multi-model scheduling system comprises a model node cluster, a scheduling center and a communication queue. The model node cluster comprises a plurality of model nodes with different functions, and one or more identical neural network models are deployed in the model nodes; the dispatching center comprises a dispatching module and a condition judgment module; and the communication queue comprises a message queue for realizing communication between the client and the dispatching center and a message queue for connecting the dispatching center and each model node. According to the method, a data processing flow for a specific task is split into a plurality of different model nodes, and transmission through all message queues is not needed in data processing, so that efficient utilization of computing resources is realized. Data distribution is achieved through the dispatching center, the overall structure is more flexible, message queue resources are saved, and the data transmission efficiency between model nodes is effectively improved.

Description

Technical Field

[0001] The present invention discloses a star-connected multi-model scheduling system and a data identification and processing method thereof, belonging to the field of artificial intelligence technology. Background Art

[0002] Neural network models are widely used in natural language processing, computer vision and other fields. On the one hand, as the scale of neural network model parameters increases, the model becomes more and more powerful and can handle more complex tasks, but it also means higher computing power and memory requirements. If all tasks are processed using large-scale models without considering the complexity of the task, it will lead to a waste of computing resources. Therefore, by setting up multiple models of different scales and scheduling appropriate scale models to provide services for tasks of different complexities, computing resources can be effectively saved and real-time requirements can be met in scenarios with high access volume. On the other hand, multimodal capabilities have become an important direction for model development, which requires models to be able to simultaneously process and understand data from different modalities, such as text, images, audio, etc. Despite this, existing models are still limited in their multimodal processing capabilities, especially when facing complex and changeable actual business scenarios. In order to meet diverse needs, it is often necessary to combine multiple models, schedule different models for processing according to different needs, and achieve more comprehensive functional coverage.

[0003] For the model scheduling problem, the existing technical solutions mainly focus on scheduling models with different functions for different scenarios. The existing patent CN118035677A (application publication date: May 14, 2024) discloses a large language model scheduling method and system based on a dynamic balance strategy. First, the user input is classified into intent tasks, and the intent is divided into one of the categories of ordinary chat, real-life query, knowledge base question and answer reasoning, code writing or long text summary. The classification result is used as the basis for the required model function, and then the candidate language model with the required function is extracted, the model dynamic vector is constructed, and then the distance between the user request vector and the candidate model dynamic vector is calculated, and the model with the closest distance is used as the target model for scheduling. In addition, the existing patent CN117742970A (application publication date: March 22, 2024) discloses a task scheduling method, apparatus, computer equipment, storage medium and program product, which aims to identify the user's request data through a multi-task learning model, analyze the semantic information and attribute information therein, and use the semantic recognition branch and the task classification branch to classify it into one of multiple preset semantic categories and task categories, and then determine the task label based on the two types of information, and then select the target model. In the above-mentioned model scheduling technical solution, the main strategy is to schedule models with corresponding functions according to different scenarios. However, when faced with specific tasks and functional requirements, the complexity of data processing requests is not fully considered. If the complexity of the task is not used as the basis for model scheduling, and the highest-scale model is used directly, it will not only lead to a waste of computing resources, but also make it more difficult to cope with high concurrency requirements.

[0004] For specific task scenarios, multiple different models are configured according to the required model functional conditions to cope with data processing requirements of different complexity. A commonly used strategy is the pipeline scheduling algorithm, such as Figure 1 As shown, these models of different sizes are scheduled in order; specifically, assuming that there are N models, these models are connected in series. During the data processing process, the data will pass through models M1, M2, M3...M in turn. N After each model is processed, a conditional judgment is performed to confirm whether the processing conditions are met. If the conditions are met, the processing results are returned immediately. If not, the data is passed to the next model. However, the pipeline scheduling algorithm still has disadvantages: since there are only message queues between adjacent models, when there is additional data processing logic between non-adjacent models, the data must be transmitted through all message queues between them, which leads to relatively low data transmission efficiency between different models.

[0005] Therefore, how to schedule different models according to actual needs to achieve optimal configuration and conservation of computing resources while optimizing the processing flow has become an issue that urgently needs in-depth research. Summary of the invention

[0006] In view of the shortcomings of the existing technology, different models are scheduled according to actual needs. When processing data between adjacent models or non-adjacent models, the data processing flow for specific tasks is broken down and distributed through the message center to adapt to data processing needs with different needs and different complexities, optimize the processing flow, and achieve optimal configuration and conservation of computing resources. It efficiently utilizes computing resources and improves the efficiency of data transmission between different models. A star-connected multi-model scheduling system and its data identification and processing method are proposed.

[0007] A star-connected multi-model scheduling system, comprising a model node cluster, a scheduling center and a communication queue;

[0008] The model node cluster includes a plurality of model nodes with different functions, and one or more identical neural network models are deployed in the model nodes;

[0009] The model node cluster is used to identify and process the data sent by the dispatch center;

[0010] The scheduling center is used to receive data processing requests sent by the client from the message queue and distribute the data to the model node cluster;

[0011] The scheduling center includes a scheduling module and a condition judgment module; wherein the scheduling module is used to schedule the model nodes; the condition judgment module is used to evaluate the recognition results returned by the model nodes to determine whether they meet the preset data recognition requirements;

[0012] The communication queue includes a message queue for realizing communication between the client and the dispatch center and a message queue for connecting the dispatch center and each model node;

[0013] The message queue used to realize communication between the client and the dispatch center is used to receive data requests sent by the client and return processing results;

[0014] The message queue connecting the dispatch center and each model node is used for the dispatch center to send the data to be identified to the target model node, and the model node to return the identification result to the dispatch center. No message queue is set between different model nodes.

[0015] The model node cluster includes model nodes for different tasks, model nodes for the same task but used in different processing stages, and model nodes for the same task but with different model function complexity.

[0016] The neural network model includes a computer vision model and a natural language model.

[0017] The data types that can be processed by the neural network model include text data, image data and audio data.

[0018] The multiple neural network models within the single model node receive and process data from the associated message queue in a preemptive manner.

[0019] The numbering basis of the model nodes and the message queues is: the numbering basis of model nodes for the same task but used in different processing stages is: the order of the single neural network model in the model node in the corresponding task processing flow is used as the sorting basis; the numbering basis of model nodes for the same task but different model function complexity is: the functional complexity of the single neural network model in the model node is used as the sorting basis.

[0020] The model nodes and the message queues are numbered. After the numbering process, the model nodes are numbered as M1, M2, M3, ... N Correspondingly, the message queues between the model node and the dispatch center are also numbered as Q1, Q2, Q3...Q N ; The message queue number for communication between the client and the server scheduling center is represented by Q0.

[0021] A data identification and processing method for a star-connected multi-model scheduling system specifically comprises the following steps:

[0022] S1, the scheduling center receives the data processing request from the client in the message queue Q0, starts the scheduling process, and takes the first model node for this task as the target model node;

[0023] S2, the dispatch center sends the data to be processed to the target model node through the dedicated message queue between the target model node and the dispatch center;

[0024] S3, the target model node receives the data to be processed and performs recognition processing. After completion, the processing result is sent back to the dispatch center through the message queue between the dispatch center and the dispatch center;

[0025] S4, the condition judgment module in the dispatch center evaluates the recognition result to determine whether it meets the predetermined data recognition processing requirements. If the result meets the requirements, the result is returned to the client through the message queue Q0 to complete the data recognition processing flow; if the result does not meet the requirements, the dispatch module will sequentially select the next model node for this task as the new target model node, and repeat the steps of S2 data sending, S3 recognition processing, and S4 result evaluation until the processing result meets the requirements or has been dispatched to the last model node;

[0026] If there is a data processing sequence relationship between two model nodes, the communication between the two model nodes is achieved through a message queue.

[0027] The beneficial effects of the present invention are as follows:

[0028] 1. The present invention splits the data processing flow for a specific task into multiple different model nodes, or deploys model nodes of different complexities. In view of the differences in data processing difficulty or the data processing logic between non-adjacent model nodes, there is no need to transmit through all message queues. Instead, one model node is scheduled each time to identify and process the data, and then it is transmitted through its corresponding message queue, achieving the expected identification target at the minimum cost, thereby realizing efficient utilization of computing resources.

[0029] 2. Compared with the conventional pipeline scheduling method, the present invention realizes data distribution through the scheduling center, and the overall structure is more flexible, which not only saves message queue resources, but also effectively improves the data transmission efficiency between model nodes. This makes the processing flow of the multi-model scheduling system more efficient and flexible, and can better adapt to data processing requirements of different complexities. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flowchart of the pipeline scheduling algorithm in the background technology.

[0031] Figure 2 The present invention is a schematic flow chart of a star-connected multi-model scheduling system, and the chart is used as a summary figure.

[0032] Figure 3 The present invention is a flowchart of a data identification and processing method for a star-connected multi-model scheduling system. DETAILED DESCRIPTION

[0033] The present invention will be further described below in conjunction with the accompanying drawings.

[0034] A star-connected multi-model scheduling system, such as Figure 2 As shown, it includes a model node cluster, a scheduling center, and a communication queue.

[0035] The model node cluster includes multiple model nodes with different functions; one or more identical neural network models are deployed in the model node; the model node cluster is used to identify and process the data sent by the dispatch center.

[0036] The model node cluster includes model nodes for different tasks, model nodes for the same task but used in different processing stages, and model nodes for the same task but with different model function complexity.

[0037] Neural network models include, but are not limited to, computer vision neural network models and natural language neural network models. The data types that can be processed using neural network models include text data, image data, and audio data.

[0038] One or more identical neural network models are deployed in the model node to meet the real-time requirements of high-concurrency scenarios. Multiple neural network models in a single model node will receive and process data from the associated message queue in a preemptive manner.

[0039] According to actual work needs, model nodes and message queues are numbered based on the following: for model nodes with the same task but used in different processing stages: the order of individual neural network models in the model node in the task processing flow; for model nodes with the same task but different model function complexity: the functional complexity of individual neural network models in the model node.

[0040] After the model nodes and message queues are numbered, the model nodes are numbered as M1, M2, M3...M N ; Correspondingly, the message queues between the model nodes and the dispatch center are also numbered as Q1, Q2, Q3...Q N ; The message queue for communication between the client and the scheduling center is represented by Q0.

[0041] Specifically, M1 is a model node composed of a simple OCR model, whose function is to recognize text in an image. It is suitable for images with good shooting conditions, horizontal placement of certificates, and no text interference in the background. M2 is a model node composed of a target detection model, whose function is to locate the position of certificates in the image. It is suitable for images with interference in the background of certificates, such as placing certificates on other documents during shooting, resulting in other text in the background of certificates in the image. M3 is a model node composed of a complex OCR model, whose functions include text detection and text recognition, supporting text position positioning and text content recognition, and is suitable for images with complex text content on the certificate surface. M4 is a model node composed of an image super-resolution model, whose function is to improve the resolution of the image and make the image clearer. It is used for unclear images caused by shaking or too far away during shooting. M5 is a model node composed of image key information extraction models. It can identify and extract key information from images and provide semantic entity recognition and relationship extraction functions. Semantic entity recognition can realize text recognition and classification in images. Relationship extraction is used to analyze and extract text relationships and identify key-value pairs of key fields. It is suitable for certificate images with complex information and high density. M6 is a model node composed of multimodal large models, supporting image, text and other data processing. It can obtain the expected image analysis results through prompts, and is suitable for certificate images with complex information and complex recognition requirements.

[0042] The scheduling center is used to receive data processing requests sent by the client from the message queue and distribute data to the model node cluster; the scheduling center includes a scheduling module and a condition judgment module; the scheduling module is used to schedule the model nodes; the condition judgment module is used to evaluate the recognition results returned by the model nodes to determine whether they meet the preset data recognition requirements.

[0043] Specifically, the data request body sent by the client includes the certificate image to be recognized and processed and a routing key, wherein the routing key specifies the model node for the recognition of this type of image.

[0044] Communication queues include message queues used to implement communication between the client and the dispatch center and message queues connecting the dispatch center and various model nodes.

[0045] The message queue used to realize the communication between the client and the dispatch center is used to receive the data request sent by the client and return the processing results. The message queue connecting the dispatch center and each model node is used for the dispatch center to send the data to be recognized to the corresponding model node, and the model node to return the recognition result to the dispatch center. No message queue is set between different model nodes.

[0046] Specifically, the communication queue between the client and the scheduling center consists of a switch and a message queue bound to the switch; the communication queue from the scheduling center to the model node consists of a switch and a message queue bound to the switch, and the communication queue from the model node to the scheduling center consists of k switches and message queues bound to the switches, where k is the number of models deployed in the model node.

[0047] A data recognition and processing method for a star-connected multi-model scheduling system, such as Figure 3 As shown, the specific steps include:

[0048] S1, the scheduling center receives the data processing request from the client in the message queue Q0, starts the scheduling process, and takes the first model node for this task as the target model node;

[0049] S2, the dispatch center sends the data to be processed to the target model node through the dedicated message queue between the target model node and the dispatch center;

[0050] S3, the target model node receives the data to be processed and performs recognition processing. After completion, the processing result is sent back to the dispatch center through the message queue between the dispatch center and the dispatch center;

[0051] S4, the condition judgment module in the dispatch center evaluates the recognition result to determine whether it meets the predetermined data recognition processing requirements. If the result meets the requirements, the result is returned to the client through the message queue Q0 to complete the data recognition processing flow; if the result does not meet the requirements, the dispatch module will sequentially select the next model node for this task as the new target model node, and repeat the steps of S2 data sending, S3 recognition processing, and S4 result evaluation until the processing result meets the requirements or has been dispatched to the last model node.

[0052] If there is a data processing sequence relationship between two model nodes, the communication between the two model nodes is realized through a message queue.

[0053] The process of the online certificate registration business scenario is that the user uploads the certificate image to the server, and the certificate recognition system automatically recognizes and analyzes the uploaded image, automatically extracts the key information required for various certificate registration forms, and then realizes the automatic registration function based on the certificate image. At the same time, certificate verification is also an important link, such as checking the authenticity and compliance of seals, signatures and portraits to ensure that the certificates uploaded by users meet the requirements.

[0054] The information density of different certificates varies significantly. For example, the national emblem side of the ID card only contains two fields: the issuing authority and the validity period, while the personal page of the household registration booklet includes more than 20 detailed fields such as name, household head or relationship with the household head, former name, birthplace, native place, citizen ID number, etc. In addition, some certificates are also attached with additional elements such as seals or portraits. For certificates with simple layout and low information density, a lightweight model can meet the recognition requirements; for certificate images with complex layout and high information density, a more complex model is required to ensure the recognition effect.

[0055] Embodiment 1: Taking the identification card image recognition as an example, since the information density of the identification card is relatively low, a lightweight functional model is used to provide identification processing.

[0056] S1. The dispatch center receives the ID card image processing request to be processed from the message queue Q0, starts the dispatch process, and takes the first model node for the ID card image recognition processing task as the target node.

[0057] Specifically, the routing key in the data processing request specifies that the corresponding model is the first-stage model node M1 for ID card image recognition processing.

[0058] S2. The dispatch center sends the data to be processed to the target model node through a dedicated message queue between the target model node and the dispatch center.

[0059] Specifically, the dispatch center sends the ID card image to the model node M1 through the message queue Q1.

[0060] S3. The target model node receives the data to be processed and performs recognition processing. After completion, the processing result is sent back to the dispatch center through the message queue between the dispatch center and the dispatch center.

[0061] Specifically, the simple OCR model in the model node M1 receives the ID card image and performs OCR recognition to identify all the text in the image. In addition to the image recognition result, the processing result also includes a routing key, which is used to specify that the next target model is the second-stage model M2 for ID card image recognition processing. The processing result is sent back to the dispatch center through the message queue Q1.

[0062] S4, the condition judgment module in the dispatch center evaluates the recognition result to determine whether it meets the predetermined data recognition processing requirements. If the result meets the requirements, the result is returned to the client through the message queue Q0 to complete the data recognition processing flow; if the result does not meet the requirements, the dispatch module will sequentially select the next model node for this task as the new target model node, and repeat the steps of S2 data sending, S3 recognition processing, and S4 result evaluation until the processing result meets the requirements or has been dispatched to the last model node.

[0063] Specifically, if the ID card image is placed horizontally and has no background interference when photographed, the recognition result of the model node M1 can meet the recognition requirements. The required information can be extracted from the OCR result through regular expressions or key field segmentation, and then the recognition result is returned through the message queue Q0 to complete the ID card image recognition process;

[0064] Specifically, if the image acquisition conditions are not ideal, such as the presence of other text interference in the background, it is necessary to combine other models such as license detection for further recognition processing. The scheduling module uses the model specified by the routing key in the message return body as the target model node;

[0065] Specifically, the ID card image is sent to the model node M2 ​​through the message queue Q2; after receiving the image data, the model node M2 ​​performs ID card target detection, obtains the bounding box information of the ID card in the image, and cuts the ID card area image from the original image according to the bounding box information, uses the routing key to specify the next target node as M1, and sends the result to the target model node M1 through the message queues Q2 and Q1 for re-recognition, and the recognized result is sent back to the dispatch center again;

[0066] Specifically, the condition judgment module in the dispatch center evaluates the recognition result. If the result meets the requirements, the result is returned to the client. If it still does not meet the recognition requirements, the dispatch module will dispatch the subsequent M3 and M4 model nodes in sequence, repeating the above data sending, recognition processing, and result evaluation process until the processing result meets the requirements or has been dispatched to the last model node;

[0067] If there is a data processing sequence relationship between two model nodes, the communication between the two model nodes is achieved through a message queue.

[0068] Specifically, after the model node M2 ​​performs ID card target detection, it sends the target detection result to the model node M1 through the message queues Q2 and Q1 to perform OCR recognition.

[0069] Example 2: Taking the image recognition of the personal page of the household registration book as an example, since it involves forms and seals, the information density is high and the layout is complex, a complex functional model is required to provide recognition processing.

[0070] S1. The dispatch center receives the pending household page image processing request from the message queue Q0 and starts the dispatch process. The target model node is selected according to the routing key in the data processing request.

[0071] Specifically, the target model specified by the routing key in the data processing request is the first-stage model M2 for user image recognition processing.

[0072] S2. The dispatch center sends the data to be processed to the target model node through a dedicated message queue between the target model node and the dispatch center.

[0073] Specifically, the dispatch center sends the household registration image to the target model node M2 ​​through the message queue Q2.

[0074] S3. The target model node receives the data to be processed and performs recognition processing. After completion, the processing result is sent back to the dispatch center through the message queue between the dispatch center and the dispatch center.

[0075] Specifically, after receiving the image data, the model node M2 ​​performs household registration page target detection, obtains the bounding box information of the household registration page in the image, and cuts the household registration area image from the original image according to the bounding box information, and uses the routing key to specify the next stage model node as M5, and sends the result to the target model node through the message queues Q2 and Q5 for recognition; after receiving the household registration area image, the model node M5 performs entity relationship extraction processing, identifies the key-value pairs of the key fields in the household registration table, and constructs a message return body consisting of the recognition result and the routing key. The routing key specifies the next stage model M6 for household registration image recognition processing. The message return body is sent back to the dispatch center through the message queue Q5.

[0076] S4, the conditional judgment module of the dispatch center evaluates the recognition result of the model node M5 to determine whether it meets the predetermined image recognition requirements. If it meets the requirements, that is, all the required key field information is extracted, the recognition result is returned through the message queue Q0 to complete the household registration page image recognition process; if it does not meet the requirements, that is, some key field information is missing, or further analysis of the elements of the household registration page is required, such as verification through seals, etc., the model node M6 specified by the routing key is used as the target model node;

[0077] The dispatch center sends the household registration image to the target model node M6 through the message queue Q6;

[0078] According to the requirements of household registration page image recognition and processing, prompt words are constructed and sent together with the household registration page image to the multimodal large model node M6 for processing. The constructed prompt words are such as "Please identify and analyze the household registration page image provided, extract and return the following information from it: name, household head or relationship with household head, gender, birthplace, ethnicity, native place, citizen ID number, whether stamped. Ensure that the extracted information is accurate, exclude other interference elements in the image, and focus on the recognition and extraction of the above key information", and finally send the recognition result back to the dispatch center through the message queue Q6, and the dispatch center sends the recognition processing result back to the client through the message queue Q0;

[0079] If there is a data processing sequence relationship between two model nodes, the communication between the two model nodes is achieved through a message queue.

[0080] Specifically, after the model node M2 ​​performs household registration target detection, the target detection result is sent to the model node M5 through the message queues Q2 and Q5 to perform entity relationship extraction processing.

Claims

1. A star-connected multi-model scheduling system, characterized in that: Includes model node cluster, dispatch center and communication queue; The model node cluster includes a plurality of model nodes with different functions, and one or more identical neural network models are deployed in the model nodes; The model node cluster is used to identify and process the data sent by the dispatch center; The scheduling center is used to receive data processing requests sent by the client from the message queue and distribute the data to the model node cluster; The scheduling center includes a scheduling module and a condition judgment module; wherein the scheduling module is used to schedule the model nodes; the condition judgment module is used to evaluate the recognition results returned by the model nodes to determine whether they meet the preset data recognition requirements; The communication queue includes a message queue for realizing communication between the client and the dispatch center and a message queue for connecting the dispatch center and each model node; The message queue used to realize communication between the client and the dispatch center is used to receive data requests sent by the client and return processing results; The message queue connecting the dispatch center and each model node is used for the dispatch center to send the data to be identified to the target model node, and the model node to return the identification result to the dispatch center. No message queue is set between different model nodes.

2. A star-connected multi-model scheduling system according to claim 1, characterized in that: The model node cluster includes model nodes for different tasks, model nodes for the same task but used in different processing stages, and model nodes for the same task but with different model function complexities.

3. A star-connected multi-model scheduling system according to claim 1, characterized in that: The neural network model includes a computer vision model and a natural language model.

4. A star-connected multi-model scheduling system according to claim 3, characterized in that: The data types that can be processed by the neural network model include text data, image data and audio data.

5. A star-connected multi-model scheduling system according to claim 1, characterized in that: The multiple neural network models within the single model node receive and process data from the associated message queue in a preemptive manner.

6. A star-connected multi-model scheduling system according to claim 1, characterized in that: The numbering basis of the model nodes and the message queues is: the numbering basis of model nodes for the same task but used in different processing stages is: the order of the single neural network model in the model node in the corresponding task processing flow is used as the sorting basis; the numbering basis of model nodes for the same task but different model function complexity is: the functional complexity of the single neural network model in the model node is used as the sorting basis.

7. A star-connected multi-model scheduling system according to claim 6, characterized in that: The model nodes and the message queues are numbered. After the numbering process, the model nodes are numbered as M1, M2, M3, ... N Correspondingly, the message queues between the model node and the dispatch center are also numbered as Q1, Q2, Q3...Q N ; The message queue number for communication between the client and the server scheduling center is represented by Q0.

8. A data identification and processing method for a star-connected multi-model scheduling system, characterized in that: The specific steps include: S1, the scheduling center receives the data processing request from the client in the message queue Q0, starts the scheduling process, and takes the first model node for this task as the target model node; S2, the dispatch center sends the data to be processed to the target model node through the dedicated message queue between the target model node and the dispatch center; S3, the target model node receives the data to be processed and performs recognition processing. After completion, the processing result is sent back to the dispatch center through the message queue between the dispatch center and the dispatch center; S4, the condition judgment module in the dispatch center evaluates the recognition result to determine whether it meets the predetermined data recognition processing requirements. If the result meets the requirements, the result is returned to the client through the message queue Q0 to complete the data recognition processing process; If the result does not meet the requirements, the scheduling module will sequentially select the next model node for this task as the new target model node, and repeat the steps of S2 data sending, S3 recognition processing, and S4 result evaluation until the processing result meets the requirements or has been scheduled to the last model node; If there is a data processing sequence relationship between two model nodes, the communication between the two model nodes is achieved through a message queue.

Images