Task processing method and device and electronic equipment

Abstract

The invention provides a task processing method and device and electronic equipment, and can be applied to the technical field of natural language processing and the technical field of computers. The task processing method comprises the steps of determining stage features of a text task in response to a received processing request of the text task; performing stage division on the text task according to the stage features to obtain a plurality of sub-tasks; under the condition that the plurality of sub-tasks comprise a first target sub-task, executing the first target sub-task by utilizing a first text processing model deployed in the first equipment; and under the condition that the plurality of sub-tasks comprise a second target sub-task, executing the second target sub-task by utilizing at least one of a second text processing model deployed in the first equipment and a second text processing model deployed in the second equipment according to available resources of the second equipment, the first text processing model and the second text processing model are models of the same type and are obtained through training based on different task training data.

Description

Technical Field

[0001] The present disclosure relates to the fields of natural language processing technology and computer technology, and more specifically to a task processing method, device and electronic device. Background Art

[0002] With the rapid development of artificial intelligence, a large number of applications and systems use large models to provide services to users, but large models have high hardware resource requirements. For example, in intelligent question-answering applications, recommendation systems, or retrieval-augmented generation (RAG) systems, the information input by users may go through text preprocessing, semantic analysis, language translation, image retrieval, and other processing stages, which consume a lot of computing resources and have low processing efficiency for text tasks. Summary of the invention

[0003] In view of the above problems, the present disclosure provides a task processing method, device and electronic device.

[0004] According to a first aspect of the present disclosure, a task processing method is provided, including: in response to receiving a processing request for a text task, determining a stage feature of a text task; dividing the text task into stages according to the stage feature to obtain a plurality of subtasks; when the plurality of subtasks include a first target subtask, executing the first target subtask using a first text processing model deployed in a first device; when the plurality of subtasks include a second target subtask, executing the second target subtask using at least one of a second text processing model deployed in the first device and a second text processing model deployed in the second device according to available resources of the second device, wherein the first text processing model and the second text processing model are of the same type of models and are trained based on different task training data.

[0005] A second aspect of the present disclosure provides a task processing device, including: a determination module, for determining a stage feature of a text task in response to receiving a processing request for a text task; a stage division module, for performing stage division on the text task according to the stage feature to obtain a plurality of subtasks; a first execution module, for executing the first target subtask using a first text processing model deployed in a first device when the plurality of subtasks include a first target subtask; and a second execution module, for executing the second target subtask using at least one of a second text processing model deployed in the first device and a second text processing model deployed in the second device according to available resources of the second device when the plurality of subtasks include a second target subtask, wherein the first text processing model and the second text processing model are of the same type of model and are trained based on different task training data.

[0006] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0007] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the above computer program or instructions are executed by a processor.

[0008] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor.

[0009] According to an embodiment of the present disclosure, in response to receiving a processing request for a text task, the stage characteristics of the text task are determined; the text task is divided into stages according to the stage characteristics to obtain at least one subtask, and a complex text processing process can be simplified into multiple subtasks. In the case where multiple subtasks include a first target subtask, the first target subtask is executed using the first text processing model deployed in the first device. In the case where multiple subtasks include a second target subtask, the second target subtask is executed using at least one of the second text processing model deployed in the first device and the second text processing model deployed in the second device according to the available resources of the second device, and the first text processing model and the second text processing model are the same type of models and are trained based on different task training data. Therefore, different subtasks are processed respectively using the text processing models deployed in the first device and the second device, so that the computing resources consumed by each device are reduced, and the processing efficiency of the text task is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0011] Figure 1 The application scenario diagram of the task processing method according to the embodiment of the present disclosure is schematically shown;

[0012] Figure 2 A flowchart schematically shows a task processing method according to an embodiment of the present disclosure;

[0013] Figure 3 A schematic diagram schematically shows text processing in an intelligent question-answering system according to an embodiment of the present disclosure;

[0014] Figure 4A The following schematically shows a processing diagram of a text task for an image according to an embodiment of the present disclosure;

[0015] Figure 4B The structural block diagram of the text processing model according to the embodiment of the present disclosure is schematically shown;

[0016] Figure 5 A flowchart of a text processing method according to another embodiment of the present disclosure is schematically shown;

[0017] Figure 6 A structural block diagram of a task processing device according to an embodiment of the present disclosure is schematically shown; and

[0018] Figure 7 A block diagram of an electronic device suitable for implementing a task processing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0019] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0020] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0021] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0022] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0023] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0024] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0025] With the rapid development of artificial intelligence, a large number of applications and systems use large models to provide services to users. In intelligent question-answering applications, recommendation systems, and retrieval-augmented generation (RAG) systems, large models require high hardware resources for deployment. For example, user input information may go through semantic analysis, semantic search, language translation, text classification, image retrieval, and other processing processes, which consumes a lot of computing resources and reduces the processing efficiency of text tasks.

[0026] In view of this, an embodiment of the present disclosure provides a task processing method including: in response to receiving a processing request for a text task, determining the stage characteristics of the text task; dividing the text task into stages according to the stage characteristics to obtain multiple subtasks; when the multiple subtasks include a first target subtask, using a first text processing model deployed in a first device to execute the first target subtask; when the multiple subtasks include a second target subtask, based on the available resources of the second device, using at least one of a second text processing model deployed in the first device and a second text processing model deployed in the second device to execute the second target subtask, the first text processing model and the second text processing model are of the same type of models and are trained based on different task training data.

[0027] Figure 1 The application scenario diagram of the task processing method according to an embodiment of the present disclosure is schematically shown.

[0028] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0029] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0030] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0031] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0032] It should be noted that the task processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the task processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The task processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the task processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0033] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0034] Figure 2 The flowchart of the task processing method according to the embodiment of the present disclosure is schematically shown.

[0035] like Figure 2 As shown, the task processing method of this embodiment includes operations S210 to S240.

[0036] In operation S210 , in response to receiving a processing request for a text task, a phase feature of the text task is determined.

[0037] According to an embodiment of the present disclosure, a processing request for a text task may be generated based on information input by a user into an interactive interface, and the interactive interface may be an application or a system.

[0038] For example, the information that a user can input includes at least one of the following: text, image, and file. The file can be a document, a scanned file, a table, etc. The information that a user inputs to a language translation application can be "Express gratitude in English and appear sincere." The text information that a user inputs to a search and recommendation system can be "Please recommend a more affordable home appliance based on the information in the attachment."

[0039] According to an embodiment of the present disclosure, the stage feature of a text task represents the processing feature of the text task. The stage feature may be determined based on information input by a user in an interactive interface.

[0040] For example, a text task may be text preprocessing, and the processing stages of text preprocessing may include text cleaning and text normalization.

[0041] For example, the text task may be text classification, and the processing stages of text classification may include at least one of the following: text preprocessing, text conversion into vectors, and text classification using vectors.

[0042] For example, the text task may be semantic search. The processing stage of semantic retrieval may include at least one of the following: word embedding, sentence embedding, and semantic reasoning.

[0043] For example, the text task may be image retrieval. The processing stages of image retrieval may include at least one of the following: text recognition in the image, text segmentation, text conversion into vectors, and similarity matching with text in a database.

[0044] In operation S220, the text task is divided into stages according to the stage characteristics to obtain a plurality of subtasks.

[0045] According to an embodiment of the present disclosure, multiple subtasks may be associated with each other. For example, subtask A and subtask B may be processed in series, and subtask A and subtask C may be processed in parallel.

[0046] For example, in semantic retrieval, word vectors and sentence vectors can be processed in parallel, and word vectors and sentence vectors are processed in series with semantic reasoning. Word vectors can be obtained by training the context of words, and sentence vectors can capture the context of the entire sentence. Semantic reasoning is performed on the information input by the user in the interactive interface based on word vectors and sentence vectors.

[0047] In operation S230, when the plurality of subtasks include a first target subtask, the first target subtask is performed using a first text processing model deployed in the first device.

[0048] According to an embodiment of the present disclosure, the first device and the second device are different processors. For example, the first device may be a CPU (Central Processing Unit), and the second device may be a GPU (Graphics Processing Unit). The first device also needs to run a system or application and control the basic operation of the computer.

[0049] For example, the first target subtask may be a subtask that consumes less computing resources than a preset subtask resource threshold. It should be noted that the preset subtask resource threshold may be determined based on the available resources of the second device. The more available resources the second device has, the smaller the preset subtask resource threshold may be, so that the available resources of the first device are more sufficient, making the operation of the system or application lighter and more responsive.

[0050] When the computing resources consumed by the text task are less than the preset subtask resource threshold, it means that the computing resources consumed by multiple subtasks of the text task are all less than the set subtask resource threshold. The multiple subtasks of the text task are determined as the first target subtasks, and the text task can be executed using the text processing model deployed in the first device.

[0051] For example, the first target subtask may be a subtask that is simple to process and takes a short time. The first target subtask may be text denoising, text standardization, and the like.

[0052] For example, the first target subtask may be a subtask with a smaller processing amount in serial processing.

[0053] According to an embodiment of the present disclosure, the text processing model may be deployed on the first device and the second device respectively according to a preset deployment plan. The first device and the second device may respectively deploy multiple text processing models to improve text processing efficiency.

[0054] According to an embodiment of the present disclosure, the text processing model may be a neural network model based on natural language processing, a large language model, a RAG (Retrieval-Augmented Generation) model, or the like.

[0055] In operation S240, when the plurality of subtasks include a second target subtask, the second target subtask is executed using at least one of a second text processing model deployed in the first device and a second text processing model deployed in the second device according to available resources of the second device.

[0056] According to an embodiment of the present disclosure, the first text processing model and the second text processing model are the same type of models and are trained based on different task training data. For example, the first text processing model can be trained based on the training data of the first target subtask. The second text processing model is trained using the training data of the second target subtask.

[0057] For example, the second target subtask may be a subtask that consumes computing resources greater than a preset subtask resource threshold.

[0058] For example, the first target subtask may be a subtask that is complex and time-consuming to process. The first target subtask may be text recognition in an image, semantic reasoning, and the like.

[0059] For example, the first target subtask may be a subtask with a larger processing volume in serial processing.

[0060] According to an embodiment of the present disclosure, in response to receiving a processing request for a text task, the stage characteristics of the text task are determined; the text task is divided into stages according to the stage characteristics to obtain at least one subtask, and a complex text processing process can be simplified into multiple subtasks. In the case where multiple subtasks include a first target subtask, the first target subtask is executed using the first text processing model deployed in the first device. In the case where multiple subtasks include a second target subtask, the second target subtask is executed using at least one of the second text processing model deployed in the first device and the second text processing model deployed in the second device according to the available resources of the second device, and the first text processing model and the second text processing model are the same type of models and are trained based on different task training data. Therefore, different subtasks are processed respectively using the text processing models deployed in the first device and the second device, so that the computing resources consumed by each device are reduced, and the processing efficiency of the text task is also improved.

[0061] According to an embodiment of the present disclosure, the stage characteristics of a text task are determined according to a processing request of the text task, including: determining the task attributes of the text task according to the task identifier in the processing request of the text task; obtaining the to-be-processed file of the text task according to the processing request; determining the file format of the to-be-processed file according to the file identifier of the to-be-processed file; classifying the text task according to the text format, task attributes and the estimated processing amount of the text task to obtain a classification result; and determining the stage characteristics of the text task according to the classification result.

[0062] According to an embodiment of the present disclosure, the task identifier may be generated based on application attributes, system attributes, or user operations on the interactive interface.

[0063] For example, an application attribute can be language translation, and a system attribute can be search recommendation. Application attributes correspond to task attributes.

[0064] For example, the interactive interface may have a function button. When the user operates the "image search" button in the interactive interface, the text task is marked to obtain a task identifier. The function button corresponds to the task attribute.

[0065] According to an embodiment of the present disclosure, the processing request may include an acquisition address of the file to be processed.

[0066] For example, the file identifier may be jpg, and the file to be processed is in picture format; the file identifier may be text, doc, pdf, etc., and the file to be processed is in text format.

[0067] According to an embodiment of the present disclosure, the estimated processing volume of a text task may be determined based on performance information of a text processing model, available resources of a first device, and available resources of a second device.

[0068] According to an embodiment of the present disclosure, text tasks are classified according to text format, task attributes and estimated processing volume of text tasks to obtain classification results that are more in line with resource scheduling between devices; then, the stage characteristics of the text tasks are determined based on the classification results, so that subsequent subtasks divided according to the text tasks can accurately schedule the first device and the second device to execute the subtasks.

[0069] According to an embodiment of the present disclosure, the text task is classified according to the text format, the task attribute and the estimated processing amount of the text task to obtain a classification result; and the stage characteristics of the text task are determined according to the classification result, including:

[0070] When the estimated processing volume of the text task meets the processing volume threshold condition, the text task is marked as a target text task, and the processing volume threshold condition is determined based on a preset subtask resource threshold, and the preset subtask resource threshold can be determined based on the available resources of the second device; when the estimated processing volume of the text task does not meet the processing volume threshold condition, the text task is classified according to the text format task attributes to obtain a classification result; and the stage characteristics of the text task are determined based on the classification result.

[0071] According to an embodiment of the present disclosure, the processing volume threshold condition may be greater than the processing volume threshold.

[0072] For example, the text task may be text preprocessing, but the number of documents in the attachment is large, causing the estimated processing volume of the text task to be far greater than the processing volume threshold. The text task is marked as a target text task with a multi-stage processing process, so as to perform the target text task using a first text processing model deployed in the first device and a second text processing model deployed in the second device.

[0073] Therefore, by marking a text task with a large amount of processing and simple repetition as a target text task, the text task is divided into a first target subtask that consumes less computing resources and a second target subtask that needs to be repeatedly executed; the first target subtask is executed using the first text processing model deployed in the first device to reduce the load of the first device and improve the response speed of the system or application. In the case where multiple subtasks include a second target subtask, the second target subtask is executed using at least one of the second text processing model deployed in the first device and the second text processing model deployed in the second device according to the available resources of the second device. Since the second device is good at processing large amounts of simple and repetitive work, it can achieve effective utilization of available resources and improve processing efficiency.

[0074] When the computing resources consumed by the text task are less than the preset subtask resource threshold, it means that the computing resources consumed by multiple subtasks of the text task are all less than the set subtask resource threshold. The text task is not marked as a target text task, so as to directly execute the text task with a small processing volume using the text processing model deployed in the first device, so as to avoid complicated processing of the text task with a small processing volume and improve the reasonable scheduling of available resources between the first device and the second device.

[0075] According to an embodiment of the present disclosure, a text task is divided into stages according to stage characteristics to obtain multiple subtasks, including: when the stage characteristics characterize that the text task is a target text task with a multi-stage processing process, the text task is divided into a first target subtask and a second target subtask; when the stage characteristics characterize that the text task is not a target text task, the text task is determined as the first target subtask.

[0076] For example, the text task can be divided into a first target subtask and a second target subtask according to the association relationship between the multiple subtasks. The first target subtask can be processed in series with the second target subtask, and the computing resource consumption of the first target subtask is small. Multiple second target subtasks can be processed in parallel to take advantage of the parallel processing of the second device.

[0077] For example, the text task may be divided into a first target subtask and a second target subtask according to the processing difficulty of the multiple subtasks.

[0078] For example, text preprocessing and text segmentation can be simple subtasks, while semantic analysis, text vectorization, and recognition of text in images can be complex subtasks. Simple subtasks can be divided into first target subtasks, and complex subtasks can be divided into second target subtasks.

[0079] According to the embodiments of the present disclosure, the method of dividing text tasks is not limited to association relationships and processing difficulty, and can be limited according to actual available resources of the device.

[0080] According to an embodiment of the present disclosure, the first device may deploy a first text model and a second text model. The first text processing model or the second text model may have multiple available modules. The multiple available modules of the first text processing model may correspond to the type of the first target subtask. The multiple available modules of the second text model may correspond to the type of the second target subtask.

[0081] According to an embodiment of the present disclosure, the first target subtask includes: text segmentation, the first text processing model includes a text segmentation module, the second target subtask includes at least one of the following: image processing and text vectorization, and the second text processing model includes: an image processing module and a text vectorization module.

[0082] Figure 3 A schematic diagram of text processing in an intelligent question-answering system according to an embodiment of the present disclosure is schematically shown.

[0083] like Figure 3 As shown, the intelligent question answering system of this embodiment includes a text processing model 301, a database 302 and a large language model 303.

[0084] The question text or image input by the user in the intelligent question answering system is obtained; the question text or image is input into the text processing model 301 respectively to obtain the question text vector or the character vector in the image. The text processing model can process the question text through text preprocessing, text segmentation, text conversion to vector and other processing processes. The text processing model can process the image through character recognition, text segmentation, and text conversion to vector.

[0085] The question text vector or the character vector in the image is matched with the files in the database 302 for similarity, so as to obtain files related to the question text or the image.

[0086] The question text or image, and files related to the question text or image are input into the large language model 303 to obtain an answer.

[0087] When multiple subtasks include a first target subtask, the first target subtask is executed using a first text processing model deployed in the first device, including: when the first target subtask is text segmentation, the text of the first target subtask is processed using a text segmentation module of the first text processing model to obtain text after text segmentation.

[0088] According to an embodiment of the present disclosure, the first text processing model may include a text segmentation model for performing text segmentation and a preset text processing model. Each module of the preset text processing model is in an available state and can process text tasks with small computing resources, thereby avoiding task complication and improving text task processing efficiency.

[0089] For a target text task with multiple stages and the available module of the first text processing model is a segmentation module, the first text processing model can be directly used to perform text segmentation, thereby improving the processing efficiency of text segmentation in the target text task.

[0090] When the computing resources consumed by the text task are less than a preset subtask resource threshold, the text task is divided into multiple first target subtasks, and the preset text processing model deployed by the first device can be directly used to execute multiple first target subtasks in the text task.

[0091] For example, a text task is to vectorize user input information. The available modules of the preset text processing model may include a text segmentation module, an image processing module, and a text vectorization module. The computing resources consumed by the text task are less than the preset subtask resource threshold, so the text task is divided into multiple first target subtasks. Multiple first target subtasks include text segmentation and text vectorization. The text segmentation module of the preset text processing model is used to process the user input information to obtain the text after text segmentation. The text after text segmentation is input into the vectorization module of the preset text processing model to obtain the target text vector.

[0092] In a case where multiple subtasks include a second target subtask, based on the available resources of the second device, the second target subtask is executed using at least one of the second text processing model deployed in the first device and the second text processing model deployed in the second device, including: in a case where the second target subtask includes image processing or text vectorization, the second target subtask is executed using the second text processing model; in a case where the second target subtask includes image processing and text vectorization, the second target subtask is reallocated based on the available resources of the second device to execute the second target subtask using at least one of the second text processing model deployed in the first device and the second text processing model deployed in the second device.

[0093] According to an embodiment of the present disclosure, it is determined whether the second device is capable of simultaneously and in parallel processing image processing and text vectorization based on available resources of the second device.

[0094] Since the text processing model can be called by multiple applications or systems, there may be multiple text task processing requests received simultaneously.

[0095] In response to receiving a processing request for multiple text tasks, the multiple text tasks are first divided to obtain the first target subtasks and second target subtasks of the multiple text tasks, that is, the first target subtasks of the multiple text tasks and the second target subtasks of the multiple text tasks need to be processed simultaneously.

[0096] Multiple first target subtasks can be processed in sequence by the first processing model. When the second target subtask includes only one of image processing and text vectorization, the second target subtask can be directly executed by the second text processing model deployed by the second device, and multiple text tasks can be processed in parallel to improve the reasonable scheduling of available resources of the device and improve the text processing efficiency.

[0097] In response to receiving a processing request for multiple text tasks, but the second target subtasks in the multiple text tasks include image processing and text vectorization, the second target subtasks are reallocated according to the available resources of the second device to prevent the situation where the available resources of the first device are sufficient while the available resources of the second device are scarce.

[0098] According to an embodiment of the present disclosure, when the second target subtask includes image processing or text vectorization, the second text processing model is used to execute the second target subtask to achieve rapid processing of the second target subtask; when the second target subtask includes image processing and text vectorization, in order to prevent the situation where the available resources of the first device are sufficient while the available resources of the second device are scarce, the second target subtask can be reallocated according to the available resources of the second device to execute the second target subtask using at least one of the second text processing model deployed in the first device and the second text processing model deployed in the second device, thereby achieving rational utilization of device resources and improving text processing efficiency.

[0099] According to an embodiment of the present disclosure, when the second target subtask includes image processing or text vectorization, the second target subtask is executed using a text processing model of the second device, including: when the second target subtask is image processing, using an image processing module in the second text processing model to process the image in the second target subtask to recognize text in the image; when the second target subtask is text vectorization processing, using a text vectorization module in the second text processing model to process the text in the second target subtask.

[0100] For example, when multiple text tasks need to be processed, it can be determined that the second target subtask includes a text task of image processing or text vectorization. The second target subtask including image processing or text vectorization is directly processed by the second device, which can improve the processing progress of the text task and reduce the amount of text tasks to be processed.

[0101] According to an embodiment of the present disclosure, in the case where the second target subtask includes only one of image processing or text vectorization, the second device can be used for direct processing, which can speed up the processing progress of the text task.

[0102] According to an embodiment of the present disclosure, the text in the second target subtask is processed using a text vectorization module in a second text processing model deployed in a second device, including: obtaining text after text segmentation from the first device; and processing the text after text segmentation using the text vectorization module to obtain a target text vector.

[0103] According to an embodiment of the present disclosure, the subtasks may be processed in series. The text segmented by the first text processing model may be obtained through a communication signal between the first device and the second device. The text segmented by the text vectorization module of the second processing model may be processed.

[0104] According to an embodiment of the present disclosure, the second text processing model includes an image processing model for performing image processing and a vectorization model for performing text vectorization, and the second target subtask is reallocated according to the available resources of the second device, including: when the available resources meet the preset resource conditions, image processing is allocated to the image processing module of the image processing model; and text vectorization is allocated to the text vectorization module of the vectorization model; when the available resources of the second device do not meet the preset resources, according to the first processing time required for the image processing and the second processing time required for the text vectorization, the second target subtask to be allocated that meets the preset time condition is determined from the image processing and text vectorization processing; and the second target subtask to be allocated is allocated to the second text processing model deployed on the first device.

[0105] According to an embodiment of the present disclosure, the preset resource condition may be that the available resources are greater than the consumed resources for performing text vectorization and image processing in parallel. If the available resources meet the preset resource condition, the available resources of the second device are sufficient. If the available resources do not meet the preset resource condition, the available resources of the second device are insufficient.

[0106] According to an embodiment of the present disclosure, when the available resources of the second device are sufficient, image processing and text vectorization can be selected for parallel processing, which can improve the processing efficiency of the text task with parallel processing subtasks.

[0107] According to an embodiment of the present disclosure, if the available resources of the second device are insufficient, the first processing time required for image processing and the second processing time required for text vectorization may be determined first. The first processing time and the second processing time may be calculated by a task scheduling system. The task scheduling system is used to monitor the resources of the first device and the second device, and may monitor the task processing status of the text processing model, etc. Therefore, the task scheduling system may combine information such as the resources of the first device and the second device, the task processing status of the text processing model, and estimate the first processing time and the second processing time.

[0108] According to an embodiment of the present disclosure, the preset duration condition may be a short processing duration, that is, the second target subtask to be assigned may be a second target subtask with a short processing duration.

[0109] According to an embodiment of the present disclosure, the second target subtask with a short processing time is assigned to the second text processing model deployed on the first device, which does not affect the normal text task processing, ensures the normal operation of the application or system of the first device, and can also reduce the load on the second device.

[0110] According to an embodiment of the present disclosure, the text vectorization module of the image processing model is unavailable, and the image recognition module of the vectorization model is unavailable.

[0111] According to an embodiment of the present disclosure, the text processing model can make a certain module in the text processing model unavailable by freezing module parameters, masking, and other operations.

[0112] According to the embodiments of the present disclosure, the image processing model is directly used to perform only image recognition processing, and the vectorization model is used to perform only text vectorization, thereby reducing redundant calculations and speeding up the parallel processing of image recognition and text vectorization.

[0113] Figure 4A The figure schematically shows a processing diagram of a text task for an image according to an embodiment of the present disclosure.

[0114] like Figure 4A As shown, the image 401 is subjected to text recognition processing to obtain the to-be-processed text 402. The to-be-processed text 402 is subjected to text segmentation processing to obtain the segmented text 403. The segmented text 403 is subjected to text vectorization processing to obtain the target text vector 404. The target text vector 404 can be matched with the text in the database for similarity to obtain the text related to the image. The text related to the image is input into the large language model to obtain the semantic information in the image.

[0115] Figure 4B The structural block diagram of the text processing model according to the embodiment of the present disclosure is schematically shown.

[0116] like Figure 4B As shown, the text processing module 500 may include an image processing module 501 , a text segmentation module 502 , and a text vectorization module 503 .

[0117] The text processing model can be customized. The image processing module 501, the text segmentation module 502, and the text vectorization module 503 can set the running state respectively. For example, when the text processing model performs the first target subtask, the text segmentation module 502 of the text processing model is in an available state, while the image processing module 501 and the text vectorization module 503 are in an unavailable state.

[0118] For example, the text vectorization module and text segmentation module of the image processing model are unavailable, and the image recognition module and text segmentation module of the vectorization model are unavailable.

[0119] Figure 5 The flowchart of a text processing method according to another embodiment of the present disclosure is schematically shown.

[0120] like Figure 5 As shown, the text processing method of this embodiment includes operations S501 to S512.

[0121] In operation S501 , in response to receiving a processing request for a text task, a phase feature of the text task is determined.

[0122] In operation S502, the text task is divided into stages according to the stage characteristics to obtain a plurality of subtasks.

[0123] In operation S503, when the multiple subtasks include a first target subtask and a second target subtask and the first target subtask is text segmentation, the text of the first target subtask is processed using a text segmentation module of a first text processing model to obtain a text after text segmentation.

[0124] In operation S504, it is determined whether the second target subtask includes image processing or text vectorization. If so, operation S505 is performed; if not, operation S507 is performed.

[0125] In operation S505, when the second target subtask is image processing, the image in the second target subtask is processed by using the image processing module in the second text processing model to recognize text in the image.

[0126] In operation S506, when the second target subtask is text vectorization processing, the text in the second target subtask is processed using the text vectorization module in the second text processing model.

[0127] In operation S507 , it is determined that the second target subtask includes image processing and text vectorization.

[0128] In operation S508, it is determined whether the available resources of the second device meet the preset resource condition. If yes, operation S509 is executed; if not, operation S511 is executed.

[0129] In operation S509, image processing is assigned to an image processing module of an image processing model.

[0130] In operation S510 , text vectorization is assigned to a text vectorization module of a vectorization model.

[0131] In operation S511, according to a first processing time required for image processing and a second processing time required for text vectorization, a second target subtask to be assigned that meets a preset time condition is determined from the image processing and the text vectorization processing.

[0132] In operation S512, the second target subtask to be assigned is assigned to the second text processing model deployed on the first device.

[0133] Figure 6 The structure block diagram of a task processing device according to an embodiment of the present disclosure is schematically shown.

[0134] like Figure 6 As shown, the task processing device 600 of this embodiment includes a determination module 610 , a stage division module 620 , a first execution module 630 and a second execution module 640 .

[0135] The determination module 610 is used to determine the phase characteristics of the text task in response to receiving a processing request for the text task. In one embodiment, the determination module 610 can be used to perform the operation S210 described above, which will not be described in detail here.

[0136] The stage division module 620 is used to divide the text task into stages according to the stage characteristics to obtain a plurality of subtasks. In one embodiment, the stage division module 620 can be used to perform the operation S220 described above, which will not be described in detail here.

[0137] The first execution module 630 is used to execute the first target subtask using the first text processing model deployed in the first device when the multiple subtasks include the first target subtask. In one embodiment, the first execution module 630 can be used to execute the operation S230 described above, which will not be repeated here.

[0138] The second execution module 640 is used to execute the second target subtask using at least one of the second text processing model deployed in the first device and the second text processing model deployed in the second device according to the available resources of the second device when the multiple subtasks include the second target subtask, and the first text processing model and the second text processing model are the same type of models and are trained based on different task training data. In one embodiment, the second execution module 640 can be used to execute the operation S240 described above, which will not be repeated here.

[0139] According to an embodiment of the present disclosure, the determination module 610 includes a first determination submodule, an acquisition submodule, a second determination submodule, a classification submodule and a third determination submodule. The first determination submodule is used to determine the task attribute of the text task according to the task identifier in the processing request of the text task; the acquisition submodule is used to obtain the to-be-processed file of the text task according to the processing request; the second determination submodule is used to determine the file format of the to-be-processed file according to the file identifier of the to-be-processed file; the classification submodule is used to classify the text task according to the text format, task attributes and the estimated processing amount of the text task to obtain the classification result; the third determination submodule is used to determine the stage characteristics of the text task according to the classification result.

[0140] According to an embodiment of the present disclosure, the stage division module 620 includes a division submodule and a fourth determination submodule. The division submodule is used to divide the text task into a first target subtask and a second target subtask when the stage feature characterization text task is a target text task with a multi-stage processing process; the fourth determination submodule is used to determine the text task as the first target subtask when the stage feature characterization text task is not a target text task.

[0141] According to an embodiment of the present disclosure, the first target subtask includes: text segmentation, the first text processing model includes a text segmentation module, the second target subtask includes at least one of the following: image processing and text vectorization, and the second text processing model includes: an image processing module and a text vectorization module.

[0142] The first execution module includes a first processing submodule. The first processing submodule is used to process the text of the first target subtask using the text segmentation module of the first text processing model to obtain the text after the text segmentation when the first target subtask is text segmentation.

[0143] The second execution module includes a first execution submodule and a reallocation submodule. The first execution submodule is used to use the second text processing model to execute the second target subtask when the second target subtask includes image processing or text vectorization; the reallocation submodule is used to reallocate the second target subtask according to available resources of the second device when the second target subtask includes image processing and text vectorization, so as to use at least one of the second text processing model deployed in the first device and the second text processing model deployed in the second device to execute the second target subtask.

[0144] According to an embodiment of the present disclosure, the first execution submodule includes a first processing unit and a second processing unit. The first processing unit is used to process the image in the second target subtask using the image processing module in the second text processing model to recognize the text in the image when the second target subtask is image processing. The second processing unit is used to process the text in the second target subtask using the text vectorization module in the second text processing model when the second target subtask is text vectorization processing.

[0145] According to an embodiment of the present disclosure, the second processing unit includes an acquisition subunit and a processing subunit. The acquisition subunit is used to acquire the text after text segmentation from the first device; the processing subunit is used to process the text after text segmentation using a text vectorization module to obtain a target text vector.

[0146] According to an embodiment of the present disclosure, the second text processing model includes an image processing model for performing image processing and a vectorization model for performing text vectorization. The reallocation submodule includes a first allocation unit, a determination unit, and a second allocation unit. The first allocation unit is used to allocate image processing to the image processing module of the image processing model when the available resources meet the preset resource conditions; and to allocate text vectorization to the text vectorization module of the vectorization model; the determination unit is used to determine the second target subtask to be allocated that meets the preset time condition from the image processing and text vectorization processing according to the first processing time required for the image processing and the second processing time required for the text vectorization when the available resources of the second device do not meet the preset resources; the second allocation unit is used to allocate the second target subtask to be allocated to the second text processing model deployed on the first device.

[0147] According to an embodiment of the present disclosure, the text vectorization module of the image processing model is unavailable, and the image recognition module of the vectorization model is unavailable.

[0148] According to an embodiment of the present disclosure, any multiple modules of the determination module 610, the phase division module 620, the first execution module 630, and the second execution module 640 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the determination module 610, the phase division module 620, the first execution module 630, and the second execution module 640 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the determination module 610, the stage division module 620, the first execution module 630 and the second execution module 640 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.

[0149] Figure 7 A block diagram of an electronic device suitable for implementing a task processing method according to an embodiment of the present disclosure is schematically shown.

[0150] like Figure 7As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0151] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0152] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0153] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0154] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0155] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the task processing method provided by the embodiment of the present disclosure.

[0156] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 701. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0157] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0158] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0159] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0160] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0161] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0162] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A task processing method, characterized in that: The method comprises: In response to receiving a processing request for a text task, determining a phase feature of the text task; Dividing the text task into stages according to the stage characteristics to obtain a plurality of subtasks; In a case where the plurality of subtasks include a first target subtask, executing the first target subtask using a first text processing model deployed in a first device; In the case that the multiple subtasks include a second target subtask, the second target subtask is performed using at least one of a second text processing model deployed in the first device and the second text processing model deployed in the second device according to available resources of the second device, and the first text processing model and the second text processing model are of the same type of models and are trained based on different task training data.

2. The method according to claim 1, characterized in that The step of determining the stage characteristics of the text task according to the processing request of the text task includes: Determining a task attribute of the text task according to a task identifier in the processing request of the text task; According to the processing request, obtaining the to-be-processed file of the text task; Determining the file format of the file to be processed according to the file identifier of the file to be processed; Classifying the text task according to the text format, the task attributes and the estimated processing amount of the text task to obtain a classification result; The stage characteristics of the text task are determined according to the classification result.

3. The method according to claim 1, characterized in that The text task is divided into stages according to the stage characteristics to obtain multiple subtasks, including: In the case where the stage feature represents that the text task is a target text task with a multi-stage processing process, dividing the text task into the first target subtask and the second target subtask; When the stage feature indicates that the text task is not the target text task, the text task is determined as the first target subtask.

4. The method according to claim 1, characterized in that: The first target subtask includes: text segmentation, the first text processing model includes a text segmentation module, the second target subtask includes at least one of the following: image processing and text vectorization, the second text processing model includes: an image processing module and a text vectorization module; In the case where the multiple subtasks include a first target subtask, executing the first target subtask using a first text processing model deployed in a first device includes: In the case where the first target subtask is the text segmentation, using the text segmentation module of the first text processing model to process the text of the first target subtask to obtain a text after the text segmentation; In the case where the multiple subtasks include a second target subtask, performing the second target subtask using at least one of a second text processing model deployed in the first device and a second text processing model deployed in the second device according to available resources of the second device, comprises: In a case where the second target subtask includes the image processing or the text vectorization, performing the second target subtask using the second text processing model; In the case where the second target subtask includes the image processing and the text vectorization, the second target subtask is reallocated according to the available resources of the second device to perform the second target subtask by utilizing at least one of the second text processing model deployed in the first device and the second text processing model deployed in the second device.

5. The method according to claim 4, characterized in that In the case where the second target subtask includes the image processing or the text vectorization, using the text processing model of the second device to perform the second target subtask includes: In the case where the second target subtask is the image processing, using the image processing module in the second text processing model to process the image in the second target subtask to recognize text in the image; When the second target subtask is the text vectorization processing, the text in the second target subtask is processed using the text vectorization module in the second text processing model.

6. The method according to claim 5, characterized in that The processing of the text in the second target subtask by using the text vectorization module in the second text processing model deployed in the second device includes: Acquire the text after the text segmentation from the first device; The text vectorization module is used to process the segmented text to obtain a target text vector.

7. The method according to claim 4, characterized in that The second text processing model includes an image processing model for performing the image processing and a vectorization model for performing the text vectorization. The reallocating the second target subtask according to the available resources of the second device includes: In the case where the available resources meet the preset resource conditions, the image processing is assigned to the image processing module of the image processing model; and the text vectorization is assigned to the text vectorization module of the vectorization model; In a case where the available resources of the second device do not meet the preset resources, determining a second target subtask to be assigned that meets the preset duration condition from the image processing and the text vectorization processing according to a first processing duration required for the image processing and a second processing duration required for the text vectorization; The second target subtask to be assigned is assigned to the second text processing model deployed on the first device.

8. The method according to claim 7, characterized in that The text vectorization module of the image processing model is unavailable, and the image recognition module of the vectorization model is unavailable.

9. A task processing device, characterized in that: The device comprises: A determination module, configured to determine a phase feature of the text task in response to receiving a processing request for the text task; A stage division module, used for dividing the text task into stages according to the stage characteristics to obtain a plurality of subtasks; A first execution module, configured to execute a first target subtask by using a first text processing model deployed in a first device when the plurality of subtasks include the first target subtask; A second execution module is used to execute the second target subtask by utilizing at least one of a second text processing model deployed in the first device and a second text processing model deployed in the second device according to available resources of the second device when the multiple subtasks include the second target subtask, wherein the first text processing model and the second text processing model are of the same type of models and are trained based on different task training data.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images