Data processing method and device, equipment and storage medium

Abstract

The present disclosure provides a data processing method and device, equipment and storage medium, relates to the technical field of artificial intelligence, in particular to the technical field of cloud native, container technology, resource scheduling and the like. The specific implementation scheme is: a management server deployed in a k8s cluster acquires a data processing request of a user; in response to the data processing request for requesting to process the to-be-processed data based on a target model within a target time length, in a case where a pod in which the target model has been deployed in the k8s cluster does not meet the data processing request, the management server determines a target query rate per second (qps) corresponding to the data processing request according to the target time length and the size of the to-be-processed data; further, the management server determines a plurality of target pods from idle pods of a plurality of pods based on the target qps, and processes the to-be-processed data based on the plurality of target pods to obtain a processing result; wherein the total qps of the target models corresponding to the plurality of target pods is greater than or equal to the target qps.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to cloud-native, container technology, and resource scheduling technologies, and specifically to data processing methods, devices, equipment, and storage media. Background Technology

[0002] Currently, service providers can offer data processing services such as data analysis, data classification, and prediction to customers through intelligent insight platforms.

[0003] Specifically, the intelligent insight platform deploys multiple application container engine Docker containers. Each Docker container deploys a model that processes data based on artificial intelligence or machine learning. When a user initiates a data processing service request, the user sends the data to be processed to the Docker container through the user's device. The model in the Docker container then processes the data to be processed and obtains the processing result, thereby providing the user with data processing services. Summary of the Invention

[0004] This disclosure provides a data processing method, apparatus, device, and storage medium for providing data processing services.

[0005] According to one aspect of this disclosure, a data processing method is provided, applied to a management server in a data processing system; the data processing system is built on a Kubernetes cluster and also includes multiple minimum scheduling units (pods); the method includes:

[0006] Obtain the user's data processing request; the data processing request is used to request that the data to be processed be completed based on the target model within the target time period;

[0007] If a pod with the target model already deployed does not meet the data processing request, determine the target query rate per second (QPS) for the data processing request based on the target duration and the size of the data to be processed.

[0008] Based on the target QPS, multiple target pods are determined from the idle pods of multiple pods; these multiple target pods are used to deploy the target model, and the total QPS of the target model corresponding to the multiple target pods is greater than or equal to the target QPS.

[0009] The data to be processed is processed based on multiple target pods to obtain the processing results.

[0010] According to another aspect of this disclosure, a data processing apparatus is provided, applied to a management server in a data processing system; the data processing system is built on a Kubernetes cluster and further includes multiple minimum scheduling units (pods); the apparatus includes an acquisition unit, a determination unit, and a processing unit.

[0011] The acquisition unit is used to acquire the user's data processing request; the data processing request is used to request that the data to be processed be completed based on the target model within the target time period.

[0012] The determination unit is used to determine the target query rate per second (QPS) for the data processing request based on the target duration and the size of the data to be processed, when the pod with the target model already deployed does not meet the data processing request.

[0013] The unit is also used to determine multiple target pods from the idle pods of multiple pods based on the target QPS; the multiple target pods are used to deploy the target model, and the total QPS of the target model corresponding to the multiple target pods is greater than or equal to the target QPS;

[0014] The processing unit is used to process data to be processed based on multiple target pods and obtain the processing results.

[0015] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0016] At least one processor; and a memory communicatively connected to the at least one processor;

[0017] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the data processing method provided in this disclosure.

[0018] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the data processing method provided in this disclosure.

[0019] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the data processing method provided in this disclosure.

[0020] The technical solution provided in this disclosure, since the data processing system is built on a Kubernetes cluster, allows the management server to schedule and maintain multiple pods based on the distributed management characteristics of the Kubernetes cluster. Therefore, when the total QPS of the target model deployed in multiple target pods is greater than or equal to the target QPS, the time consumed by the multiple target pods to process the data to be processed will be less than the target duration, thus meeting the user's requirement for the target duration.

[0021] Meanwhile, since the target QPS is calculated based on the target duration, and the target QPS is used as the basis for pod scheduling, resources can be scheduled on demand during data processing, enabling precise scheduling of pods.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0023] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0024] Figure 1 This is a schematic diagram of the structure of an intelligent insight platform according to an embodiment of the present disclosure;

[0025] Figure 2 This is a schematic diagram of the structure of a data processing system according to an embodiment of the present disclosure;

[0026] Figure 3 This is a schematic flowchart illustrating a data processing method according to an embodiment of the present disclosure;

[0027] Figure 4 This is a schematic flowchart illustrating a data processing method according to an embodiment of the present disclosure;

[0028] Figure 5 This is a schematic flowchart illustrating a data processing method according to an embodiment of the present disclosure;

[0029] Figure 6 This is a schematic flowchart illustrating a data processing method according to an embodiment of the present disclosure;

[0030] Figure 7 This is a schematic flowchart illustrating a data processing method according to an embodiment of the present disclosure;

[0031] Figure 8 This is a structural block diagram of a data processing apparatus used to implement the data processing method of the embodiments of this disclosure.

[0032] Figure 9 This is a structural block diagram of an electronic device used to implement the data processing method of the embodiments of this disclosure. Detailed Implementation

[0033] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0034] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0035] First, the inventive principles of the embodiments of this disclosure will be described:

[0036] Currently, service providers can offer data processing services such as data analysis, data classification, and prediction to customers through intelligent insight platforms.

[0037] Figure 1 A schematic diagram of the intelligent insight platform is shown, such as... Figure 1 As shown, the intelligent insight platform 10 includes a server 101 and multiple Docker containers ( Figure 1 An example is shown with Docker containers 102 and 103, and server 101 connected to multiple Docker containers.

[0038] Specifically, when a user sends a data processing request to server 101, the request includes the data to be processed and the Uniform Resource Locator (URL) of the target Docker container where the target model resides. Upon receiving the request, server 101 sends the data to be processed to the target Docker container based on its URL, enabling the target model within the container to process the data and obtain the processing result. Furthermore, the target Docker container returns the processing result to server 101, allowing server 101 to send the result to the user.

[0039] However, in the aforementioned intelligent insight platform, when too many users initiate data processing requests for the same model, the limited processing capacity of Docker containers causes users who initiate data processing requests later to be in a queue, resulting in longer waiting times and a reduced user experience.

[0040] Meanwhile, since each model in a Docker container exists as a resident model, daily operations, maintenance, scheduling, and other CRUD operations are all performed manually by operations and maintenance personnel, resulting in low operational efficiency.

[0041] Based on this, the embodiments of this disclosure consider deploying the intelligent insight platform in a Kubernetes (k8s) cluster and deploying the model in the smallest scheduling unit pod of the k8s cluster. In this way, leveraging the distributed management characteristics of the k8s cluster, models can be deployed automatically and dynamically across multiple pods to provide users with timely and efficient data processing services. Furthermore, based on the user's expectation of processing the data within a target timeframe when initiating a data processing request, the intelligent insight platform deployed on the k8s cluster can also determine the target pods where the target service needs to be deployed, according to the target timeframe and the user's desired target model, and process the data to be processed based on the target pods to meet the user's requirement for the target timeframe.

[0042] Secondly, the application scenarios of the embodiments of this disclosure will be described:

[0043] Figure 2 This is a schematic diagram of the structure of a data processing system for implementing a data processing method according to an embodiment of the present disclosure. See also: Figure 2 The data processing system 20 includes a management server 201 and multiple application servers. Figure 2 Application servers 202, 203, and 204 are shown only as examples; in actual applications, there may be more or fewer application servers. Figure 2 (Not shown again). Management server 201 is connected to application server 202, application server 203 and application server 204 respectively.

[0044] The data processing system described above can be the intelligent insight platform in this embodiment, and the management server in the data processing system can be a server in the intelligent insight platform. In this case, the management server 201 can be the master node in the Kubernetes cluster, and the application server can be a node in the Kubernetes cluster.

[0045] The management server 201 is configured with an application program interface server (APIServer) component, which allows the management server 201 to manage multiple application servers.

[0046] It should be noted that the APIServer component mentioned above can also be a device that is relatively independent of the management server 201. In this case, the communication process between the management server 201 and the APIServer component is the same as the communication process between the internal modules of the management server 201.

[0047] Each application server can be a physical machine or a virtual machine, and each application server is configured with a Kubelet component and one or more minimum scheduling units (pods). The application server can manage, schedule, and configure one or more pods within it through the Kubelet component.

[0048] Each pod can deploy models, each with different functions to process user-provided data, such as image recognition, data classification, and hotspot prediction.

[0049] For example, Figure 2 Application servers 202, 203, and 204 are shown. Pods 1 and 2 are deployed in application server 202, pods 3, 4, and 5 are deployed in application server 203, and pods 6, 7, and 8 are deployed in application server 204. Image recognition models are deployed in pods 1, 2, and 5, a classification model is deployed in pod 3, and no models are deployed in pods 4, 6, and 7.

[0050] In one possible implementation, the data processing method provided in this embodiment is executed by the management server 201, or by a data processing device internal or external to the management server 201, or by other similar electronic devices. The following description uses the example of the data processing method being executed by the management server.

[0051] Figure 3 This is a flowchart illustrating a data processing method according to an embodiment of the present disclosure. The data processing method is applied to a management server in a data processing system. The data processing system is built on a Kubernetes cluster and also includes multiple application servers and multiple pods deployed on the application servers.

[0052] like Figure 3 As shown, the method includes the following steps:

[0053] S301, The management server obtains user data processing requests.

[0054] The data processing request is used to request that the data to be processed be completed based on the target model within the target duration. The data processing request includes the data to be processed, the target duration, and the identifier of the target model.

[0055] As one possible implementation, the management server can receive data processing requests sent by the user's user device.

[0056] It should be noted that a data processing request may also include the data to be processed, the target time when the user expects to complete the processing of the data to be processed, and the target processing type for the data to be processed requested by the user.

[0057] In this scenario, the management server can determine the target duration based on the time interval between the target time and the current time.

[0058] At the same time, the management server can also query the target model corresponding to the target processing type from the preset first mapping relationship based on the above target processing type.

[0059] The first mapping relationship includes the correspondence between different processing types and different models. This first mapping relationship can be pre-set in the management server by the data processing system's operation and maintenance personnel.

[0060] For example, if a user requests that the target processing type for the data to be processed be classification, the management server can query the first mapping relationship to find that the target model is a classification model. If a user requests that the target processing type for the data to be processed be emotion recognition, the management server can query the first mapping relationship to find that the target model is an emotion recognition model.

[0061] As another possible implementation, the management server can display a user interface where users can upload data to be processed, enter the target duration, and select the target model. Furthermore, in response to the user's confirmation in the user interface, the management server generates a data processing request.

[0062] The specific implementation method of this step can be found in the subsequent description of the embodiments of this disclosure, and will not be repeated here.

[0063] S302, The management server determines the pods from multiple pods that have the target model deployed.

[0064] As one possible implementation, the management server, based on the APIServer component, queries each of the multiple application servers to see if the target model is deployed in the pods deployed on each application server.

[0065] It should be noted that each application server can obtain the deployment status of each pod based on the Kubelet component, such as the models deployed in each pod and the physical resources configured in each pod (number of central processing units (CPUs), amount of memory, and number of graphics processing units (GPUs)).

[0066] For example, such as Figure 2 As shown, when the target model for the user's request to process the data is an image recognition model, the management server can identify three pods that have already deployed the target model as pod1, pod2, and pod5.

[0067] S303, The management server determines whether the pods with the target model already deployed meet the data processing request.

[0068] As one possible implementation, after the management server determines that a pod with the target model has been deployed, it determines whether the target model deployed in the pod can process the data to be processed within the target time.

[0069] It should be noted that since the management server may identify multiple pods with the target model already deployed, the management server needs to determine whether each pod with the target model already deployed meets the data processing request.

[0070] If the management server determines that any pod with the target model deployed can process the data to be processed within the target time period, then the pod with the target model deployed is deemed to satisfy the above data processing request. Conversely, if the management server determines that none of the target models deployed in any pod with the target model can process the data to be processed within the target time period, then the pod with the target model deployed is deemed not to satisfy the above data processing request.

[0071] The specific implementation of this step can be found in the subsequent description of the embodiments of this disclosure, and will not be repeated here.

[0072] S304. If the pod with the target model already deployed does not meet the data processing request, the management server determines the target queries per second (qps) corresponding to the data processing request based on the target duration and the size of the data to be processed.

[0073] As one possible implementation, if the management server determines that the pods with the target model cannot process the data to be processed within the target duration, it determines the target QPS for the data processing requests based on the size of the data to be processed and the target duration.

[0074] Specifically, the management server can determine the target QPS as the ratio of the size of the data to be processed to the target duration. Alternatively, the management server can also determine the target QPS by multiplying the above ratio by a preset coefficient, where the preset coefficient is a value greater than 1.

[0075] Understandably, the management server has preset coefficients that can make the target QPS greater than the user's expected QPS. This allows for a certain amount of time redundancy, ensuring that the data to be processed can be completed within the target time.

[0076] S305, the management server determines multiple target pods from the idle pods of multiple pods based on the target QPS.

[0077] Multiple target pods are used to deploy the target model, and the total QPS of the target model corresponding to the multiple target pods is greater than or equal to the target QPS. Idle pods include pods that are currently not in use.

[0078] As one possible implementation, the management server obtains the preset QPS of the target model and determines the number of target pods based on the ratio of the target QPS to the preset QPS. Further, the management server identifies idle pods from the multiple pods and, based on the number of target pods, selects multiple target pods from the identified idle pods.

[0079] It should be noted that the total QPS of the target models corresponding to multiple target pods is specifically the sum of the preset QPS of the multiple target models deployed in multiple target pods after the target models are deployed in each of the multiple target pods.

[0080] The management server determines multiple target pods from the available pods based on the number of target pods. This can be done randomly or according to preset rules. For example, preset rules could be based on the order of pods' physical resources from highest to lowest.

[0081] The specific implementation of this step can be found in the subsequent description of the embodiments of this disclosure, and will not be repeated here.

[0082] S306. The management server processes the data to be processed based on multiple target pods and obtains the processing results.

[0083] As one possible implementation, the management server deploys the target model in each target pod and sends the data to be processed to the application server where each target pod resides. This allows the application server to process the data to be processed based on the target pod and the target model within it, obtain the processing result, and return the processing result to the management server.

[0084] The specific implementation of this step can be found in the subsequent description of the embodiments of this disclosure, and will not be repeated here.

[0085] S307. If the pod with the target model already deployed satisfies the data processing request, the management server processes the data to be processed based on the pod with the target model already deployed, and obtains the processing result.

[0086] As one possible implementation, the management server processes the data to be processed based on the target model in the pod where the target model has been deployed, and obtains the processing result.

[0087] Understandably, since the data processing system is built on a Kubernetes cluster, the management server can schedule and maintain multiple pods based on the distributed management features of the Kubernetes cluster. Therefore, when the total QPS of the target model deployed in multiple target pods is greater than or equal to the target QPS, the time consumed by multiple target pods to process the data will be less than the target duration, thus meeting the user's requirement for the target duration.

[0088] Meanwhile, since the target QPS is calculated based on the target duration, and the target QPS is used as the basis for pod scheduling, resources can be scheduled on demand during data processing, enabling precise scheduling of pods.

[0089] In some embodiments, such as Figure 4 As shown, this disclosure also provides a possible implementation of the above-mentioned S301, including the following steps:

[0090] S3011, Management Server Display User Interface.

[0091] The user interface includes a data upload box, a model selection box, a duration input box, and a request processing control.

[0092] As one possible implementation, the management server responds to access requests sent by the user's device through an application program interface (API) or a global wide area network (WAN) web interface, and displays a user interface on the user's device.

[0093] It should be noted that the data upload box is used to provide an interface for users to upload data to be processed.

[0094] After a user clicks the model selection box, an optional list will be displayed. This list provides the user with multiple model options and processing types. When multiple model options are displayed in the list, model-related information such as version information, size, update time, and the number of times it has been selected by other users can also be shown.

[0095] The duration input field provides an interface for users to input the target duration or target time.

[0096] Request processing controls are used to provide users with a final confirmation action. For example, a request processing control can be an OK button that the user can click.

[0097] S3012. The management server responds to user upload operations in the data upload box, user selection of target model in the model selection box, user input of target duration in the duration input box, and user triggering of request processing control, and obtains data processing requests.

[0098] As one possible implementation, after uploading the data to be processed in the data upload box, users can click the model selection box and choose the target model from the displayed list of options. Furthermore, users can also enter the target duration in the duration input box and click the request processing control.

[0099] In this scenario, the user's device acquires the data to be processed, the identifier of the target model, and the target duration, and generates a data processing request that includes the acquisition of the data to be processed, the identifier of the target model, and the target duration.

[0100] As mentioned above, in some cases, a data processing request may also include the data to be processed, the type of processing for the data to be processed, and the target time.

[0101] It is understood that a data processing request may include at least one of the above-mentioned processing type and target model identifier, and that a data processing request may include at least one of the above-mentioned target time and target duration.

[0102] Furthermore, the user equipment sends a data processing request to the management server based on the aforementioned API interface or web page interface. Correspondingly, the management server receives the data processing request sent by the user equipment.

[0103] Understandably, obtaining user data processing requests through a user interface offers a more convenient and simpler experience compared to existing technologies. It eliminates the need for users to input or select container URLs, and the visual interface allows for easier operation. Furthermore, users don't need to understand the internal resource configuration of the data processing system; they simply submit their requests to receive the results, thus improving the user experience.

[0104] In some embodiments, in order to determine whether a pod with a target model deployed satisfies the data processing request, the following are provided in this disclosure: the target model in the pod with the target model deployed is in an idle state, and the preset QPS of the target model is less than the target QPS.

[0105] Meanwhile, if a pod with the target model already deployed does not meet the data processing requirements, it also includes cases where the sum of the remaining usage time and the preset time is greater than the target time.

[0106] The remaining occupied time is the time required for the target model to process the remaining data when the target model in a pod that has already deployed the target model is in an occupied state. The preset time is the preset time required for the target model to process the data to be processed alone.

[0107] Specifically, such as Figure 5 As shown, S303 provided in this embodiment of the present disclosure specifically includes the following steps:

[0108] S3031. The management server determines whether the target model in a pod where the target model has been deployed is in an idle state.

[0109] As one possible implementation, the model information obtained by the management server from the application server also includes the current usage status of the target model, specifically whether the target model is in an idle state.

[0110] If the target model in a pod that has already been deployed is currently providing data processing services to other users, the management server determines that the target model in that pod is in an occupied state (also known as a non-idle state).

[0111] If the target model in a pod that has been deployed is not currently providing data processing services to task users, the management server determines that the target model in that pod is in an idle state.

[0112] For example, such as Figure 2 As shown, the pods with the target model deployed include pod1, pod2, and pod3. The image recognition model in pod1 is currently processing data for other users, so pod1 is in an occupied state. The image recognition models in pod2 and pod3 are not processing data, so pod2 and pod3 are in an idle state.

[0113] S3032. When the target model in a pod where the target model has been deployed is in an idle state, the management server determines whether the preset QPS of the target model in the pod where the target model has been deployed is less than the target QPS.

[0114] As one possible implementation, the management server, upon determining that the target model is not providing data processing services to any user, obtains the preset QPS of the target model and determines whether the preset QPS of the target model is less than the target QPS.

[0115] As another possible implementation, if the management server determines that the target model is not providing data processing services to any user, it obtains the target model's preset QPS and determines the preset time required for the target model to complete processing the model to be processed based on the preset QPS. Further, the management server compares the preset time with the target time. If the preset time is greater than the target time, the management server determines that the target model's preset QPS is less than the target QPS. If the preset time is less than or equal to the target time, the management server determines that the target model's preset QPS is greater than or equal to the target QPS.

[0116] The following section describes how the management server obtains the preset QPS of the target model:

[0117] In some cases, the management server can query the preset QPS corresponding to the target model's identifier from the second mapping relationship based on the target model's identifier. In the second mapping relationship, one model corresponds to one preset QPS, and the preset QPS is negatively correlated with the number of parameters and structural complexity of the target model. Structural complexity characterizes the complexity of the model's structure.

[0118] It should be noted that the second mapping relationship can be obtained by the operation and maintenance personnel of the data processing system or the management server through statistical analysis based on the actual operation of the model in the historical time period, and the settings can be stored in the management server in advance.

[0119] Understandably, the more parameters a model has, the lower its expected QPS will be. Similarly, the more layers and complex the connections in a model's network structure, the higher its structural complexity, and consequently, the lower its expected QPS.

[0120] S3033. If the preset QPS of the target model in a pod where the target model has been deployed is less than the target QPS, the management server determines that the pod where the target model has been deployed does not meet the data processing request.

[0121] As one possible implementation, if the management server determines that the preset QPS of the target model is less than the target QPS, or the preset duration is greater than the target duration, it determines that the pods with the target model already deployed do not meet the data processing requirements, that is, the pods with the target model already deployed cannot process the data to be processed within the target duration.

[0122] S3034. If the preset QPS of the target model in a pod that has already deployed the target model is greater than or equal to the target QPS, then it is determined that the pod that has already deployed the target model satisfies the data processing request.

[0123] As one possible implementation, if the management server determines that the preset QPS of the target model is greater than or equal to the target QPS, or the preset duration is less than or equal to the target duration, then the pods that have deployed the target model can satisfy the data processing request, that is, the pods that have deployed the target model can process the data to be processed within the target duration.

[0124] Understandably, considering that the target model is in an idle state, using the model's preset QPS as the basis for judgment is more comprehensive, the analysis basis is more accurate, and the judgment result is more accurate, which can further ensure that user needs can be met in the future.

[0125] S3035. When the target model in a pod where the target model has been deployed is in an occupied state, the management server determines the remaining occupied time.

[0126] As one possible implementation, after determining that the target model is providing data processing services to other users, the management server obtains the current remaining data of the current data being processed by the target model, and determines the remaining occupancy time of the occupied state based on the ratio of the size of the current remaining data to the target model's preset QPS.

[0127] In some embodiments, if there is other unprocessed data in the processing queue of the target model besides the current data, before the data to be processed, the management server shall also include the time required to process the other data in the remaining time when determining the remaining time.

[0128] S3036. The management server determines the preset time required for the target model to process the data to be processed individually, based on the preset QPS of the target model and the size of the data to be processed.

[0129] As one possible implementation, the management server determines the ratio of the size of the data to be processed to the QPS of the target model as the preset duration.

[0130] In practical applications, the management server can execute S3035 first and then S3036, or it can execute S3036 first and then S3035, or it can execute S3035 and S3036 simultaneously. This disclosure does not limit this.

[0131] S3037. The management server determines that the sum of the remaining occupied time and the preset time is less than the target time.

[0132] S3038. If the sum of the remaining usage time and the preset time is greater than the target time, the management server determines that the pod with the target model already deployed does not meet the data processing request.

[0133] S3039. If the sum of the remaining occupied time and the preset time is less than or equal to the target time, the management server determines that the pods with the target model deployed can meet the data processing requests.

[0134] Understandably, considering that the target model is in an idle state, the remaining usage time and the preset time are determined based on the model's preset QPS, and the sum of the remaining usage time and the preset time is used as the basis for judgment. This approach is more comprehensive, the analysis is more in line with reality, and the judgment results are more accurate, which can further ensure that user needs can be met in the future.

[0135] In some embodiments, in order to determine multiple target pods from idle pods based on a target QPS, the above-described S305 provided in this disclosure specifically includes the following steps:

[0136] S3051, The management server obtains the preset QPS of the target model.

[0137] For the specific implementation of this step, please refer to the content of S3032 in the above embodiment of this disclosure, which will not be repeated here.

[0138] S3052. The management server determines the target quantity based on the ratio of the target QPS to the preset QPS.

[0139] As one possible implementation, after the management server calculates the ratio of the target QPS to the preset QPS, it rounds the ratio up and uses the rounded integer as the target quantity.

[0140] S3053, The management server determines a target number of pods from the idle pods as multiple target pods.

[0141] As one possible implementation, the management server determines the idle pods from multiple pods, and then determines the target number of target pods from the idle pods.

[0142] In some cases, the aforementioned idle pods include a first pod that has not deployed the target model, and / or a second pod that has deployed the target model and the target model is in an idle state.

[0143] For example, such as Figure 2As shown, idle pods among multiple pods can include pod2, pod5, pod4, pod6, pod7, and pod8. pod4, pod6, pod7, and pod8 are the first pods where the target model is not deployed, while pod2 and pod5 are the second pods where the target model is deployed and is currently idle. If the number of targets is 2, the management server can directly designate pod2 and pod5 as multiple target pods. If the number of targets is 3, the management server can designate pod2, pod5 from the first pod, and any one of the second pods as multiple target pods. Alternatively, the management server can designate any three pods from the second pod as multiple target pods.

[0144] Understandably, using the ratio of the target QPS to the preset QPS to determine the number of target pods firstly ensures the accuracy of the target number of pods. Secondly, for the target model, on-demand scheduling can be achieved while ensuring that user needs are met, and resources can be used rationally.

[0145] Meanwhile, when identifying idle pods, the first pod of the undeployed model and the second pod of the deployed target model are taken into account, making the identified idle pods more comprehensive. This enables the rational use of current resources and the rational scheduling of pods.

[0146] In some embodiments, since the determined target number may be greater than the number of idle pods, it is necessary to scale up multiple pods in the data processing system. To enable scaling up multiple pods, the data processing method provided in this disclosure further includes the following steps:

[0147] S401, The management server obtains the target physical resources corresponding to the target model.

[0148] The target physical resources include the number of target CPUs, the amount of target memory, and the number of target GPUs.

[0149] As one possible implementation, the management server retrieves the target physical resources from a third mapping relationship based on the identifier of the target model.

[0150] The third mapping relationship includes the correspondence between model identifiers and physical resources. The target physical resources are positively correlated with the number of parameters and structural complexity of the target model. That is, the larger the number of parameters in the target model, the more target physical resources need to be configured for the target pod. Similarly, the greater the structural complexity of the target model, the more target physical resources need to be configured for the target pod.

[0151] The third mapping relationship can be obtained by operations and maintenance personnel or management servers based on the model's operation status over a historical period, by statistically analyzing the physical resources used, and then stored and maintained in the management server.

[0152] S402. The management server creates an idle pod in the application server based on the target physical resources.

[0153] One possible implementation is that the management server controls the application server through the APIServer component, creating new idle pods within the application server. Correspondingly, the application server creates the idle pods based on the Kubelet component and allocates the appropriate target physical resources to them.

[0154] In some cases, to ensure the rational use of physical resources, application servers can release pods whose physical resources are below a preset threshold and use the released physical resources to create new idle pods. For example, an application server can release two pods and merge them into a single idle pod that meets the target physical resource requirement.

[0155] Understandably, the physical resources of each target pod determined by the management server are greater than or equal to the target physical resources.

[0156] In some embodiments, since there are first pods among multiple target pods that have not yet deployed the target model, it is necessary to deploy the target model in the first pods. To automate the deployment of the target model in pods, such as... Figure 2 As shown, the data processing system 20 also includes a mirror repository 205, which stores mirror data of each model, as well as target mirror data of the target model.

[0157] like Figure 6 As shown, prior to S306 in the embodiments of this disclosure, the method further includes the following steps:

[0158] S501. When there is a first pod in multiple target pods where the target model has not been deployed, the management server controls the application server where the first pod is located to execute the model scheduling task.

[0159] The model scheduling task includes retrieving target image data from the image repository and deploying the target model in the first pod based on the target image data.

[0160] As one possible implementation, when the management server determines that the first pod exists among multiple target pods, it sends a model scheduling instruction to the application server where the first pod resides via the APIServer component. Correspondingly, upon receiving the model scheduling instruction, the application server executes the model scheduling task within it. This includes: retrieving the target image data corresponding to the target model from the image repository using the application server's Kubelet component; and, after retrieving the target image data, running the target image data in the first pod to deploy the target model within the first pod.

[0161] In practical applications, when the APIServer component and the management server are configured relatively independently, the management server also includes a model scheduler. The model scheduler communicates with the APIServer component to control the execution of model scheduling tasks by each application server.

[0162] In one scenario, the model scheduler can also be configured relatively independently of the management server. In this case, the communication process between the model scheduler and the management server is the same as the communication process between the various modules within the management server.

[0163] Understandably, based on the distributed management characteristics of Kubernetes clusters, a model scheduling method based on Kubernetes is proposed, which can automatically deploy models in each pod. Compared with the manual operation by operation and maintenance personnel in the existing technology, this greatly improves the operation and maintenance efficiency of the data processing system.

[0164] In some cases, since the management server also maintains task queues for executing different model scheduling tasks, the data processing method provided in this disclosure embodiment further includes:

[0165] If there is a preset task in the task queue of the management server that precedes the model scheduling task, the management server will control the application server where the first pod is located to start executing the model scheduling task after the preset task is executed.

[0166] In addition, the management server can assign different priorities to different users and models.

[0167] For example, the user who initiates the data processing request has a higher priority than the user who initiates the access request. If the user interface display task is included before the model scheduling task in the task queue, the management server can execute the model scheduling task first and then execute the user interface display task.

[0168] For example, if the priority of an image recognition model is higher than that of a data classification model, and if the model scheduling task for the image recognition model is preceded by the model scheduling task for the data classification model in the task queue, then the management server can execute the model scheduling task for the image recognition model first, and then execute the model scheduling task for the data classification model.

[0169] Understandably, queue management is implemented in practical applications, providing a specific implementation method for model scheduling.

[0170] In some embodiments, in order to enable parallel processing of data to be processed based on multiple target pods, such as Figure 7 As shown, the above-described S306 of this embodiment includes the following steps:

[0171] S3061. The management server segments the data to be processed based on the target quantity, resulting in multiple data segments to be processed.

[0172] Each data segment to be processed is of the same size, and the number of data segments to be processed is an integer multiple of the number of target pods.

[0173] S3062. The management server sends multiple data segments to be processed to the application server where the corresponding target pod is located, so that the application server controls the target pod to process the data segments to be processed based on the target model and obtain the sub-processing results.

[0174] One possible implementation is that the management server allocates a corresponding data segment to be processed for each target pod and distributes the data segment to the application server where the target pod resides. Upon receiving the data segment, the application server controls the target pod to process the data segment using the Kubelet component, obtains a sub-processing result, and sends the sub-processing result to the management server.

[0175] For example, with Figure 2 Taking pod2 and pod7 as target pods as an example, the management server can divide the data to be processed into 4 data segments according to the data size, and send 2 data segments to pod2 and pod7 respectively, so that pod2 and pod7 can process the corresponding 2 data segments.

[0176] In practical applications, the management server sends data segments to be processed to the target pod in the application server. This can be done by polling or by sending all data at once. This embodiment does not limit the method.

[0177] S3063. The management server receives the sub-processing results sent by the application server, and merges the received sub-processing results to obtain the processing result.

[0178] Understandably, by adopting the above technical features, it is possible to process the data to be processed in parallel based on multiple target pods, thereby ensuring that the data to be processed is completed within the target time period to meet user needs.

[0179] In some embodiments, after obtaining the processing result, in order to improve resource utilization, such as... Figure 6 As shown in the embodiments of this disclosure, the data processing method further includes the following steps:

[0180] S601: After the management server processes the data to be processed based on multiple target pods and obtains the processing results, it releases the target models in the multiple target pods.

[0181] As one possible implementation, the management server can control the application server where the target pod resides to release the target model based on the APIServer component.

[0182] In practical applications, the management server can also control the application server to release the target pod after the target model is released, in order to reclaim more physical resources.

[0183] Understandably, releasing the model within each pod in a Kubernetes cluster after it has finished processing data provides more free pods for subsequent data processing services, thus improving the resource utilization of the Kubernetes cluster.

[0184] Figure 8 This is a structural block diagram of a data processing apparatus according to an embodiment of the present disclosure. The apparatus is used as a management server in a data processing system. The data processing system is built on a Kubernetes cluster and also includes multiple minimum scheduling units (pods). See also... Figure 8 The device includes an acquisition unit 701, a determination unit 702, and a processing unit 703.

[0185] The acquisition unit 701 is used to acquire the user's data processing request. The data processing request is used to request that the data to be processed be completed based on the target model within a target time period.

[0186] Unit 702 is used to determine the target query rate per second (QPS) for the data processing request based on the target duration and the size of the data to be processed, when the pod with the target model already deployed does not meet the data processing request.

[0187] Unit 702 is also used to determine multiple target pods from the idle pods of multiple pods based on the target QPS. These multiple target pods are used to deploy the target model, and the total QPS of the target model corresponding to the multiple target pods is greater than or equal to the target QPS.

[0188] Processing unit 703 is used to process data to be processed based on multiple target pods and obtain processing results.

[0189] In one possible implementation, such as Figure 8 As shown, the determining unit 702 is specifically used for:

[0190] Get the preset QPS of the target model.

[0191] The target quantity is determined based on the ratio of the target QPS to the preset QPS.

[0192] Determine the target number of pods from the idle pods and use them as multiple target pods.

[0193] In one possible implementation, an idle pod includes a first pod that has not deployed the target model, and / or a second pod that has deployed the target model and the target model is in an idle state.

[0194] In one possible implementation, the data processing system also includes an application server and an image repository. The application server is used to deploy pods, and the image repository stores target image data of the target model.

[0195] Processing unit 703 is also used for:

[0196] If a first pod among multiple target pods does not have the target model deployed, the application server hosting the first pod is controlled to execute a model scheduling task. The model scheduling task includes retrieving target image data from the image repository and deploying the target model in the first pod based on the target image data.

[0197] In one possible implementation, the processing unit 703 is further configured to: if there is a preset task in the task queue of the management server that precedes the model scheduling task, then after the preset task is executed, control the application server where the first pod is located to start executing the model scheduling task.

[0198] In one possible implementation, the pod that has deployed the target model does not satisfy the data processing request, including: the target model in the pod that has deployed the target model is in an idle state, and the preset QPS of the target model is less than the target QPS.

[0199] In one possible implementation, the pod with the target model already deployed does not meet the data processing request, including: the sum of the remaining occupied time and the preset time is greater than the target time. The remaining occupied time is the time required for the target model to process the remaining data when the target model in the pod with the target model deployed is in an occupied state. The preset time is the preset time required for the target model to process the data to be processed alone.

[0200] In one possible implementation, the processing unit 703 is further configured to: after processing the data to be processed based on multiple target pods and obtaining the processing results, release the target models in the multiple target pods.

[0201] In one possible implementation, the data processing system also includes an application server, which is used to deploy pods.

[0202] Processing unit 703 is specifically used for:

[0203] Based on the target quantity, the data to be processed is segmented into multiple data segments.

[0204] Multiple data segments to be processed are sent to the application server where the corresponding target pod is located, so that the application server controls the target pod to process the data segments to be processed based on the target model and obtain the sub-processing results.

[0205] Receive the sub-processing results sent by the application server, and merge the received sub-processing results to obtain the final processing result.

[0206] In one possible implementation, the acquisition unit 701 is specifically used for:

[0207] The user interface includes a data upload box, a model selection box, a duration input box, and a request processing control.

[0208] In response to user upload operations (upload data to be processed in the data upload box), user selection of target model in the model selection box, user input of target duration in the duration input box, and user triggering of request processing control, data processing requests are obtained.

[0209] According to embodiments of this disclosure, an electronic device is also provided, including at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the data processing method provided in this disclosure.

[0210] According to embodiments of the present disclosure, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the data processing method provided in the present disclosure.

[0211] According to embodiments of this disclosure, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the data processing method provided in this disclosure.

[0212] Figure 9 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0213] like Figure 9 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0214] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as a keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as a disk, optical disk, etc.; and communication unit 809, such as a network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0215] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform data processing methods by any other suitable means (e.g., by means of firmware).

[0216] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0217] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0218] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0219] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user. For example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback). Input from the user can be received in any form (including sound input, voice input, or tactile input).

[0220] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0221] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0222] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0223] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data processing method applied to a management server in a data processing system; the data processing system is built on a Kubernetes cluster, and further includes multiple minimum scheduling units (pods); the data processing system also includes an application server and an image repository, the application server being used to deploy the pods, and the image repository storing target image data of the target model; the method includes: Obtain user data processing requests; The data processing request is used to request that the data to be processed be completed based on the target model within the target time period; If the pods that have deployed the target model do not meet the data processing requests, the target query rate per second (QPS) is determined based on the target duration and the size of the data to be processed. Based on the target QPS, multiple target pods are determined from the idle pods of the multiple pods; the multiple target pods are used to deploy the target model, and the total QPS of the target model corresponding to the multiple target pods is greater than or equal to the target QPS; The idle pod includes a first pod that has not deployed the target model, and / or a second pod that has deployed the target model and the target model is in an idle state; If there is a first pod among the multiple target pods that has not deployed the target model, the application server where the first pod is located is controlled to execute a model scheduling task; the model scheduling task includes obtaining the target image data from the image repository, and deploying the target model in the first pod based on the target image data; Based on the target quantity, the data to be processed is segmented to obtain multiple data segments to be processed; The multiple data segments to be processed are sent to the application server where the corresponding target pod is located, so that the application server controls the target pod to process the data segments to be processed based on the target model to obtain sub-processing results; The system receives the sub-processing results sent by the application server and merges the received sub-processing results to obtain the processing result.

2. The data processing method according to claim 1, wherein, The step of determining multiple target pods from the idle pods of the multiple pods based on the target QPS includes: Obtain the preset QPS of the target model; The target quantity is determined based on the ratio of the target QPS to the preset QPS; The target number of pods are determined from the idle pods as the plurality of target pods.

3. The data processing method according to claim 1, wherein, The step of controlling the application server where the first pod resides to execute the model scheduling task includes: If there is a preset task in the task queue of the management server that precedes the model scheduling task, then after the preset task is executed, the application server where the first pod is located is controlled to start executing the model scheduling task.

4. The data processing method according to any one of claims 1-3, wherein, The fact that a pod with the target model already deployed does not satisfy the data processing request includes: the target model in the pod with the target model already deployed is in an idle state, and the preset QPS of the target model is less than the target QPS.

5. The data processing method according to any one of claims 1-3, wherein, The pod that has deployed the target model does not meet the data processing request, including: the sum of the remaining occupied time and the preset time is greater than the target time; wherein, the remaining occupied time is the time required for the target model to process the remaining data when the target model in the pod that has deployed the target model is in an occupied state; the preset time is the preset time required for the target model to process the data to be processed alone.

6. The data processing method according to any one of claims 1-3, further comprising: After processing the data to be processed based on the multiple target pods and obtaining the processing result, the target model in the multiple target pods is released.

7. The data processing method according to any one of claims 1-3, wherein, The process of obtaining user data processing requests includes: The user interface includes a data upload box, a model selection box, a duration input box, and a request processing control. In response to the user's upload operation of uploading the data to be processed in the data upload box, the user's selection operation of the target model in the model selection box, the user's input operation of the target duration in the duration input box, and the user's trigger operation of the request processing control, the data processing request is obtained.

8. A data processing apparatus, applied to a management server in a data processing system; the data processing system is built on a Kubernetes cluster, and the data processing system further includes multiple minimum scheduling units (pods); the data processing system further includes an application server and an image repository, the application server being used to deploy the pods, and the image repository storing target image data of a target model; the apparatus includes an acquisition unit, a determination unit, and a processing unit. The acquisition unit is used to acquire the user's data processing request; the data processing request is used to request that the data to be processed be completed based on the target model within a target time period; The determining unit is used to determine the target query rate per second (qps) corresponding to the data processing request based on the target duration and the size of the data to be processed, when the pod with the target model already deployed does not meet the data processing request. The determining unit is further configured to determine multiple target pods from the idle pods of the multiple pods based on the target QPS; the multiple target pods are used to deploy the target model, and the total QPS of the target model corresponding to the multiple target pods is greater than or equal to the target QPS; The idle pod includes a first pod that has not deployed the target model, and / or a second pod that has deployed the target model and the target model is in an idle state; The processing unit is configured to, when there is a first pod among the plurality of target pods where the target model is not deployed, control the application server where the first pod is located to execute a model scheduling task; the model scheduling task includes obtaining the target image data from the image repository, and deploying the target model in the first pod based on the target image data; The processing unit is also used to segment the data to be processed based on the target quantity to obtain multiple data segments to be processed. The multiple data segments to be processed are sent to the application server where the corresponding target pod is located, so that the application server controls the target pod to process the data segments to be processed based on the target model to obtain sub-processing results; the sub-processing results sent by the application server are received and merged to obtain the processing result.

9. The data processing apparatus according to claim 8, wherein, The determining unit is specifically used for: Obtain the preset QPS of the target model; The target quantity is determined based on the ratio of the target QPS to the preset QPS; The target number of pods are determined from the idle pods as the plurality of target pods.

10. The data processing apparatus according to claim 8, wherein, The processing unit is further configured to: If there is a preset task in the task queue of the management server that precedes the model scheduling task, then after the preset task is executed, the application server where the first pod is located is controlled to start executing the model scheduling task.

11. The data processing apparatus according to any one of claims 8-10, wherein, The fact that a pod with the target model already deployed does not satisfy the data processing request includes: the target model in the pod with the target model already deployed is in an idle state, and the preset QPS of the target model is less than the target QPS.

12. The data processing apparatus according to any one of claims 8-10, wherein, The pod that has deployed the target model does not meet the data processing request, including: the sum of the remaining occupied time and the preset time is greater than the target time; wherein, the remaining occupied time is the time required for the target model to process the remaining data when the target model in the pod that has deployed the target model is in an occupied state; the preset time is the preset time required for the target model to process the data to be processed alone.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images