Method, device and equipment for online identification service and timely response, and storage medium

Abstract

The application relates to an online identification service and timely response method, device, equipment and storage medium, and belongs to the technical field of machine learning. The application comprises the following steps: acquiring user demand of an online identification service, the user demand comprising a model, data and a longest waiting time of the online identification service; determining a self-defined standard of the online identification service according to the user demand; checking the state of each service instance of the online identification service to obtain a checking result; and within the longest waiting time, judging whether the checking result meets the self-defined standard, and if yes, adding each service instance of the online identification service to load balancing to provide the online identification service. Through the application, the problem that each algorithm service in the online identification service cannot perceive whether the model or data is ready can be solved, once the instance that is not ready is added to the load balancing and is called by an external service, an exception will occur, thereby affecting service reliability and stability of the service capability provided to the outside.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology, specifically relating to methods, apparatus, devices, and storage media for timely response to online recognition services. Background Technology

[0002] With the rise of machine learning and deep learning technologies, their applications are becoming increasingly widespread, and online machine learning recognition services are being applied in various technical fields. 。 For example, in artificial intelligence applications, current machine learning frameworks on the market, such as Zhejiang Lab's open-source Zhijiang Tianchu and Tencent Music's open-source Cube Studio, are all based on Kubernetes (a container cluster management system) to manage cluster resources (CPU / GPU / memory). When deploying online machine learning recognition services, users deploy multiple instances of various machine learning algorithm services through Kubernetes resource Deployments, and request CPU / GPU resources from machines through the resource request / limit in the spec. Then, under Kubernetes scheduling, the instances are distributed to resource nodes that meet the user's needs.

[0003] In the logistics and express delivery industry, when predicting order volume or identifying suspicious packages, the special nature of machine learning algorithms and the large amount of order and package data mean that most algorithm services require a long time to load models and training data during startup and need frequent updates and upgrades. Currently, in online identification services, Kubernetes uses container technology for resource management, mechanically determining whether to add instances to the load balancer based on the container's startup status. It cannot detect whether the models or data of the various machine learning algorithms in the online identification service are ready and available. If an unready instance is added to the load balancer, anomalies will occur when that instance is called by external services, affecting the normal operation of the online identification service and thus impacting its reliability and the stability of its service capabilities. Summary of the Invention

[0004] To address this issue, this application provides a method, apparatus, device, and storage medium for timely response of online identification services. This helps to solve the problem of not being able to perceive whether the models or data of various machine learning algorithm services in the online identification service are ready and available. Once an unready instance is added to the load balancer, an anomaly will occur when the instance is called by external services, affecting the normal operation of the online identification service and thus affecting the service reliability and stability of the service capability to provide services to the outside world.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, this application provides a method for timely response to online identification services, including:

[0007] The user requirements for obtaining online recognition services include the model, data, and maximum waiting time of the online recognition services.

[0008] The custom criteria for the online recognition service are determined based on the user requirements.

[0009] The status of each service instance of the online identification service is verified to obtain the verification result;

[0010] During the longest waiting time, it is determined whether the verification result meets the custom standard. If it does, each service instance of the online identification service is added to the load balancer to provide the online identification service.

[0011] Furthermore, the status of each service instance of the online identification service is verified to obtain the verification result, including:

[0012] The status of the data loading is verified. If the data loading is completed, the verification result is determined to be that the service instance of the data is ready.

[0013] The loading status of the model is verified. If the model loading is complete, the verification result is determined to be that the service instance of the model is ready.

[0014] Furthermore, the method also includes:

[0015] If the data service instance or the model service instance is not ready within the maximum waiting time, the status of each service instance of the online identification service will be re-verified until the maximum waiting time is exceeded.

[0016] Furthermore, after determining the custom criteria for the online recognition service based on the user requirements, and before verifying the status of each service instance of the online recognition service, the method further includes:

[0017] The service and instance are deployed through the container cluster management system, the corresponding machine cluster and instance for the online identification service are determined, and then started.

[0018] Furthermore, any of the methods described above also includes:

[0019] Check memory resources and disk resources separately, and determine the results;

[0020] If the inspection results meet the custom criteria of the online identification service, each service instance of the online identification service will be added to the load balancer to provide the online identification service.

[0021] Secondly, this application provides an apparatus for timely response to online identification services, comprising:

[0022] The data acquisition module is used to acquire user requirements for the online recognition service, including the model, data, and maximum waiting time of the online recognition service; and to determine the custom criteria for the online recognition service based on the user requirements.

[0023] The status verification module is used to verify the status of each service instance of the online identification service and obtain the verification result.

[0024] The execution module is used to determine whether the verification result meets the custom standard. If it does, the service instances of the online recognition service are added to the load balancer to provide the online recognition service.

[0025] Furthermore, the status verification module is also used to verify the status of data loading. If the data loading is completed, the verification result is that the service instance of the data is ready. The module also verifies the status of model loading. If the model loading is completed, the verification result is that the service instance of the model is ready.

[0026] Furthermore, the device also includes:

[0027] If the data service instance or the model service instance is not ready within the maximum waiting time, the status of each service instance of the online identification service will be re-verified until the maximum waiting time is exceeded.

[0028] Thirdly, this application provides a device for timely response to online identification services, comprising:

[0029] Memory, on which executable programs are stored;

[0030] A processor for executing the executable program in the memory to implement the steps of any of the methods described above.

[0031] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the steps of any of the methods described above.

[0032] The application employs the above technical solution and has at least the following beneficial effects:

[0033] By acquiring user requirements for the online recognition service, custom criteria for the service are determined. The status of each service instance is verified, and only when the custom criteria are met are the service instances added to the load balancer to provide the online recognition service. This user-defined standard allows for a more accurate assessment of the readiness status of each service instance, ensuring that only ready and usable instances are added to the load balancer, thus reducing anomalies. This helps address the issue of not being able to detect the readiness of models or data for machine learning algorithms within the online recognition service. If an unready instance is added to the load balancer, anomalies will occur when that instance is called by external services, affecting the normal operation of the online recognition service and impacting its reliability and stability.

[0034] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating a method for timely response of an online identification service according to an exemplary embodiment;

[0037] Figure 2 This is a block diagram illustrating an apparatus for timely response to an online identification service according to an exemplary embodiment. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be described in detail below. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for timely response of an online identification service according to an exemplary embodiment, such as... Figure 1 As shown, the method includes the following steps:

[0040] Step S11: Obtain user requirements for the online recognition service, including the model, data, and maximum waiting time of the online recognition service;

[0041] Step S12: Determine the custom standard for the online identification service based on the user requirements;

[0042] Step S13: Verify the status of each service instance of the online identification service to obtain the verification result;

[0043] Step S14: During the longest waiting time, determine whether the verification result meets the custom standard. If it does, add each service instance of the online identification service to the load balancer to provide the online identification service.

[0044] It should be noted that the online identification service timely response method provided by the present invention can be applied to various scenarios, such as prediction of logistics delivery time, prediction of logistics service quality, image recognition, order volume prediction, identification of suspicious packages, etc. The present invention does not make specific limitations in these areas.

[0045] It should be noted that when using machine learning models to provide online identification services to users, the loading of the model and data may prevent normal service from being provided during this period. Accessing the service during this time will result in anomalies. Furthermore, in the logistics industry, order numbers and data are constantly updated, and user-demanded functionalities also need to be added frequently. Consequently, the quality and functionality of the online identification service also require frequent updates. Accessing the online identification service during update periods may also result in common anomalies such as error messages, inability to open the corresponding webpage, or being stuck in a waiting state. After these anomalies occur, subsequent customer access to the online identification service will experience a large number of service request anomalies.

[0046] Specifically, user-submitted online identification services typically include user requirements. For example, a user-submitted online suspicious package identification service might include the model, data, and maximum waiting time required to perform the service. The required model includes the version of the model used to perform the service, its parameter settings, and other model-related information. It may also recommend suitable models based on the user's submitted request. The required data includes the experimental data selected by the user for performing the service and all data included in the selected criteria. This data must be in a format compatible with the model. The maximum waiting time is the longest time the user can tolerate from submitting the service request to receiving the result. If this maximum waiting time is exceeded, the user may lose patience and stop waiting for the result.

[0047] It should be noted that user requirements for online recognition services may also include other aspects, such as external dependencies and external services required to execute the online recognition service, the directory where users need to save the results of the online recognition service, etc. This disclosure does not limit these aspects.

[0048] Custom standards for online recognition services are determined based on user needs. These standards include ensuring that models that meet user needs are ready and loaded into a specific directory, that data that meets user needs and is ready for model operation is loaded into the corresponding directory, and that all other necessary external dependencies and services are ready.

[0049] Furthermore, after determining the custom criteria for the online identification service based on the user requirements, the technical solution provided in this embodiment also includes:

[0050] The service and instance are deployed through the container cluster management system, the corresponding machine cluster and instance for the online identification service are determined, and the corresponding resources are started.

[0051] Specifically, by using a container cluster management system, the online identification service can be deployed and instances can be deployed. Based on the user requirements of the online identification service, the required machine clusters and instances can be determined.

[0052] Understandably, different service instances are determined based on different user needs. Since the service instances that can run independently on each container are also different, the container that runs the instance can be determined through instance deployment, thereby starting the corresponding container, obtaining the corresponding GPU resources, and realizing the effective scheduling of computing resources.

[0053] Furthermore, the status of each service instance of the online identification service is verified to obtain the verification results, including:

[0054] The status of the data loading is verified. If the data loading is completed, the verification result is determined to be that the service instance of the data is ready.

[0055] The loading status of the model is verified. If the model loading is complete, the verification result is determined to be that the service instance of the model is ready.

[0056] Specifically, a custom readiness probe interface is used to verify the data loading status. If all data meeting user requirements has been loaded and placed in the corresponding directories, the verification result indicates that the data service instance is ready. If the data meeting user requirements has not been loaded or has not been placed in the corresponding directories, the verification result indicates that the data service instance is not ready.

[0057] Using a custom readiness probe interface, the loading status of the aforementioned models is verified. If the model that meets the user's requirements has been successfully loaded and placed in the specified directory, the verification result indicates that the model's service instance is ready. If the model that meets the user's requirements has not been successfully loaded, or has not been placed in the specified directory, the verification result indicates that the model's service instance is not ready.

[0058] During the longest waiting time, it is determined whether the above data verification results and model verification results meet the custom criteria. That is, if the service instances of the data and the service instances of the model are ready and meet the custom criteria, it means that the preparation work of the online recognition service has been completed and can be called. At this time, each service instance of the online recognition service is added to the load balancer to provide the online recognition service to the user.

[0059] Furthermore, if the data service instance or the model service instance is not ready within the longest waiting time, the status of each service instance of the online identification service is re-verified until the longest waiting time is exceeded.

[0060] Specifically, within the maximum waiting time, a custom interface is used to verify the loading status of the model and data. If the verification result indicates that the service instance for the model or the service instance for the data is not ready, it means that the model or data has not been fully loaded. In this case, adding each service instance to the load balancer will result in an anomaly. To prevent abnormal access, the status of each service instance of the online suspicious package identification service is re-verified until the maximum waiting time is exceeded. At this point, the identification task ends, and re-verification is no longer performed. The time interval for re-verification is not specifically limited in this example.

[0061] Understandably, by acquiring user requirements for the online recognition service and determining custom standards for the service based on those requirements, the status of each service instance is verified. Only when the custom standards are met are the service instances added to the load balancer to provide the online recognition service. This process, entirely determined by user-defined standards, allows for a more accurate assessment of the readiness status of each service instance, ensuring that only ready and usable instances are added to the load balancer, thus reducing the occurrence of anomalies. This helps address the issue of not being able to detect whether the models or data of the machine learning algorithms in the online recognition service are ready and usable. If an unready instance is added to the load balancer, anomalies will occur when that instance is called by external services, affecting the normal operation of the online recognition service and consequently impacting service reliability and the stability of its service capabilities.

[0062] In another embodiment, the technical solution provided in this embodiment further includes:

[0063] Check memory resources and disk resources separately, and determine the results;

[0064] If the inspection results meet the custom criteria of the online identification service, each service instance of the online identification service will be added to the load balancer to provide the online identification service.

[0065] Specifically, the memory and disk resources used for providing the online suspicious package identification service are checked separately to see if they meet the custom criteria for executing the service. If both memory and disk resources meet the criteria, the service instances for the online suspicious package identification service are added to the load balancer to provide the service. If either memory or disk resources do not meet the criteria, the service instances are not added to the load balancer, and access to the online identification service is terminated.

[0066] Understandably, by checking memory and disk resources, it can be determined whether the online identification service has an executable environment. Only after the execution environment is met can the service instances of the online identification service be added to the load balancer for operation. This ensures that the services added to the load balancer are executable, thereby reducing the occurrence of anomalies.

[0067] This application also provides a device for timely response to online identification services; please refer to [link / reference]. Figure 2 , Figure 2 This is a block diagram illustrating an apparatus for timely response to an online identification service according to an exemplary embodiment, comprising:

[0068] Data acquisition module 21 is used to acquire user requirements for the online recognition service, including the model, data, and maximum waiting time of the online recognition service; and to determine the custom criteria for the online recognition service based on the user requirements.

[0069] The status verification module 22 is used to verify the status of each service instance of the online identification service and obtain the verification result;

[0070] The execution module 23 is used to determine whether the verification result meets the custom standard. If it does, it adds each service instance of the online identification service to the load balancer to provide the online identification service.

[0071] The status verification module 22 is also used to verify the status of the data loading. If the data loading is completed, the verification result is that the service instance of the data is ready. The module also verifies the status of the model loading. If the model loading is completed, the verification result is that the service instance of the model is ready.

[0072] Furthermore, if the service instance of the data or the service instance of the model is not ready within the maximum waiting time, the status of each service instance of the online identification service is re-verified until the maximum waiting time is exceeded.

[0073] Furthermore, after determining the custom criteria for the online recognition service based on the user requirements, and before verifying the status of each service instance of the online recognition service, the above method further includes:

[0074] The service and instance are deployed through the container cluster management system, the corresponding machine cluster and instance for the online identification service are determined, and then started.

[0075] Furthermore, any of the above methods also includes:

[0076] Check memory resources and disk resources separately, and determine the results;

[0077] If the inspection results meet the custom criteria of the online identification service, each service instance of the online identification service will be added to the load balancer to provide the online identification service.

[0078] It is understood that the technical solution provided in this embodiment, by obtaining user requirements for online recognition services as mentioned in the above embodiments, determining custom standards for online recognition services based on user requirements, and verifying the status of each service instance of the online recognition service, only adding each service instance of the online recognition service to the load balancer when the custom standards are met, provides online recognition services. This is entirely determined by the user's custom judgment criteria, enabling a more accurate assessment of the readiness status of each service instance. This ensures that only service instances added to the load balancer are ready and available, reducing the occurrence of anomalies. This helps solve the problem of not being able to perceive whether the models or data of various machine learning algorithm services are ready and available. Once an unready instance is added to the load balancer, anomalies will occur when that instance is called by external services, affecting the normal operation of other businesses, thereby impacting service reliability and the stability of the ability to provide services to the outside world.

[0079] This application also provides a device for timely response to online identification services, including:

[0080] Memory, on which executable programs are stored;

[0081] A processor for executing the executable program in the memory to implement the steps of any of the methods described above.

[0082] Furthermore, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the steps of any of the methods described above. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0083] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0084] It should be noted that in the description of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "multiple" or "more" means at least two.

[0085] It should be understood that when an element is referred to as "fixed to" or "set on" another element, it may be directly on the other element or may have an intervening element present at the same time; when an element is referred to as "connected to" another element, it may be directly connected to the other element or may have an intervening element present at the same time. In addition, the term "connected" as used herein may include wireless connections; the word "and / or" as used includes any unit and all combinations of one or more of the associated listed items.

[0086] Any process or method description in the flowchart or otherwise herein can be understood as: representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0087] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0088] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0090] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0091] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0092] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for timely response to online identification services, characterized in that, include: The user requirements for obtaining online recognition services include the model, data, and maximum waiting time of the online recognition services. The custom criteria for the online recognition service are determined based on the user requirements. The service and instance are deployed through the container cluster management system, the corresponding machine cluster and instance for the online identification service are determined, and then started. The status of each service instance of the online recognition service is verified to obtain a verification result, including verifying the status of data loading. If the data loading is completed, the verification result is determined to be that the service instance of the data is ready. The status of model loading is verified. If the model loading is completed, the verification result is determined to be that the service instance of the model is ready. During the longest waiting time, it is determined whether the verification result meets the custom standard. If it does, each service instance of the online identification service is added to the load balancer to provide the online identification service.

2. The method according to claim 1, characterized in that, Also includes: If the data service instance or the model service instance is not ready within the maximum waiting time, the status of each service instance of the online identification service will be re-verified until the maximum waiting time is exceeded.

3. The method according to any one of claims 1 to 2, characterized in that, Also includes: Check memory resources and disk resources separately, and determine the results; If the inspection results meet the custom criteria of the online identification service, each service instance of the online identification service will be added to the load balancer to provide the online identification service.

4. A device for timely response to online identification services, characterized in that, The device includes: The data acquisition module is used to acquire user requirements for the online recognition service, including the model, data, and maximum waiting time of the online recognition service; determine the custom standards for the online recognition service based on the user requirements; and deploy the service and instances through the container cluster management system to determine the corresponding machine clusters and instances for the online recognition service and start them. The status verification module is used to verify the status of each service instance of the online recognition service and obtain the verification result; it is also used to verify the status of data loading, and if the data loading is completed, the verification result is that the service instance of the data is ready; and to verify the status of model loading, and if the model loading is completed, the verification result is that the service instance of the model is ready. The execution module is used to determine whether the verification result meets the custom standard. If it does, the service instances of the online recognition service are added to the load balancer to provide the online recognition service.

5. The apparatus according to claim 4, characterized in that, Also includes: If the data service instance or the model service instance is not ready within the maximum waiting time, the status of each service instance of the online identification service will be re-verified until the maximum waiting time is exceeded.

6. A device for timely response to online identification services, characterized in that, include: Memory, on which executable programs are stored; A processor for executing the executable program in the memory to implement the steps of the method according to any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the steps of the method according to any one of claims 1 to 3.

Images