Test method and device of operator service and electronic equipment

Abstract

The present disclosure provides a test method and device of operator service and electronic equipment, relates to the technical field of artificial intelligence, in particular to the technical field of machine learning, data processing, intelligent testing and the like. The specific implementation scheme is: obtaining an operator service to be tested; deploying the operator service and a domain service in a container; testing the operator service based on the domain service and test data in the container according to a business processing flow, and obtaining a corresponding test result; wherein the domain service is configured to invoke the operator service to process the test data according to a preset processing flow in the process of testing the operator service. Thus, the operator service can be automatically deployed in the container through the virtualization container technology, the deployment mode of the operator service is simple and fast, and the operator service can be tested without being deployed in a production environment, thereby saving the time of deploying the operator service in the production environment and improving the test efficiency of the operator service.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical fields of machine learning, data processing, intelligent testing and the like, and in particular to a method and apparatus for testing an operator service and an electronic device. BACKGROUND

[0002] An operator service is an encapsulation of an operator into an independent service entity, so that the operator can exist in the form of a standardized and directly callable service for convenient access and use by external systems or applications. During the development process or the iterative process after deployment in a production environment, the operator service needs to be tested to determine whether the operator service meets the standard for use in the production environment. How to achieve efficient testing of the operator service is a problem to be solved. SUMMARY

[0003] The present disclosure provides a method and apparatus for testing an operator service and an electronic device.

[0004] According to an aspect of the present disclosure, a method for testing an operator service is provided, the method comprising: obtaining an operator service to be tested; deploying the operator service and a domain service in a container; testing the operator service based on the domain service and test data in the container to obtain a corresponding test result; wherein the domain service is configured to invoke the operator service to process the test data according to a preset processing flow during the testing of the operator service.

[0005] According to another aspect of the present disclosure, a testing apparatus for an operator service is provided, the apparatus comprising: an obtaining module configured to obtain an operator service to be tested; a deploying module configured to deploy the operator service and a domain service in a container; and a testing module configured to test the operator service based on the domain service and test data in the container to obtain a corresponding test result; wherein the domain service is configured to invoke the operator service to process the test data according to a preset processing flow during the testing of the operator service.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for testing an operator service as described above.

[0007] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the test method of the operator service as set forth above.

[0008] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the steps of the test method of the operator service as set forth above.

[0009] It should be understood that the contents described in this section are not intended to identify key or important features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0011] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0012] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0013] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0014] Figure 4 is an example diagram of container scheduling management;

[0015] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0016] Figure 6 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0017] Figure 7 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0018] Figure 8 is a schematic diagram according to a seventh embodiment of the present disclosure;

[0019] Figure 9 is a schematic diagram according to an eighth embodiment of the present disclosure;

[0020] Figure 10 is a block diagram of an electronic device for testing of an operator service to implement an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which are meant to be exemplary in nature, and include various details intended to facilitate understanding of the present disclosure. Accordingly, it should be understood that various changes and modifications to the embodiments described herein can be made by those having ordinary skill in the art without departing from the scope and spirit of the present disclosure. Also, for the purpose of clarity and a concise description, descriptions of well-known functions and constructions are omitted from the following description.

[0022] An operator service, i.e., encapsulating an operator as an independent service entity, enables the operator to exist in the form of a standardized, directly callable service for convenient access and use by external systems or applications. The operator service needs to be tested in the development process or in the iteration process after being deployed in the production environment to determine whether the operator service meets the standard for use in the production environment.

[0023] In related technologies, when testing the operator service, a static deployment method is adopted, i.e., deploying the operator service on an operating system, which requires manual configuration of the environment and dependencies. The deployment of the operator service is limited by the version difference of the operating system, and the deployment process of the operator service is tedious and time-consuming. Moreover, the operator service can only be tested when deployed in the production environment (online), and the online of the operator service requires the online of the entire operating system, which results in a long testing time of the operator service and an inability to respond to changes in business requirements in a timely manner.

[0024] To solve the above problems, the present disclosure provides an operator service testing method, device and electronic equipment.

[0025] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure. It should be noted that the operator service testing method of the embodiments of the present disclosure can be applied to an operator service testing device, which can be an electronic equipment or can be configured in an electronic equipment to enable the electronic equipment to perform the operator service testing function. The operator service testing device can be, for example, an algorithm iteration platform configured in an electronic equipment.

[0026] The electronic equipment can be any device with computing capability, such as a personal computer (PC), a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a smart speaker, etc. The mobile terminal can have various operating systems, touch screens and / or display screens.

[0027] As shown in Figure 1 The operator service testing method can include the following steps:

[0028] Step 101: Obtain an operator service to be tested.

[0029] In the embodiments of the present disclosure, the number of operator services can be one or more, and the present disclosure does not make any limitation in this regard. The operator encapsulated in the operator service can be an algorithm model or other computing unit, and the present disclosure does not make any limitation in this regard. The operator service can be an operator service in the development process, or an operator service in the iteration process after being deployed in the production environment, that is, the operator service to be tested has a historical version that has been deployed in the production environment, and the present disclosure does not make any limitation in this regard.

[0030] In step 102, the operator service and the domain service are deployed in the container.

[0031] The domain service is a service that can simulate the processing flow corresponding to each application scenario in a certain technical field, and it realizes the processing flow corresponding to different application scenarios by different scenario configuration, such as configuring how to combine multiple operator services in different application scenarios to realize the corresponding processing flow, thereby realizing the simulation of the processing flow corresponding to different application scenarios.

[0032] For example, the domain service can simulate the processing flow corresponding to each application scenario in the business security field, wherein the application scenarios can include, for example, an abnormal information recognition scenario in a picture or a video, an abnormal face recognition scenario in a picture or a video, and the like. Alternatively, the domain service can simulate the processing flow corresponding to each application scenario in the financial credit field, wherein the application scenarios can include, for example, a credit evaluation scenario, a financial risk prediction scenario, and the like. Alternatively, the domain service can simulate the processing flow corresponding to each application scenario in the information recommendation field, wherein the application scenarios can include, for example, an article recommendation scenario, a video recommendation scenario, an article recommendation scenario, and the like.

[0033] In the embodiments of the present disclosure, the processing flow corresponding to a certain application scenario can be realized by one operator service alone or by multiple operator services jointly. One or more operator services that realize the processing flow corresponding to the same application scenario can be deployed in the same container. The codes of the domain services deployed in different containers can be the same.

[0034] For example, assuming that the operator service a and the operator service b jointly realize the processing flow corresponding to the video recommendation scenario, which can include: classifying a user based on the feature information of the user by the operator service a to obtain the category to which the user belongs, and recommending a video for the user based on the category to which the user belongs by the operator service b. Then the operator service a and the operator service b can be deployed in the same container, and the domain service can be deployed in the container. The domain service can configure the combination mode of the operator service a and the operator service b in the video recommendation scenario to realize the processing flow, thereby realizing the simulation of the processing flow corresponding to the video recommendation scenario.

[0035] In step 103, the operator service is tested based on the domain service and test data in the container, and corresponding test results are obtained; wherein the domain service is used to call the operator service to process the test data according to a preset processing flow in the process of testing the operator service.

[0036] The preset processing flow is a processing flow corresponding to an arbitrary application scenario that is set in advance, such as a processing flow in a real scenario in a production environment.

[0037] The test data is data used for testing the operator service. The operator service can process the test data according to the preset processing flow to achieve testing of the operator service. For example, the preset processing flow is a processing flow corresponding to a video recommendation scenario in the above example, and the test data can include feature information of a user to be recommended for a video. Alternatively, the preset processing flow is a processing flow corresponding to an abnormal information identification scenario, and the test data can include multimedia data such as an image or a video to be identified for abnormal information.

[0038] In the embodiments of the present disclosure, in the process of testing the operator service, the domain service can call the operator service to process the test data according to the preset processing flow to simulate testing of the operator service according to the preset processing flow.

[0039] The testing method of the operator service in the embodiments of the present disclosure includes: obtaining an operator service to be tested; deploying the operator service and a domain service in a container; testing the operator service based on the domain service and test data in the container according to a business processing flow, and obtaining corresponding test results; wherein the domain service is used to call the operator service to process the test data according to a preset processing flow in the process of testing the operator service. Thus, the operator service can be automatically deployed in the container by using virtual container technology, and the operator service can be tested in the container. The deployment of the operator service is simple and fast, is not limited by version differences of operating systems, and since the domain service can call the operator service to process the test data according to the preset processing flow, the operator service can be tested according to the preset processing flow, thereby eliminating the need to deploy the operator service in a production environment to test the operator service according to the processing flow in the production environment. The time for deploying the operator service in the production environment is saved, the testing efficiency of the operator service is greatly improved, and changes in business requirements can be responded to in a timely manner.

[0040] In addition, in the case that the operator service has defects, the deployment of the operator service in the container can reduce the rollback time and avoid the impact on the production environment caused by deploying the operator service with defects in the production environment when the operator service is tested, thereby reducing the risk of deploying the operator service in the production environment.

[0041] To clearly illustrate the process of testing operator services in a container based on domain-specific services and test data, this disclosure also provides a method for testing operator services.

[0042] like Figure 2 As shown, Figure 2 This is a schematic diagram based on the second embodiment of the present disclosure. Figure 2 The illustrated embodiment may include the following steps:

[0043] Step 201: Obtain the operator service to be tested.

[0044] Step 202: Deploy operator services and domain services in the container.

[0045] Step 203: Obtain test data through domain-specific services, and process the test data by calling operator services according to the preset processing flow.

[0046] In the embodiments of this disclosure, the preset processing flow can be implemented by a single operator service or by multiple operator services working together. At least one operator service implementing the preset processing flow can be deployed in the same container. The code for domain-specific services deployed in different containers can be identical.

[0047] In the embodiments of this disclosure, an anomaly identification scenario in the field of business security is taken as an example. Test data may include multimedia data to be identified for anomaly identification, and the multimedia data includes at least one of text, video, images, and audio. At least one operator service implementing a preset processing flow may include a first operator service, wherein the first operator service independently implements the preset processing flow. The preset processing flow may include: identifying anomalies in the multimedia data through the first operator service. The processing result obtained by processing the test data may include the confidence level that anomalies exist in the multimedia data.

[0048] The amount of multimedia data can be set as needed.

[0049] In other words, the domain-specific service can acquire multimedia data to be identified for anomaly information, and according to the preset processing flow, call the first operator service to identify anomaly information in the multimedia data, thereby obtaining the confidence level that anomaly information exists in the multimedia data.

[0050] Therefore, the domain-specific service can call the first operator service to process multimedia data according to the processing flow corresponding to the abnormal information identification scenario, so as to simulate the processing flow corresponding to the abnormal information identification scenario and thus realize the operator service test under the abnormal information identification scenario.

[0051] In an embodiment of the present disclosure, an application scenario is taken as an example of an abnormal face recognition scenario in a business security field. The test data can include a video and / or a picture to be subjected to abnormal face recognition. The at least one operator service implementing the preset processing procedure can include a second operator service and a third operator service, where the second operator service and the third operator service jointly implement the preset processing procedure. The preset processing procedure can include: performing face detection on the video and / or the picture by the second operator service to obtain a detection frame including a face, and determining whether the face in the detection frame is a face of an abnormal user by the third operator service. The processing result obtained by processing the test data can include a confidence degree of the presence of the face of the abnormal user in the video and / or the picture.

[0052] The number of videos and / or pictures can be set as needed.

[0053] That is, the domain service can obtain the video and / or the picture to be subjected to abnormal face recognition, and call the second operator service to perform face detection on the video and / or the picture according to the preset processing procedure to obtain a detection frame including a face. The domain service can call the third operator service to determine whether the face in the detection frame is a face of an abnormal user, and obtain a confidence degree of the presence of the face of the abnormal user in the video and / or the picture.

[0054] Therefore, the domain service can call the second operator service and the third operator service to process the video and / or the picture according to the processing procedure corresponding to the abnormal face recognition scenario, so as to simulate the processing procedure corresponding to the abnormal face recognition scenario, thereby realizing the operator service test in the abnormal face recognition scenario.

[0055] In an embodiment of the present disclosure, an application scenario is taken as an example of an item recommendation scenario in an information recommendation field. The test data can include feature information of a target user to be subjected to item recommendation. The at least one operator service implementing the preset processing procedure can include a fourth operator service and a fifth operator service, where the fourth operator service and the fifth operator service jointly implement the preset processing procedure. The preset processing procedure can include: classifying the target user based on the feature information of the target user by the fourth operator service to obtain a type to which the target user belongs, and recommending a target item to the target user based on the type to which the target user belongs by the fifth operator service. The processing result obtained by processing the test data can include a confidence degree of a plurality of candidate items as the target item.

[0056] The number of target users can be set as needed.

[0057] That is, the field service can obtain the feature information of the target user to be recommended for the item, and according to the preset processing flow, call the fourth operator service to classify the target user based on the feature information of the target user, obtain the type to which the target user belongs, and the field service can call the fifth operator service to recommend the target item for the target user based on the type to which the target user belongs, wherein in the process of recommending the target item for the target user based on the type to which the target user belongs, a plurality of candidate items can be obtained as the confidence of the target item, and the confidence can be used to determine the final recommended target item from the plurality of candidate items.

[0058] Therefore, the field service can call the fourth operator service and the fifth operator service to process the feature information of the target user according to the processing flow corresponding to the item recommendation scene, to simulate the processing flow corresponding to the item recommendation scene, thereby realizing the operator service test in the item recommendation scene.

[0059] Step 204, based on at least one of the following information, generate the test result corresponding to the operator service: the processing result of the test data, the data processing amount of the test data and the resource usage of the operator service in the processing process of the test data.

[0060] In the embodiments of the present disclosure, the test on the operator service can include an effect test, and the operator service to be tested can include at least one operator service that implements the preset processing flow alone or jointly. Wherein, the effect test is to test the prediction effect of the operator service. Correspondingly, step 204 can be implemented by the following way:

[0061] Based on the processing result of the at least one operator service for the test data, generate the test result of the corresponding effect test, wherein the test result includes the prediction effect of the at least one operator service, and the prediction effect is represented by at least one of the following indicators: accuracy, precision, recall, and harmonic mean F1 score of precision and recall.

[0062] In one example of the embodiments of the present disclosure, taking the processing result of the test data including the confidence in any of the above application scenarios as an example, the field service can generate the prediction result of the at least one operator service by the following preset strategy: based on the confidence, determine the final prediction result of the at least one operator service, and then the test device of the operator service can determine the prediction effect of the operator service based on the prediction result, or display the prediction result, so as to determine the prediction effect of the operator service based on the prediction result by manual.

[0063] Taking the application scenario as an abnormal information identification scenario, for any multimedia data in the plurality of multimedia data, the domain service can compare the confidence that there is abnormal information in the multimedia data with the confidence threshold value, obtain the prediction result that there is abnormal information in the multimedia data in the case of the confidence being greater than the confidence threshold value, and obtain the prediction result that there is no abnormal information in the multimedia data in the case of the confidence not being greater than the confidence threshold value. Further, the test device of the operator service can determine the accuracy, precision, recall, and harmonic mean F1 score of the precision and recall of the prediction of the operator service based on the prediction result of whether there is abnormal information in the plurality of multimedia data and whether there is actually abnormal information in the plurality of multimedia data.

[0064] Taking the application scenario as an abnormal face identification scenario, for any video or picture in the plurality of videos and / or pictures, the domain service can compare the confidence that there is an abnormal user's face in the video or picture with the confidence threshold value, obtain the prediction result that there is an abnormal user's face in the video or picture in the case of the confidence being greater than the confidence threshold value, and obtain the prediction result that there is no abnormal user's face in the video or picture in the case of the confidence not being greater than the confidence threshold value. Further, the test device of the operator service can determine the accuracy, precision, recall, and harmonic mean F1 score of the precision and recall of the prediction of the operator service based on the prediction result of whether there is an abnormal user's face in the plurality of videos and / or pictures and whether there is actually an abnormal user's face in the plurality of videos and / or pictures.

[0065] Taking the application scenario as an item recommendation scenario, for a plurality of target users, the domain service can compare the confidence that a plurality of candidate items are target items with the confidence threshold value, and determine a preset number of candidate items whose confidence is greater than the confidence threshold value as target items, which are the prediction results of the operator service. Further, the test device of the operator service can determine the accuracy, precision, recall, and harmonic mean F1 score of the precision and recall of the prediction of the operator service based on the prediction results corresponding to the plurality of target users and the feedback of the plurality of target users.

[0066] Therefore, without deploying the operator service that implements the same processing flow alone or jointly in the production environment, the effect of the operator service can be tested in the container according to the processing flow in the production environment.

[0067] In the embodiments of the present disclosure, the test on the operator service can include a stress test, a data volume of test data is greater than a preset threshold, and the operator service includes at least one operator service that individually or jointly implements a preset processing flow. The stress test is to simulate a plurality of concurrent processing requests of users by a stress testing tool, and to test the service performance of the operator service under high load. Correspondingly, step 204 can be implemented in the following manner:

[0068] Based on the data processing volume of the at least one operator service for the test data and / or the resource usage of the at least one operator service in the process of processing the test data, a test result of the corresponding stress test is generated, wherein the test result includes the service performance of the at least one operator service, and the service performance is represented by at least one of the following indicators: data processing speed, resource usage. The resource usage can include the usage of resources such as memory, CPU (Central Processing Unit), and the like.

[0069] In one example of the embodiments of the present disclosure, the test device of the operator service can generate the service performance of the at least one operator service by the following preset strategy: based on the total processing volume of the at least one operator service for the test data, the data processing volume of the at least one operator service for the test data per unit time is obtained, and the resource usage of the at least one operator service in the process of processing the test data is obtained, so as to obtain the data processing speed and resource usage of the at least one operator service.

[0070] Therefore, without deploying the operator services that individually or jointly implement the same processing flow in the production environment, the stress test on the operator services in the container according to the processing flow in the production environment can be implemented, so as to obtain the service performance of the operator services under high load.

[0071] In the embodiments of the present disclosure, the test on the operator service can include an A / B test, the operator service includes a plurality of operator service sets, and the same operator service set includes at least one operator service that individually or jointly implements a preset processing flow. The A / B test is to compare and test the plurality of operator service sets, so as to compare the difference between the plurality of operator service sets. Correspondingly, step 204 can be implemented in the following manner:

[0072] Based on at least one of the processing result and the data processing volume of the plurality of operator service sets for the test data, and the resource usage in the process of processing the test data, a test result of the corresponding A / B test is generated, wherein the test result includes the difference between the plurality of operator service sets.

[0073] The processing flows implemented by different operator service sets are the same.

[0074] The plurality of operator service sets can be deployed in the same container or in different containers, and the disclosure does not limit this.

[0075] The difference of the plurality of operator service sets can include differences in prediction effect, service performance, etc.

[0076] In the embodiment of the disclosure, the test device of the operator service can generate the difference of the plurality of operator service sets through the following preset strategy: based on any operator service set, based on the processing result of the operator service set on the test data, obtaining the prediction effect of the operator service set, and based on the data processing amount of the operator service set on the test data, obtaining the data processing speed of the operator service set on the test data, and obtaining the resource usage of the operator service set in the process of testing the test data, and then comparing the prediction effect, data processing speed and resource usage of the plurality of operator service sets to obtain the difference of the plurality of operator service sets in prediction effect and service performance.

[0077] Therefore, without deploying the plurality of operator service sets in the production environment, the contrast test of the plurality of operator service sets in the container according to the processing flow in the production environment can be realized, so as to obtain the difference of the plurality of operator service sets.

[0078] In the embodiment of the disclosure, the test result is generated based on the preset strategy, and the test method of the operator service can further include:

[0079] In response to receiving the update request of the preset strategy sent by the cloud server, updating the preset strategy based on the configuration information of the preset strategy carried in the update request.

[0080] In the embodiment of the disclosure, the test device of the operator service can be a client, and has a corresponding cloud server. The user can configure the preset strategy at any time in the interactive interface of the cloud server, such as configuring the confidence threshold and / or the preset number in the above example. The test device of the operator service can interact with the cloud server, so as to update the preset strategy based on the configuration information of the preset strategy carried in the update request in response to receiving the update request of the preset strategy sent by the cloud server. After the preset strategy is updated, the updated preset strategy can be quickly loaded, and the test process of the operator service is re-executed based on the updated preset strategy.

[0081] Therefore, the preset strategy can be flexibly configured according to the demand through the cloud server, and the preset strategy can be dynamically updated based on the configuration information of the preset strategy.

[0082] In an embodiment of the present disclosure, the operator service deployed in the container can also be dynamically updated according to user needs, for example, the test device of the operator service can provide an interactive interface, and the user can trigger a request for updating the version of the operator service on the interactive interface, so that the test device of the operator service can quickly redeploy a new version of the operator service in the container to quickly update the deployed operator service in the container to the new version of the operator service. Since the operator service does not need to be put online, the operator service can be redeployed without stopping the operation of the operating system, thereby realizing hot deployment of the operator service.

[0083] In one example of an embodiment of the present disclosure, the test data can be automatically obtained from the online log by the container service provided by the container at a regular time, and the test data can be packaged into a data package together with the operator service when the operator service and the domain service are deployed in the container, so that when the test data is obtained by the domain service in step 203, the test data can be obtained from the data package in the container by the domain service.

[0084] The online log is a file generated by recording events and processed data occurring when the operator service runs in an actual production environment. It should be noted that the operator service can be a historical version of the operator service to be tested.

[0085] By packaging the test data obtained from the online log into a data package together with the operator service and deploying the data package in the container, the test data and the operator service are integrated, and since the test data is obtained from the online log, the test can be performed without difference from the real data in the production environment, thereby ensuring the performance stability and consistency of the operator service that passes the test in the production environment.

[0086] In another example of an embodiment of the present disclosure, the test data can also be offline data with or without annotations specified by the user, and the test data can be packaged into a data package together with the operator service when the operator service and the domain service are deployed in the container, so that when the test data is obtained by the domain service in step 203, the test data can be obtained from the data package in the container by the domain service.

[0087] In an embodiment of the present disclosure, the test data can be put into a message queue, so that the domain service can asynchronously obtain the test data from the message queue and test the operator service, thereby further improving the test efficiency of the operator service.

[0088] It should be noted that similar steps in other embodiments of the present disclosure can be described in other embodiments, and will not be described in detail here.

[0089] The test method of the operator service provided in the embodiments of the present disclosure can obtain test data through the domain service, and call the operator service to process the test data according to a preset processing flow. The test result corresponding to the operator service is generated based on at least one of the following information: the processing result of the test data, the data processing amount of the test data, and the resource usage of the operator service in the processing process of the test data. The operator service can be tested based on the virtual container technology and the preset processing flow, so that the operator service can be tested according to the processing flow in the production environment without being deployed in the production environment, thereby saving the time of deploying the operator service in the production environment, greatly improving the test efficiency of the operator service, and responding to the changes in business requirements in a timely manner. In addition, the operator service with defects can be avoided from being deployed in the production environment and affecting the production environment when the operator service is tested, thereby reducing the risk of deploying the operator service in the production environment.

[0090] In the embodiments of the present disclosure, the environment pool including a plurality of containers can also be managed. Before the deployment of the operator service and the domain service, the idle container is determined from the environment pool, and then the operator service and the domain service to be tested are deployed in the idle container. The test method of the operator service provided in the embodiments of the present disclosure will be further described below. Figure 3 The test method of the operator service provided in the embodiments of the present disclosure will be further described below.

[0091] Figure 3 is a schematic diagram according to the third embodiment of the present disclosure, as shown in Figure 3 The test method of the operator service can include the following steps:

[0092] Step 301, in response to the test request for the operator service triggered in the interactive interface, an idle container is allocated for the operator service.

[0093] In the embodiments of the present disclosure, the test device of the operator service can provide an interactive interface, and the user can trigger a test request for the operator service in the interactive interface. The container scheduling management module or unit can be included in the test device of the operator service, so that the container scheduling management module or unit can allocate an idle container for the operator service in response to the test request for the operator service triggered in the interactive interface.

[0094] The idle container is a container that is currently not deployed with any operator service and domain service. The container deployed with the operator service and the domain service is in an occupied state.

[0095] In the embodiments of the present disclosure, the container scheduling management module or unit in the test device of the operator service can be based on a K8s platform (a platform for managing containers) to dynamically manage an environment pool including multiple containers, to realize dynamic allocation of container resources, including application, allocation, release, and the like of the containers. Among them, resources can be dynamically allocated in the resource pool according to the test request of the operator service, and the automation of resource allocation can be realized according to an automation script or an API (Application Programming Interface, application programming interface). And a release mechanism can be set to ensure that the container can be released in time when it is no longer needed.

[0096] In step 302, the code corresponding to the operator service is obtained from the code library through the container service provided by the container.

[0097] In step 303, the operator service and the domain service are deployed in the container based on the code.

[0098] In step 304, the operator service is tested in the container based on the domain service and the test data to obtain the corresponding test result; wherein the domain service is used to call the operator service to process the test data according to the preset processing flow in the process of testing the operator service.

[0099] In the embodiments of the present disclosure, the container is in an occupied state during the test of the operator service, and is released when the test of the operator service is completed.

[0100] Reference Figure 4 For example, the container scheduling management unit 401 in the test device of the operator service can schedule the container according to any test task to deploy the operator service and the domain service in the idle container. Among them, any test request triggered by the user to test the operator service is a test task. For example, the test task of the operator service M can schedule the K8s instance M 402 to deploy the operator service M and the domain service, and the test task of the operator service N can schedule the K8s instance N 403 to deploy the operator service N and the domain service. Among them, the K8s instance refers to the K8s container instance, that is, the container, which is a running container entity. Among them, any K8s instance in the K8s instance M 402 or the K8s instance N 403 can perform dynamic deployment of the operator service, dynamic deployment of the scenario service, and dynamic acquisition of the test data, and can report the synchronization data of the test task to the container scheduling management unit 401 through the task synchronization component. Among them, the synchronization data of the test task can include the state of the corresponding container (occupied state or released state, etc.), test result, etc.

[0101] In the test method of the operator service in the embodiments of the present disclosure, the idle container is allocated for the operator service in response to the test request for the operator service triggered in the interactive interface, and the container is released in the case that the test of the operator service is completed, so that the dynamic allocation of the container is implemented, and the resource utilization is improved. Moreover, the code corresponding to the operator service is obtained from the code library through the container service provided by the container in response to the test request for the operator service triggered in the interactive interface, and the operator service and the corresponding domain service are deployed in the container based on the code, so that the dynamic deployment of the operator service can be flexibly implemented.

[0102] The effect test process of the operator service A is exemplarily described below taking the application scenario as an abnormal information identification scenario.

[0103] Figure 5 is a schematic diagram according to the fourth embodiment of the present disclosure.

[0104] In step 501, the idle container 1 is allocated for the operator service A in response to the test request for the effect test of the operator service A triggered in the interactive interface.

[0105] The operator service A is the first operator service in the above embodiments.

[0106] The operator service A implements the preset processing flow alone, and the preset processing flow includes: performing abnormal information identification on the multimedia data by the operator service A.

[0107] In step 502, the code corresponding to the operator service A is obtained from the code library through the container service provided by the container 1.

[0108] In step 503, the operator service A and the domain service are deployed in the container 1 based on the code.

[0109] In step 504, the test data is obtained through the domain service, and the test data includes the multimedia data to be subjected to abnormal information identification, and the multimedia data includes at least one of text, video, picture and audio.

[0110] In the embodiments of the present disclosure, the test data can be obtained from the online log through the container service provided by the container 1, and then the test data can be packaged into a data packet together with the operator service A and deployed in the container 1. Thus, the test data can be obtained from the data packet in the container 1 by the domain service.

[0111] In step 505, the abnormal information identification on the test data is performed by the operator service A according to the preset processing flow through the domain service, and the processing result of the operator service A for the test data is obtained, wherein the processing result includes the confidence of the abnormal information existing in the multimedia data.

[0112] Step 506, based on the processing result of the test data by the operator service A, a test result corresponding to the effect test is generated, wherein the test result includes the predicted effect of the operator service A, and the predicted effect is represented by at least one of the following indicators: accuracy, precision, recall, and harmonic mean F1 score of precision and recall.

[0113] The following takes the application scenario of abnormal face recognition scenario as an example to illustrate the stress test process of the operator service B and the operator service C.

[0114] Figure 6 is a schematic diagram according to the fifth embodiment of the present disclosure.

[0115] Step 601, in response to the test request for stress testing of the operator service B and the operator service C triggered in the interaction interface, the idle container 2 is allocated to the operator service B and the operator service C.

[0116] Wherein, the operator service B is the second operator service in the above embodiment, and the operator service C is the third operator service in the above embodiment.

[0117] Wherein, the operator service B and the operator service C jointly implement a preset processing flow, and the preset processing flow includes: performing face detection on the video and / or picture through the second operator service to obtain a detection frame including a face, and determining whether the face in the detection frame is the face of an abnormal user through the third operator service.

[0118] Step 602, obtaining the code corresponding to the operator service B and the operator service C from the code library through the container service provided by the container 2.

[0119] Step 603, based on the code, deploying the operator service B, the operator service C and the domain service in the container 2.

[0120] Step 604, obtaining the test data through the domain service, wherein the test data includes video and / or picture to be subjected to abnormal face recognition, and the data volume of the test data is greater than a preset threshold.

[0121] In the embodiment of the present disclosure, the test data can be obtained from the online log through the container service provided by the container 2, and then the test data can be packaged into a data packet together with the operator service B and the operator service C, and deployed in the container 2. Thus, the domain service can obtain the test data from the data packet in the container 2.

[0122] Step 605, through the domain service, calling the operator service B to perform face detection on the video and / or picture according to the preset processing flow to obtain a detection frame including a face, and calling the operator service C to determine whether the face in the detection frame is the face of an abnormal user.

[0123] In step 606, based on the data processing amount of the operator service B and the operator service C for the test data and the resource usage of the operator service B and the operator service C in the process of processing the test data, a test result corresponding to the stress test is generated, wherein the test result includes the service performance of the operator service B and the operator service C, and the service performance is represented by at least one of the following indicators: data processing speed, resource usage.

[0124] The following is an example of the comparison test process of the operator service D and the operator service E taking the application scenario of the item recommendation scenario as an example.

[0125] Figure 7 is a schematic diagram according to the sixth embodiment of the present disclosure.

[0126] In step 701, in response to the test request for comparing the operator service set 1 and the operator service set 2 triggered in the interactive interface, the idle container 3 is allocated to the operator service set 1, and the idle container 4 is allocated to the operator service set 2.

[0127] Among the operator service set 1, the operator service D1 and the operator service E1 jointly implement the preset processing flow, and among the algorithm service set 2, the operator service D2 and the operator service E2 jointly implement the preset processing flow.

[0128] Among the operator service D1 and the operator service D2, that is, the fourth operator service in the above embodiment, the operator service E1 and the operator service E2, that is, the fifth operator service in the above embodiment.

[0129] The preset processing flow includes: classifying a target user based on feature information through a fourth operator service to obtain a type to which the target user belongs, and recommending a target item to the target user based on the type to which the target user belongs through a fifth operator service.

[0130] In step 702, the code corresponding to the operator service D1 and the operator service E1 is obtained from the code library through the container service provided by the container 3.

[0131] In step 703, the code corresponding to the operator service D2 and the operator service E2 is obtained from the code library through the container service provided by the container 4.

[0132] Among them, step 702 and step 703 can be executed in parallel or in sequence, and the present disclosure does not limit this.

[0133] In step 704, based on the code corresponding to the operator service D1 and the operator service E1, the operator service D1, the operator service E1 and the domain service are deployed in the container 3.

[0134] At step 705, the operator service D2 and the operator service E2 are deployed in the container 4 based on the corresponding codes of the operator service D2 and the operator service E2.

[0135] Steps 704 and 705 can be executed in parallel or in sequence, and the present disclosure does not limit the execution sequence.

[0136] At step 706, the test data is obtained by the domain service in the container 3, wherein the test data includes feature information of a target user to be recommended with an item.

[0137] In an embodiment of the present disclosure, the test data can be obtained from the online log by the container service provided by the container 3 at a certain time, and then the test data can be packaged together with the operator service D1 and the operator service E1 into a data package and deployed in the container 3. Thus, the domain service in the container 3 can obtain the test data from the data package in the container 3.

[0138] At step 707, the target user is classified based on the feature information by the domain service in the container 3 according to a preset processing flow, and the type to which the target user belongs is obtained, and the target user is recommended with a target item based on the type to which the target user belongs by the operator service E1, and the processing result of the operator service set 1 for the test data is obtained.

[0139] The processing result includes a confidence degree of a plurality of candidate items as the target item.

[0140] At step 708, the test data is obtained by the domain service in the container 4, wherein the test data includes feature information of a target user to be recommended with an item.

[0141] In an embodiment of the present disclosure, the test data can be obtained from the online log by the container service provided by the container 4 at a certain time, and then the test data can be packaged together with the operator service D2 and the operator service E2 into a data package and deployed in the container 4. Thus, the domain service in the container 4 can obtain the test data from the data package in the container 4.

[0142] Steps 706 and 708 can be executed in parallel or in sequence, and the present disclosure does not limit the execution sequence.

[0143] At step 709, the target user is classified based on the feature information by the domain service in the container 4 according to a preset processing flow, and the type to which the target user belongs is obtained, and the target user is recommended with a target item based on the type to which the target user belongs by the operator service E2, and the processing result of the operator service set 2 for the test data is obtained.

[0144] The processing result includes a plurality of candidate items as target items and a confidence degree.

[0145] At step 710, based on the processing result and the data processing amount of the test data of the operator service set 1 and the operator service set 2 respectively, and the resource usage in the process of processing the test data, the corresponding test result of the contrast test is generated, wherein the test result includes the difference between the operator service set 1 and the operator service set 2.

[0146] It should be noted that the above abnormal information identification scene, abnormal face recognition scene, item recommendation scene and the preset processing flow in each application scene are only used for illustration, and in actual application, the operator service for testing any processing flow in any application scene can be tested by using the test method of the operator service provided by the embodiments of the present disclosure, and the present disclosure does not limit this.

[0147] The process of the algorithm iteration platform for implementing the effect test, the stress test and the contrast test of the operator service will be illustrated below.

[0148] Figure 8 is a schematic diagram according to the seventh embodiment of the present disclosure.

[0149] Reference Figure 8 The architecture of the algorithm iteration platform can include an admission module 801, wherein the admission module 801 includes an effect test unit 802, a stress test unit 803 and a contrast test unit 804.

[0150] Taking the currently online operator service as the initial operator service, the initial operator service is updated to obtain the V1 version of the operator service, that is, the operator service V1, which needs to be tested. The initial operator service can include an operator service that implements a preset processing flow alone or multiple operator services that jointly implement a preset processing flow. A user can trigger a test request for the effect test of the operator service V1 through an interactive interface provided by the algorithm iteration platform. In response to the request, the effect test unit 802 can apply for an idle container to the container scheduling management unit 808. The container scheduling management unit 808 can allocate an idle container to the operator service V1 based on the K8s platform 809 according to the state of the container contained in the experimental environment pool 805. The container contained in the experimental environment pool is a container specially used for effect test. Further, the idle container can be used as an experimental environment for effect test, and the domain service and the operator service V1 are deployed in the experimental environment. The test data is obtained through the domain service, the operator service V1 processes the test data according to the preset processing flow, and the corresponding test result of the effect test is generated based on the processing result of the test data by the operator service V1, which includes the prediction effect of the operator service V1. According to the prediction effect, it can be determined whether the operator service V1 meets the online standard. If it meets, it is determined that the test of the operator service V1 is passed. If it does not meet, the operator service V1 can be returned to offline iteration. Through the offline iteration of the operator service V1 combined with the test result of the effect test corresponding to the operator service V1, the operator service can better adapt to the environment and demand.

[0151] Further, the user can trigger a test request for the stress test of the operator service V1 through the interaction interface provided by the algorithm iteration platform. In response to the request, the stress test unit 803 can apply for an idle container from the container scheduling management unit 808. The container scheduling management unit 808, based on the K8s platform 809, can allocate an idle container to the operator service V1 according to the state of the container contained in the stress test environment pool 806. The container contained in the stress test environment pool is a container specially used for stress test. Further, the idle container can be used as a stress test environment for performing stress test, and the domain service and the operator service V1 are deployed in the stress test environment. The test data is obtained through the domain service, and the operator service V1 is called to process the test data according to the preset processing flow. Based on the data processing amount of the test data by the operator service V1 and the resource usage in the processing process of the test data by the operator service V1, the test result of the corresponding stress test is generated, which includes the service performance of the operator service V1. According to the service performance, it can be determined whether the operator service V1 meets the online standard. If it meets, it is determined that the test of the operator service V1 is passed. If it does not meet, the operator service V1 can be returned to offline iteration. Through the offline iteration of the operator service V1 combined with the test result of the corresponding stress test of the operator service V1, the service performance of the operator service can be improved, and the stability and efficiency of the operator service in coping with high-load scenarios can be improved.

[0152] Further, the user can trigger a test request for the contrast test of the initial operator service and the operator service V1 through the interactive interface provided by the algorithm iteration platform. In response to the request, the contrast test unit 804 can apply for an idle container from the container scheduling management unit 808. The container scheduling management unit 808, based on the K8s platform 809, can allocate idle containers to the initial operator service and the operator service V1 according to the state of the containers contained in the contrast test environment pool 807. The containers contained in the contrast test environment pool are containers specially used for contrast test. Further, the idle containers can be used as contrast test environments for contrast test. Different containers or the same container can be allocated to the initial operator service and the operator service V1, respectively. Taking the case of allocating different containers to the initial operator service and the operator service V1 as an example, the domain service and the initial operator service can be deployed in one container, and the test data can be obtained through the domain service, and the initial operator service can be called to process the test data according to the preset processing flow. The domain service and the operator service V1 can be deployed in another container, and the test data can be obtained through the domain service, and the operator service V1 can be called to process the test data according to the preset processing flow. Based on the processing results of the initial operator service and the operator service V1 on the test data, the data processing amount, and the resource usage in the process of processing the test data, the test results of the contrast test are generated, which include the difference between the initial operator service and the operator service V1. According to the difference, it can be determined whether the operator service V1 meets the online standard. If it meets, it is determined that the test of the operator service V1 is passed. If it does not meet, the operator service V1 can be sent back for offline iteration. By combining the test results of the contrast test corresponding to the initial operator service and the operator service V1 and various performance indicators and user feedback, the operator service V1 can be iterated offline, so that the service performance and prediction effect of the iterated operator service can meet the online use standard.

[0153] In addition, the prediction effect, service performance of the operator service V1, and the difference from the initial operator service can generate a result report and be provided to the user.

[0154] Taking the operator service V1 optimized through offline iteration as the operator service V2 as an example, similar to the above process, the user can trigger the effect test, stress test, and contrast test of the operator service V2 again. Further, the algorithm iteration platform can quickly update the operator service V1 deployed in the container to the operator service V2, and perform the effect test, stress test, and contrast test on the operator service V2 again to obtain the test results corresponding to the tests. Further, it can be determined whether the operator service V2 meets the online standard according to the test results. If it meets, it is determined that the test of the operator service V2 is passed. If it does not meet, the operator service V2 can be sent back for offline iteration until it meets the online use standard.

[0155] Thus, the operator service can be automatically deployed in the container by the virtualization container technology, and the operator service can be tested for effect, stress, and control in the container, so as to realize comprehensive evaluation and optimization of the operator service. The operator service is tested in the container environment, so that the consistency and stability of the operator service are ensured. The deployment mode of the operator service is simple and fast, and is not limited by version differences of operating systems, so that the flexibility and expansibility of the system are improved. Moreover, the domain service can call the operator service to process the test data according to the preset processing flow, so that the operator service can be tested according to the preset processing flow without being deployed in the production environment, the time for deploying the operator service in the production environment is saved, the test efficiency of the operator service is greatly improved, and the operator service can respond to changes in business requirements in a timely manner. In the iteration process of the operator service, the operator service before going online can be tested by using the test mode, so that the iteration efficiency of the operator service can be greatly improved, and the iteration period can be reduced. By deploying the operator service and the domain service in the container and performing scenario-based packaging of the operator service, the operator service can be uniformly packaged and has the ability to be quickly applied in different scenarios, so that the applicability and flexibility of the operator service are improved. In addition, in the case that the operator service has defects, the operator service can be deployed in the container to reduce the rollback time, and the operator service with defects can be avoided from being deployed in the production environment to affect the production environment when the operator service is tested, so that the risk of deploying the operator service in the production environment is reduced.

[0156] To achieve the above-mentioned embodiments, the present disclosure further provides an operator service testing device. As shown in Figure 9 Figure 9 is a schematic view according to an eighth embodiment of the present disclosure. The operator service testing device 90 can include an acquisition module 901, a deployment module 902, and a testing module 903.

[0157] The acquisition module 901 is configured to acquire an operator service to be tested.

[0158] The deployment module 902 is configured to deploy the operator service and a domain service in a container.

[0159] The testing module 903 is configured to test the operator service based on the domain service and test data in the container, to obtain a corresponding test result.

[0160] The domain service is configured to call the operator service to process the test data according to a preset processing flow in the process of testing the operator service.

[0161] As a possible implementation manner of the embodiments of the present disclosure, the testing module 903 includes:​

[0162] a processing unit, configured to acquire test data by means of the domain service, and invoke an operator service to process the test data according to a preset processing flow;

[0163] a generating unit, configured to generate a test result corresponding to the operator service based on at least one of the following information: a processing result of the test data, a data processing amount of the test data, and a resource usage of the operator service in a processing process of the test data.

[0164] As a possible implementation manner of the embodiment of the present disclosure, the test on the operator service includes an effect test, the operator service includes at least one operator service that implements a preset processing flow alone or jointly; the generating unit includes:

[0165] a first generating sub-unit, configured to generate a test result of the effect test corresponding to the at least one operator service based on a processing result of the test data of the at least one operator service, wherein the test result includes a predicted effect of the at least one operator service, and the predicted effect is represented by at least one of the following indexes: accuracy, precision, recall, and harmonic mean F1 score of the precision and the recall.

[0166] As a possible implementation manner of the embodiment of the present disclosure, the test on the operator service includes a stress test, a data amount of the test data is greater than a preset threshold, and the operator service includes at least one operator service that implements a preset processing flow alone or jointly;

[0167] the generating unit includes:

[0168] a second generating sub-unit, configured to generate a test result of the stress test corresponding to the at least one operator service based on a data processing amount of the test data of the at least one operator service and / or a resource usage of the at least one operator service in a processing process of the test data, wherein the test result includes a service performance of the at least one operator service, and the service performance is represented by at least one of the following indexes: data processing speed and resource usage.

[0169] As a possible implementation manner of the embodiment of the present disclosure, the test on the operator service includes a control test, the operator service includes a plurality of operator service sets, and a same operator service set includes at least one operator service that implements a preset processing flow alone or jointly;

[0170] the generating unit includes:

[0171] a third generating sub-unit, configured to generate a test result of the control test corresponding to the plurality of operator service sets based on at least one of the following information: a processing result and a data processing amount of the test data of the plurality of operator service sets, and a resource usage of the plurality of operator service sets in a processing process of the test data, wherein the test result includes a difference situation of the plurality of operator service sets.

[0172] As a possible implementation manner of the embodiment of the present disclosure, the at least one operator service includes a first operator service; the test data includes multimedia data to be subjected to abnormal information identification, and the multimedia data includes at least one of text, video, picture and audio; and the preset processing flow includes: identifying abnormal information in the multimedia data by the first operator service; and the processing result includes a confidence degree of existence of abnormal information in the multimedia data.

[0173] As a possible implementation manner of the embodiment of the present disclosure, the at least one operator service includes a second operator service and a third operator service; the test data includes video and / or picture to be subjected to abnormal face identification; the preset processing flow includes: performing face detection on the video and / or picture by the second operator service to obtain a detection frame including a face, and judging whether the face in the detection frame is a face of an abnormal user by the third operator service; and the processing result includes a confidence degree of existence of the face of the abnormal user in the video and / or picture.

[0174] As a possible implementation manner of the embodiment of the present disclosure, the at least one operator service includes a fourth operator service and a fifth operator service; the test data includes feature information of a target user to be subjected to item recommendation; the preset processing flow includes: classifying the target user based on the feature information by the fourth operator service to obtain a type to which the target user belongs, and recommending a target item to the target user based on the type to which the target user belongs by the fifth operator service; and the processing result includes a confidence degree of a plurality of candidate items as the target item.

[0175] As a possible implementation manner of the embodiment of the present disclosure, the test result is generated based on a preset strategy, and the apparatus 900 further includes:

[0176] The updating module is configured to, in response to receiving an updating request of the preset strategy sent by the cloud server, update the preset strategy based on configuration information of the preset strategy carried in the updating request.

[0177] As a possible implementation manner of the embodiment of the present disclosure, the test data is acquired from online logs by a container service provided by a container at a fixed time, and is packaged into a data packet together with the operator service and deployed in the container;

[0178] The processing unit includes:

[0179] The acquisition sub-unit is configured to acquire the test data from the data packet in the container by the domain service.

[0180] As a possible implementation manner of the embodiment of the present disclosure, the acquisition module 901 includes:

[0181] The acquisition unit is configured to acquire, in response to a test request for the operator service triggered at the interactive interface, code corresponding to the operator service from a code library through a container service provided by a container.

[0182] The deployment module 902 includes:

[0183] The deployment unit is configured to deploy the operator service and the corresponding domain service in the container based on the code.

[0184] As a possible implementation manner of the embodiments of the present disclosure, the apparatus 900 further includes:

[0185] The allocation module is configured to allocate, in response to the test request, an idle container to the operator service, wherein the container is in an occupied state during the test of the operator service and is released in the case that the test of the operator service is completed.

[0186] The test apparatus for the operator service according to the embodiments of the present disclosure acquires the operator service to be tested, deploys the operator service and a domain service in a container, and tests the operator service based on the domain service and test data in the container according to a business processing flow to obtain a corresponding test result, wherein the domain service is configured to call the operator service to process the test data according to a preset processing flow during the test of the operator service. Thus, the operator service can be automatically deployed in the container by using the virtualization container technology, and the operator service can be tested in the container. The deployment manner of the operator service is simple and fast, is not limited by version differences of operating systems, and since the domain service can call the operator service to process the test data according to the preset processing flow, the operator service can be tested according to the preset processing flow, so that the operator service can be tested according to the processing flow in the production environment without being deployed in the production environment, the time for deploying the operator service in the production environment is saved, the test efficiency of the operator service is greatly improved, and the change in business demand can be responded in a timely manner.

[0187] In addition, in the case that the operator service has defects, the deployment of the operator service in the container can reduce the rollback time and can avoid the influence on the production environment caused by the deployment of the operator service with defects in the production environment during the test of the operator service, thereby reducing the risk caused by the deployment of the operator service in the production environment.

[0188] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all performed on the premise of obtaining the consent of the user and in accordance with the relevant laws and regulations and do not violate public order and good customs.

[0189] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0190] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0191] As shown in Figure 10 The device 1000 includes a computing unit 1001 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the device 1000 can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0192] Various components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; the storage unit 1008, such as a magnetic disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0193] The computing unit 1001 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs various methods and processes described above, such as the method of testing an operator service. For example, in some embodiments, the method of testing an operator service can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded onto the RAM 1003 and executed by the computing unit 1001, one or more steps of the method of testing an operator service described above can be performed. Alternatively, in other embodiments, the computing unit 1001 can be configured to perform the method of testing an operator service by any other suitable means, such as by means of firmware.

[0194] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0195] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a function / operation specified in the flowchart and / or block diagram. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a remote machine or a server.

[0196] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0197] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0198] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0199] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0200] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology described in the present disclosure are achieved.

[0201] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for testing operator services, characterized in that, The method includes: In response to a test request for the operator service to be tested triggered in the interactive interface, the code corresponding to the operator service is obtained from the code repository through the container service provided by the container. In response to the test request, an idle container is allocated to the operator service. The container is occupied during the testing of the operator service and is released when the testing of the operator service is completed. Based on the code, the operator service and the corresponding domain service are deployed in a container. When deploying the operator service and the domain service, test data obtained from online logs are packaged with the operator service into a data package and deployed in the container. The domain service is used to call the operator service to process the test data according to a preset processing flow during the testing of the operator service. Test data is obtained from the data packet in the container through the domain-specific service. In the container, the operator service is called to process the test data according to the preset processing flow. Based on the processing result of the test data, the amount of data processed by the test data, and / or the resource usage of the operator service during the test data processing, the test result corresponding to the operator service is generated. The test result includes the predicted effect of the effect test, the service performance of the stress test, and the difference of the operator service set in the control test.

2. The method according to claim 1, wherein, The operator service includes at least one operator service that implements the preset processing flow individually or in combination; Based on at least one of the following information, test results corresponding to the operator service are generated: the processing result of the test data, the data processing volume of the test data, and the resource usage of the operator service during the processing of the test data, including: Based on the processing results of the test data by the at least one operator service, a corresponding prediction result of the effect test is generated, and the prediction result is characterized by at least one of the following indicators: accuracy, precision, recall, and the harmonic mean F1 score of the precision and recall.

3. The method according to claim 1, wherein, The amount of test data is greater than a preset threshold, and the operator service includes at least one operator service that implements the preset processing flow individually or in combination. Based on at least one of the following information, test results corresponding to the operator service are generated: the processing result of the test data, the data processing volume of the test data, and the resource usage of the operator service during the processing of the test data, including: Based on the data processing volume of the at least one operator service for the test data and / or the resource usage of the at least one operator service during the processing of the test data, the service performance of the corresponding stress test is generated, and the service performance is characterized by at least one of the following indicators: data processing speed and resource usage.

4. The method according to claim 1, wherein, The operator service includes multiple operator service sets, and the same operator service set includes at least one operator service that implements the preset processing flow individually or in combination. Based on at least one of the following information, test results corresponding to the operator service are generated: the processing result of the test data, the data processing volume of the test data, and the resource usage of the operator service during the processing of the test data, including: Based on at least one of the processing results and data processing volume of the multiple operator service sets for the test data, and the resource usage during the processing of the test data, the differences between the multiple operator service sets for the corresponding control test are generated.

5. The method according to any one of claims 1-4, wherein, At least one operator service includes a first operator service; the test data includes multimedia data to be used for anomaly identification, and the multimedia data includes at least one of text, video, images, and audio; The preset processing flow includes: identifying abnormal information in the multimedia data through the first operator service; The processing result includes the confidence level that there is abnormal information in the multimedia data.

6. The method according to any one of claims 1-4, wherein, At least one operator service includes a second operator service and a third operator service; the test data includes videos and / or images of abnormal faces to be identified; The preset processing flow includes: performing face detection on the video and / or image through the second operator service to obtain a detection box including the face, and determining whether the face in the detection box is the face of an abnormal user through the third operator service; the processing result includes the confidence level that there is an abnormal user's face in the video and / or image.

7. The method according to any one of claims 1-4, wherein, At least one operator service includes a fourth operator service and a fifth operator service; the test data includes feature information of the target users to be recommended items; The preset processing flow includes: classifying the target user based on the feature information using the fourth operator service to obtain the type to which the target user belongs, and recommending target items to the target user based on the type to which the target user belongs using the fifth operator service; the processing result includes multiple candidate items as the confidence level of the target item.

8. The method according to any one of claims 1-4, wherein, The test results are generated based on a preset strategy, and the method further includes: In response to receiving an update request for the preset policy from the cloud server, the preset policy is updated based on the configuration information of the preset policy carried in the update request.

9. A testing apparatus for an operator service, the apparatus comprising: The acquisition module is used to respond to the test request of the operator service to be tested triggered in the interactive interface, and to obtain the code corresponding to the operator service from the code library through the container service provided by the container; An allocation module is used to allocate an idle container to the operator service in response to the test request. The container is occupied during the test of the operator service and is released when the test of the operator service is completed. The deployment module is used to deploy the operator service and the corresponding domain service in a container based on the code. When deploying the operator service and the domain service, test data obtained from online logs is packaged with the operator service into a data package and deployed in the container. The domain service is used to call the operator service to process the test data according to a preset processing flow during the testing of the operator service. The testing module is used to obtain test data from the data packet in the container through the domain-specific service, call the operator service to process the test data in the container according to the preset processing flow, and generate test results corresponding to the operator service based on the processing results of the test data, the data processing volume of the test data, and / or the resource usage of the operator service in the process of processing the test data. The test results include the predicted effect of the effect test, the service performance of the stress test, and the difference of the operator service set in the control test.

10. The apparatus according to claim 9, wherein, The operator service includes at least one operator service that implements the preset processing flow individually or in combination; Based on at least one of the following information, test results corresponding to the operator service are generated: the processing result of the test data, the data processing volume of the test data, and the resource usage of the operator service during the processing of the test data, including: Based on the processing results of the test data by the at least one operator service, a corresponding prediction result of the effect test is generated, and the prediction result is characterized by at least one of the following indicators: accuracy, precision, recall, and the harmonic mean F1 score of the precision and recall.

11. The apparatus according to claim 9, wherein, The amount of test data is greater than a preset threshold, and the operator service includes at least one operator service that implements the preset processing flow individually or in combination. Based on at least one of the following information, test results corresponding to the operator service are generated: the processing result of the test data, the data processing volume of the test data, and the resource usage of the operator service during the processing of the test data, including: Based on the data processing volume of the at least one operator service for the test data and / or the resource usage of the at least one operator service during the processing of the test data, the service performance of the corresponding stress test is generated, and the service performance is characterized by at least one of the following indicators: data processing speed and resource usage.

12. The apparatus according to claim 9, wherein, The operator service includes multiple operator service sets, and the same operator service set includes at least one operator service that implements the preset processing flow individually or in combination. Based on at least one of the following information, test results corresponding to the operator service are generated: the processing result of the test data, the data processing volume of the test data, and the resource usage of the operator service during the processing of the test data, including: Based on at least one of the processing results and data processing volume of the multiple operator service sets for the test data, and the resource usage during the processing of the test data, the differences between the multiple operator service sets for the corresponding control test are generated.

13. The apparatus according to any one of claims 9-12, wherein, At least one operator service includes a first operator service; the test data includes multimedia data to be used for anomaly identification, and the multimedia data includes at least one of text, video, images, and audio; The preset processing flow includes: identifying abnormal information in the multimedia data through the first operator service; The processing result includes the confidence level that there is abnormal information in the multimedia data.

14. The apparatus according to any one of claims 9-12, wherein, At least one operator service includes a second operator service and a third operator service; the test data includes videos and / or images of abnormal faces to be identified; The preset processing flow includes: performing face detection on the video and / or image through the second operator service to obtain a detection box including the face, and determining whether the face in the detection box is the face of an abnormal user through the third operator service; the processing result includes the confidence level that there is an abnormal user's face in the video and / or image.

15. The apparatus according to any one of claims 9-12, wherein, At least one operator service includes a fourth operator service and a fifth operator service; the test data includes feature information of the target users to be recommended items; The preset processing flow includes: classifying the target user based on the feature information using the fourth operator service to obtain the type to which the target user belongs, and recommending target items to the target user based on the type to which the target user belongs using the fifth operator service; the processing result includes multiple candidate items as the confidence level of the target item.

16. The apparatus according to any one of claims 9-12, wherein, The test results are generated based on a preset strategy, and the device further includes: The update module is used to update the preset policy based on the configuration information of the preset policy carried in the update request in response to receiving an update request for the preset policy from the cloud server.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 8.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images