Model testing method, device, distributed cluster, electronic device and storage medium

Through the division of labor of the main node, fixed node and work node of the distributed cluster, the parallel execution of model testing tasks is achieved, which solves the problems of slow testing speed and inability to test on a large scale in the existing technology, and improves the testing efficiency and accuracy.

CN112860553BActive Publication Date: 2024-11-12YUANLI JINZHI (CHONGQING) TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110166222.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-02-05
Publication Date
2024-11-12
Estimated Expiration
2041-02-05

AI Technical Summary

Technical Problem

The existing model testing methods are limited by manpower, resulting in slow testing speed, error-proneness, and large-scale testing cannot be carried out.

Method used

The main node of the distributed cluster allocates test tasks, fixed nodes perform feature extraction, and the worker nodes execute test tasks, realize parallel execution of test tasks, and return the test results to the master node.

Benefits of technology

It improves the speed and accuracy of model testing, enables large-scale testing, and solves the problem of labor limitations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112860553B_ABST
    Figure CN112860553B_ABST
Patent Text Reader

Abstract

The present application provides a model testing method, device, distributed cluster, electronic device and storage medium, the method comprising: upon receiving a test task assigned by a master node, monitoring whether the features corresponding to the test task in the extracted features have met the requirements of the test task. The extracted features are obtained by extracting the test image by a fixed node according to a feature extraction module in the model to be tested. When the features corresponding to the test task have met the requirements of the test task, the features corresponding to the test task are used to execute the test task. In this way, the entire test link of a model is split into different nodes for automatic processing, and different nodes are configured to perform corresponding functions. When multiple test tasks are required, it is only necessary to assign the corresponding models and test tasks to the corresponding nodes, and each node can autonomously perform the corresponding functions. This solves the problem that the model testing is currently limited by manpower, the speed is slow, it is prone to errors, and large-scale testing cannot be carried out.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of performance testing, and in particular to a model testing method, device, distributed cluster, electronic device and storage medium. Background Art

[0002] At present, when testing model performance, it is usually done on a single machine for each test task. If multiple tests need to be performed simultaneously, the test engineer needs to manually configure the corresponding models to be tested on different machines and run the corresponding test instructions. However, this is limited by manpower, and there are problems such as slow testing speed, easy errors, and inability to perform large-scale testing. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a model testing method, device, distributed cluster, electronic device and storage medium to solve the current problems of slow model testing, easy errors and inability to perform large-scale testing.

[0004] An embodiment of the present application provides a model testing method, which is applied to any working node of a distributed cluster, including: when receiving a test task assigned by a master node in the distributed cluster, monitoring whether the features corresponding to the test task in the extracted features have met the requirements of the test task; the extracted features are obtained by extracting the test image by a fixed node in the distributed cluster according to a feature extraction module in the model to be tested; when the features corresponding to the test task have met the requirements of the test task, using the features corresponding to the test task to execute the test task to obtain a test result.

[0005] In the above implementation process, the test tasks are assigned through the master node of the distributed cluster, the feature extraction is performed through the fixed node, and the test tasks are executed through the working node assigned with the test tasks. In this way, the entire test link of a model is split into different nodes for automatic processing, and different nodes are configured to perform corresponding functions. When multiple test tasks are required, it is only necessary to assign the corresponding models and test tasks to the corresponding nodes, and each node can autonomously perform the corresponding functions, thereby realizing the test tasks and completing the parallel execution of multiple test tasks for the model. This solves the current problem that the model testing is slow and error-prone due to manpower constraints, and large-scale testing cannot be carried out.

[0006] Furthermore, the method further includes: after the test task is executed, returning the test result to the master node.

[0007] In the embodiment of the present application, after the test task is executed, the test result can be returned to the master node, so that the master node can perform corresponding operations according to the test result. For example, the test result can be fed back to the user.

[0008] Furthermore, the method also includes: when receiving the model to be tested assigned by the master node, extracting a feature extraction module in the model to be tested; and sending the feature extraction module to the fixed node so that the fixed node can perform feature extraction on the test image.

[0009] In practical applications, a complete model usually has multiple different modules, such as a feature extraction module for indicating how to perform feature extraction, and in addition, there are modules such as loss functions and classifiers. In an embodiment of the present application, the code for feature extraction in a fixed node can be pre-written, but the specific way of how to perform feature extraction for different models (such as how many convolutions, how many poolings, what the order of operations between convolution and pooling is, etc.) will usually be different, and the specific way of how to perform feature extraction is determined by the feature extraction module in the model. Therefore, in the above-mentioned implementation process, the model to be tested is deleted through the working node, and the feature extraction module in the model to be tested is extracted and sent to the fixed node, so that the fixed node can perform feature extraction according to the feature extraction method of the model to be tested, so that the extracted features are compatible with the capabilities of the model to be tested, and then it can be ensured that the test task can accurately implement the performance test of the model to be tested.

[0010] Furthermore, after extracting the feature extraction module in the model to be tested, the method also includes: converting the floating-point number in the feature extraction module into an integer to form a binary module of the feature extraction module; sending the feature extraction module to the fixed node includes: sending the binary module of the feature extraction module to the fixed node.

[0011] In the above implementation process, in order to facilitate the processing of fixed nodes, the feature extraction module can be converted into a binary module. At the same time, in order to reduce the processing complexity of fixed nodes and improve the feature extraction efficiency of fixed nodes, in the above implementation process, the floating point numbers in the feature extraction module are converted into integers, thereby reducing the complexity of the data and making the fixed nodes process faster when performing feature extraction.

[0012] An embodiment of the present application also provides a model testing method for use in any fixed node of a distributed cluster, comprising: receiving a feature extraction module in the model to be tested transmitted by a working node in the distributed cluster; acquiring a test image, and performing feature extraction on the test image according to the feature extraction module, so that the working node in the distributed cluster can perform a test task based on the extracted features.

[0013] In the above implementation process, the feature extraction of the test image can be realized through the feature extraction module in the model to be tested. Since the feature extraction is performed according to the feature extraction module in the model to be tested, the extracted features are adapted to the capabilities of the model to be tested, thereby ensuring that the test task can accurately realize the performance test of the model to be tested.

[0014] Furthermore, performing feature extraction on the test image according to the feature extraction module includes: embedding the feature extraction module into an embedding position of a feature extraction code; and running the feature extraction code embedded in the feature extraction module to perform feature extraction on the test image.

[0015] Furthermore, the method further comprises: outputting the extracted features to a preset database for storage.

[0016] An embodiment of the present application also provides a model testing method, which is applied to the master node of a distributed cluster, including: when receiving a model to be tested and a test task, respectively assigning the model to be tested and the test task to each working node in the distributed cluster, so that the working node assigned with the model to be tested can extract a feature extraction module from the model to be tested, and when the feature corresponding to the test task meets the requirement of the test task, the working node assigned with the test task can execute the test task using the feature corresponding to the test task to obtain a test result; and receiving the test result of the test task returned by the working node that executes the test task.

[0017] In the above implementation process, the models to be tested and the test tasks are allocated through the master node of the distributed cluster, the working nodes assigned with the models to be tested extract the feature extraction module of the models to be tested, and the working nodes assigned with the test tasks execute the test tasks, so that the entire test link of a model is split into different nodes for automatic processing, and different nodes are configured to execute corresponding functions. When multiple test tasks are required, it is only necessary to assign the corresponding models and test tasks to the corresponding nodes, and each node can autonomously execute the corresponding functions, thereby realizing the test tasks and completing the parallel execution of multiple test tasks for the model. This solves the current problem that the model testing is slow and error-prone due to limited manpower, and large-scale testing cannot be carried out.

[0018] Furthermore, the model to be tested and the test task are assigned to the working nodes of the distributed cluster, including: obtaining the idle status of each working node in the distributed cluster, and respectively assigning the model to be tested and the test task to the idle working nodes in the distributed cluster; or, obtaining the load status of each working node in the distributed cluster, and assigning the model to be tested and the test task to each working node with the lowest load in the distributed cluster in turn.

[0019] In the above implementation process, the master node allocates the models to be tested and the test tasks according to the idle state or load condition of each working node, thereby improving the load balancing of the distributed cluster and improving the resource utilization of the distributed cluster to a certain extent.

[0020] Furthermore, the method also includes: monitoring the execution status of work tasks of each working node and fixed node in the distributed cluster; when the execution of a work task of any working node or fixed node fails, sending a task restart instruction to the working node or fixed node to restart the failed work task on the working node or fixed node.

[0021] In the above implementation process, the master node monitors the execution status of the work tasks of each working node and fixed node in the distributed cluster, so that when a problem occurs at any node, it can quickly control its restart task, thereby promoting the smooth execution of the test task of the model to be tested and improving the reliability of the present application solution.

[0022] An embodiment of the present application also provides a model testing device, which is applied to any working node of a distributed cluster, and includes: a first receiving module, a monitoring module and a processing module; the first receiving module is used to receive the test task assigned by the master node in the distributed cluster; the monitoring module is used to monitor whether the features corresponding to the test task in the extracted features have met the requirements of the test task when the first receiving module receives the test task assigned by the master node; the extracted features are obtained by extracting the test image by the fixed node in the distributed cluster according to the feature extraction module in the model to be tested; the processing module is used to use the features corresponding to the test task to execute the test task and obtain the test result when the features corresponding to the test task have met the requirements of the test task.

[0023] An embodiment of the present application also provides a model testing device, which is applied to any fixed node of a distributed cluster, and includes: a second receiving module and an extraction module; the second receiving module is used to receive a feature extraction module in the model to be tested transmitted by the working node in the distributed cluster; and is used to obtain a test image; the extraction module is used to extract features from the test image according to the feature extraction module.

[0024] An embodiment of the present application also provides a model testing device, which is applied to the main node of a distributed cluster, including: an allocation processing module and a third receiving module; the allocation processing module is used to allocate the model to be tested and the test task to each working node in the distributed cluster respectively when receiving the model to be tested and the test task, so that the working node assigned with the model to be tested can perform feature extraction on the model to be tested by a feature extraction module, and the working node assigned with the test task can use the features corresponding to the test task to execute the test task and obtain the test result when the features corresponding to the test task have met the requirements of the test task; the third receiving module is used to receive the test result of the test task returned by the working node that executes the test task.

[0025] An embodiment of the present application also provides a distributed cluster, including: a master node, a working node and a fixed node; wherein: the master node is used to assign the model to be tested to the first working node among the working nodes and assign the test task to the second working node among the working nodes when receiving the model to be tested and the test task; the first working node is used to extract the feature extraction module in the model to be tested when receiving the model to be tested assigned by the master node, and send the feature extraction module to the fixed node; the fixed node is used to obtain a test image, and perform feature extraction on the test image according to the feature extraction module; the second working node is used to monitor whether the features corresponding to the test task in the extracted features have met the requirements of the test task when receiving the test task assigned by the master node, and when the features corresponding to the test task have met the requirements of the test task, use the features corresponding to the test task to execute the test task and obtain the test result.

[0026] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a communication bus; the communication bus is used to realize connection and communication between the processor and the memory; the processor is used to execute one or more programs stored in the memory to implement any of the above-mentioned model testing methods.

[0027] An embodiment of the present application also provides a readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement any of the above-mentioned model testing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 A schematic diagram of the structure of a distributed cluster provided in an embodiment of the present application;

[0030] Figure 2 An interactive schematic diagram of a model testing method provided in an embodiment of the present application;

[0031] Figure 3 An interactive schematic diagram of a model testing method applied to any working node of a distributed cluster provided in an embodiment of the present application;

[0032] Figure 4 An interactive schematic diagram of a model testing method applied to any fixed node of a distributed cluster provided in an embodiment of the present application;

[0033] Figure 5 An interactive schematic diagram of a model testing method applied to a master node of a distributed cluster provided in an embodiment of the present application;

[0034] Figure 6 A schematic diagram of a distributed cluster provided in an embodiment of the present application;

[0035] Figure 7 A schematic diagram of the structure of a model testing device applied to a working node provided in an embodiment of the present application;

[0036] Figure 8 A schematic diagram of the structure of a model testing device applied to a fixed node provided in an embodiment of the present application;

[0037] Fig. 9 A schematic diagram of the structure of a model testing device applied to a master node provided in an embodiment of the present application;

[0038] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0040] Embodiment 1:

[0041] In order to solve the problems that the current testing method is limited by manpower, has slow testing speed, is prone to errors, and cannot perform large-scale testing, a distributed cluster and a model testing method applied in a distributed cluster are provided in an embodiment of the present application.

[0042] See also Figure 1 As shown in the figure, the distributed cluster includes three types of nodes: master node, working node and fixed node. Among them: the master node is used as a management node, responsible for the work management work in the entire test process, while the fixed node is a node in the distributed cluster that has feature extraction code pre-written and is responsible for feature extraction, and the working node is a node that can freely allocate work. The master node can assign the model to be tested or the test task to the working node for processing.

[0043] See also Figure 2 As shown, Figure 2 The interactive schematic diagram of the model testing method provided in the embodiment of the present application includes:

[0044] S201: When receiving the model to be tested and the test task, the master node distributes the received model to be tested and the test task to the working nodes in the distributed cluster.

[0045] It should be understood that for a test, it needs to include a model to be tested and a test task, and the test task includes parameters such as the test content and test requirements required for the model to be tested. The model to be tested and the test task can be input into the master node by an engineer.

[0046] In an embodiment of the present application, a load balancing strategy can be pre-set in the master node, so that the master node allocates working nodes to the received models to be tested and test tasks in turn according to the load balancing strategy.

[0047] Exemplarily, the master node may obtain the idle status of each working node in the distributed cluster, and then respectively assign the models to be tested and the test tasks to the idle working nodes in the distributed cluster.

[0048] Alternatively, the master node may also obtain the load status of each working node in the distributed cluster, and sequentially distribute the models to be tested and the test tasks to the working nodes with the lowest load in the distributed cluster.

[0049] In an embodiment of the present application, in order to obtain the idle state or load condition of each working node in a distributed cluster, a communication connection between the master node and each working node can be set, so that the idle state or load condition of each working node can be monitored in real time.

[0050] It is worth noting that in the embodiment of the present application, in order to ensure the reliability of the test task, different working nodes can be assigned to each test task and each model to be tested.

[0051] For ease of description, in the embodiment of the present application, the working node to which the model to be tested is assigned is referred to as the first working node, and the working node to which the test task is assigned is referred to as the second working node.

[0052] S202: The first working node extracts a feature extraction module in the model to be tested.

[0053] It should be understood that a model usually includes multiple modules. For example, there will be a feature extraction module for indicating how to extract features, and there will also be modules such as loss functions for determining the amount of model loss. If it is a classification model, there will also be modules such as classifiers for outputting classification results.

[0054] In the embodiment of the present application, the fixed node only needs to indicate the content of the feature extraction module on how to perform feature extraction, and the other modules are not required. Therefore, the feature extraction module needs to be extracted from the model to be tested.

[0055] Exemplarily, in a feasible implementation of the embodiment of the present application, an engineer may mark the feature extraction module of the model to be tested in advance, so as to extract the feature extraction module from the model to be tested through the mark.

[0056] S203: Send the extracted feature extraction module in the model to be tested to the fixed node.

[0057] In an embodiment of the present application, the IP address and port of the fixed node may be pre-configured in each working node, thereby facilitating sending the extracted feature extraction module to the fixed node.

[0058] In order to facilitate the identification and processing of fixed nodes and to increase the speed of feature extraction of fixed nodes, in an embodiment of the present application, the floating-point numbers in the feature extraction module can be converted into integers to form a binary module of the feature extraction module, which is then sent to the fixed nodes.

[0059] S204: The fixed node obtains a test image, and performs feature extraction on the test image according to the feature extraction module.

[0060] In the embodiment of the present application, the feature extraction code may be pre-configured, so as to combine the feature extraction code and the feature extraction module to perform feature extraction on the test image.

[0061] The feature extraction code may include codes for feature extraction, such as codes for convolution operations, codes for pooling operations, etc. The feature extraction module shows how to perform feature extraction, such as how many convolutions and pooling operations are required, and what is the order of operations between convolution and pooling.

[0062] In order to be able to perform feature extraction according to the feature extraction method of the prediction model, in the embodiment of the present application, an embedding position can be set in advance at the corresponding position of the feature extraction code, and a corresponding command code can be configured at the embedding position. After the feature extraction module is embedded in the embedding position, the feature extraction code embedded in the feature extraction module is run, and the fixed node can sequentially schedule the corresponding code for feature extraction to perform feature extraction according to the feature extraction method represented by the feature extraction module, thereby extracting the features of the test image.

[0063] It should be noted that the test image can be an image pre-configured by an engineer and stored in a preset location (such as a database). After receiving the feature extraction module, the fixed node can automatically obtain it from the preset location. In addition, it can also be input by the engineer when inputting the model to be tested and the test task. After receiving the test image, the master node can send it to each fixed node.

[0064] S205: The second working node monitors whether the features corresponding to the test task in the extracted features have met the requirements of the test task.

[0065] In the embodiment of the present application, in order to facilitate the second working node to monitor, the fixed node can send the extracted features to a preset database for storage. The second working node only needs to monitor whether there are features corresponding to the test task that meet the requirements of the test task in the database.

[0066] It should be understood that the above description includes parameters such as the test content and test requirements required for the model to be tested in the test task. The test requirements include information such as the type and quantity of features required by the test task, based on which the second working node can determine whether the features corresponding to the test task have met the requirements of the test task. In other words, the second working node can determine whether the features corresponding to the test task match the test requirements.

[0067] The test content includes a specific method of performing the test using the features required by the test task, so that the second working node can perform the test task using the features corresponding to the test task to complete the performance test of the model.

[0068] S206: When the feature corresponding to the test task meets the requirement of the test task, the test task is executed using the feature corresponding to the test task.

[0069] It should be noted that, for a model to be tested, it may correspond to multiple test tasks, and in the embodiment of the present application, multiple test tasks for multiple models to be tested may be configured at the same time.

[0070] Therefore, in order to ensure the reliability of the distributed cluster testing process, engineers can configure the association between relevant test tasks and the model to be tested during input. For example, they can configure a unique model identifier for the model to be tested and carry the model identifier in the relevant test tasks.

[0071] When the first working node extracts the feature extraction module in the model to be tested, it can carry the model identification of the model to be tested and send it to the fixed node, and the fixed node can send the extracted features together with the model identification of the model to be tested to the database for storage. In this way, the second working node can effectively monitor the features corresponding to the test task based on the model identification carried in the test task.

[0072] In an embodiment of the present application, after executing the test task, the second working node can return the test result to the main node, so that the main node can feed back the test result to the user.

[0073] In an embodiment of the present application, after monitoring whether the feature corresponding to the test task in the extracted features has met the requirements of the test task, the fixed node can be notified to stop extracting the feature.

[0074] It should be noted that in the case where multiple models to be tested need to be tested, each fixed node can be configured to extract the features corresponding to each model to be tested in the order in which the feature extraction module is received; in addition, the main node can also allocate the fixed nodes, determine the models to be tested corresponding to each fixed node, and notify each first working node. At this time, each first working node will only send the feature extraction module to the corresponding fixed node.

[0075] Optionally, in order to improve the reliability of the solution of the present application, the master node can be configured to monitor the execution status of work tasks of each working node and fixed node in the distributed cluster, and then when the work task of any working node or fixed node fails to execute, a task restart instruction is sent to the working node or fixed node to restart the failed work task on the working node or fixed node.

[0076] It should be noted that the aforementioned tasks refer to: the first working node extracts the feature extraction module of the model to be tested, the fixed node extracts features of the test image according to the feature extraction module of the model to be tested, and the second working node monitors the extracted features to see whether the features corresponding to the test task have met the requirements of the test task, and when the features corresponding to the test task have met the requirements of the test task, the features corresponding to the test task are used to execute the test task.

[0077] It is worth noting that in the embodiment of the present application, each node in the distributed cluster can be a physical machine. In addition, when the resources of the physical machine are sufficient, each node in the distributed cluster can also be a virtual machine running on the physical machine.

[0078] It is also worth noting that the distributed cluster and model testing method provided in the embodiments of the present application can be applied to various image processing models such as image classification models, image segmentation models, etc., for example, face recognition models.

[0079] The distributed cluster and model testing method provided in the embodiments of the present application allocate the model to be tested and the test task through the master node of the distributed cluster, extract the feature extraction module of the model to be tested through the first working node, extract the feature through the fixed node, and execute the test task through the second working node, so that the entire test link of a model is split into different nodes for automatic processing, and different nodes are configured to perform corresponding functions. When multiple test tasks need to be performed, it is only necessary to assign the corresponding model and test task to the corresponding node, and each node can autonomously perform the corresponding function, thereby realizing the test task and completing the parallel execution of multiple test tasks for the model. This solves the problem that the model testing is currently limited by manpower, the speed is slow, it is easy to make mistakes, and large-scale testing cannot be carried out.

[0080] Embodiment 2:

[0081] To facilitate the explanation of the solution of the embodiment of the present application, the solution of the embodiment of the present application is described below from the perspectives of the working node, fixed node, and master node of the distributed cluster.

[0082] See also Figure 3 As shown, Figure 3 The model testing method applied to any working node of a distributed cluster in an embodiment of the present application is shown, including:

[0083] S301: When receiving a test task assigned by a master node in a distributed cluster, monitoring whether the features corresponding to the test task in the extracted features have met the requirements of the test task.

[0084] In the embodiment of the present application, the extracted features are obtained by extracting the test image by a fixed node in the distributed cluster according to a feature extraction module in the model to be tested.

[0085] S302: When the feature corresponding to the test task has met the requirement of the test task, the test task is executed using the feature corresponding to the test task to obtain a test result.

[0086] In an embodiment of the present application, the working node may also extract a feature extraction module in the model to be tested when receiving the model to be tested assigned by the master node.

[0087] For the specific execution process of the working node, please refer to the relevant content description of the working node in Example 1, which will not be repeated here.

[0088] See also Figure 4 As shown, Figure 4 The model testing method applied to any fixed node of a distributed cluster in an embodiment of the present application is shown, including:

[0089] S401: receiving a feature extraction module in a model to be tested from a working node in a distributed cluster.

[0090] S402: Acquire a test image, and extract features from the test image according to a feature extraction module, so that working nodes in the distributed cluster can perform test tasks based on the extracted features.

[0091] For the specific execution process of the fixed node, please refer to the relevant description of the fixed node in the first embodiment, which will not be repeated here.

[0092] See also Figure 5 As shown, Figure 5 The model testing method applied to the master node of the distributed cluster in an embodiment of the present application is shown, including:

[0093] S501: When receiving the model to be tested and the test task, the model to be tested and the test task are respectively allocated to each working node in the distributed cluster.

[0094] In an embodiment of the present application, the master node will respectively assign the model to be tested and the test task to each working node in the distributed cluster, so that the working node assigned the model to be tested can extract the feature extraction module of the model to be tested, and the working node assigned the test task can use the features corresponding to the test task to execute the test task and obtain the test result when the features corresponding to the test task meet the requirements of the test task.

[0095] S502: Receive the test result of the test task returned by the working node that executes the test task.

[0096] For the specific execution process of the master node, please refer to the relevant content description of the master node in Example 1, which will not be repeated here.

[0097] Embodiment three:

[0098] This embodiment, based on Embodiment 1 and Embodiment 2, takes a case where three performance tests are performed on a face recognition model at the same time as an example to illustrate the solution of the present application.

[0099] See also Figure 6 As shown, Figure 6The overall modules and processing procedures of the distributed cluster are shown, where the solid-line boxes represent the nodes in the distributed cluster, and the dotted-line boxes relative to the nodes represent the operations performed by the corresponding nodes.

[0100] First, input the trained face recognition model and the three required performance test tasks into the master node.

[0101] The master node sends the face recognition model to worker node A according to the idle status of each worker node in the current cluster, and assigns performance test tasks 1, 2, and 3 to worker nodes 1, 2, and 3 respectively.

[0102] Working node A deletes the network module of the face recognition model to obtain a feature extraction module, converts the floating point numbers in the feature extraction module into integers to form a binary module that can be loaded into the memory, and sends it to each fixed node.

[0103] Each fixed node loads the binary module of the feature extraction module into the memory and embeds it into the embedding position of the feature extraction code pre-written in the fixed node.

[0104] Run the feature extraction code embedded in the binary module to extract features from the test image and send them to the preset feature database for storage.

[0105] In an embodiment of the present application, in order to speed up feature extraction, multiple fixed nodes can be configured.

[0106] The second working nodes 1, 2, and 3 monitor the feature types and feature quantities in the feature database respectively. When the feature types and feature quantities meet the test task requirements, the corresponding features are obtained, the corresponding test tasks are run, and the test results are returned to the master node after the test is completed.

[0107] After receiving the results of all test tasks, the master node notifies the user of the entire test results.

[0108] In an embodiment of the present application, the feature database may adopt a NOSQL database so that the fixed node performs data storage and the second working node performs data monitoring and reading.

[0109] Embodiment 4:

[0110] Based on the same inventive concept, the present application also provides three model testing devices respectively applied to the working nodes, fixed nodes and master nodes of the aforementioned distributed cluster. Figures 7 to 9 As shown, Figures 7 to 9The model test devices 100, 200 and 300 corresponding to the processes performed by the working node, the fixed node and the master node in the method shown in the first embodiment are respectively shown. It should be understood that the specific functions of the model test devices 100, 200 and 300 can be referred to the description above. In order to avoid repetition, the detailed description is appropriately omitted here. The model test devices 100, 200 and 300 include at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system of the model test devices 100, 200 and 300. Specifically:

[0111] See also Figure 7 As shown, the model testing device 100 is applied to any working node of a distributed cluster, and includes a first receiving module 101, a monitoring module 102 and a processing module 103. Among them:

[0112] The first receiving module 101 is used to receive the test task assigned by the master node in the distributed cluster;

[0113] The monitoring module 102 is used to monitor whether the features corresponding to the test task in the extracted features have met the requirements of the test task when the first receiving module 101 receives the test task assigned by the master node; the extracted features are obtained by extracting the test image by the fixed node in the distributed cluster according to the feature extraction module in the model to be tested;

[0114] The processing module 103 is used to execute the test task by using the feature corresponding to the test task to obtain a test result when the feature corresponding to the test task meets the requirement of the test task.

[0115] In the embodiment of the present application, the processing module 103 is further configured to return the test result to the master node after the test task is executed.

[0116] In an embodiment of the present application, the first receiving module 101 is also used to receive the model to be tested assigned by the master node, and the processing module 103 is also used to extract a feature extraction module in the model to be tested when the first receiving module 101 receives the model to be tested assigned by the master node, and send the feature extraction module to the fixed node, so that the fixed node can perform feature extraction on the test image.

[0117] In an embodiment of the present application, the processing module 103 is specifically used to convert the floating-point numbers in the feature extraction module into integers after extracting the feature extraction module in the model to be tested, to form a binary module of the feature extraction module, and to send the binary module of the feature extraction module to the fixed node.

[0118] See also Figure 8As shown, the model testing device 200 is applied to any fixed node of the distributed cluster, and includes a second receiving module 201 and an extraction module 202. Among them:

[0119] The second receiving module 201 is used to receive the feature extraction module in the model to be tested transmitted by the working node in the distributed cluster; and to obtain the test image;

[0120] The extraction module 202 is used to extract features from the test image according to the feature extraction module, so that the working nodes in the distributed cluster can perform test tasks based on the extracted features.

[0121] In the embodiment of the present application, the extraction module 202 is specifically used to embed the feature extraction module into a preset embedding position of the feature extraction code; run the feature extraction code embedded in the feature extraction module to perform feature extraction on the test image.

[0122] In the embodiment of the present application, the extraction module 202 is also used to output the extracted features to a preset database for storage.

[0123] See also Fig. 9 As shown, the model testing device 300 is applied to the master node of the distributed cluster, and includes: an allocation processing module 301 and a third receiving module 302. Among them:

[0124] The allocation processing module 301 is used to allocate the model to be tested and the test task to each working node in the distributed cluster respectively when receiving the model to be tested and the test task, so that the working node to which the model to be tested is allocated can extract the feature extraction module of the model to be tested, and send it to the fixed node in the distributed cluster for feature extraction, so that the working node to which the test task is allocated can use the feature corresponding to the test task to execute the test task and obtain the test result when the feature corresponding to the test task meets the requirement of the test task;

[0125] The third receiving module 302 is used to receive the test result of the test task returned by the working node that executes the test task.

[0126] In an embodiment of the present application, the allocation processing module 301 is specifically used to obtain the idle status of each working node in the distributed cluster, and allocate the model to be tested and the test task to the idle working nodes in the distributed cluster respectively; or, obtain the load status of each working node in the distributed cluster, and allocate the model to be tested and the test task to each working node with the lowest load in the distributed cluster in turn.

[0127] In the embodiment of the present application, the allocation processing module 301 is also used to monitor the execution of work tasks of each working node and fixed node in the distributed cluster;

[0128] When a task fails to be executed on any working node or fixed node, a task restart instruction is sent to the working node or fixed node to restart the failed task on the working node or fixed node.

[0129] The task of the working node assigned with the model to be tested is: extracting the feature extraction module of the model to be tested;

[0130] The working task of the fixed node is: extracting features from a preset test image according to the feature extraction module of the model to be tested;

[0131] The task of the working node assigned with the test task is: when the feature corresponding to the test task has met the requirement of the test task, execute the test task using the feature corresponding to the test task to obtain the test result.

[0132] It should be understood that, for the sake of brevity, some of the contents described in the first embodiment will not be repeated in this embodiment.

[0133] Embodiment five:

[0134] This embodiment provides an electronic device, see Fig.10 As shown, it includes a processor 1001, a memory 1002 and a communication bus 1003. Among them:

[0135] The communication bus 1003 is used to realize the connection and communication between the processor 1001 and the memory 1002 .

[0136] The processor 1001 is used to execute one or more programs stored in the memory 1002 to implement the steps in the model testing method executed by the master node or any working node or fixed node in the above-mentioned embodiment 1.

[0137] Understandably, Fig.10 The modules shown are for illustration only. The electronic device may also include Fig.10 More or fewer components as shown, or with Fig.10 The different configurations shown may also include, for example, a data input / output interface, a display screen, a keyboard, and other components.

[0138] This embodiment also provides a readable storage medium, such as a floppy disk, an optical disk, a hard disk, a flash memory, a U disk, an SD (Secure Digital Memory Card) card, an MMC (Multimedia Card) card, etc., in which one or more programs for implementing the above steps are stored, and the one or more programs can be executed by one or more processors to implement the steps in the model testing method executed by the master node or any working node or fixed node in the above embodiment 1. No further details will be given here.

[0139] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0140] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0142] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0143] As used herein, a plurality refers to two or more than two.

[0144] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A model testing method, characterized in that: Applied to any working node of a distributed cluster, including: Upon receiving the model to be tested assigned by the master node in the distributed cluster, the following steps are performed: extracting a feature extraction module in the model to be tested; sending the feature extraction module to a fixed node in the distributed cluster so that the fixed node can perform feature extraction on the test image; Upon receiving a test task assigned by the master node in the distributed cluster, the following steps are performed: monitoring whether the features corresponding to the test task in the extracted features have met the requirements of the test task; the extracted features are obtained by extracting the test image by the fixed node according to the feature extraction module in the model to be tested; when the features corresponding to the test task have met the requirements of the test task, the features corresponding to the test task are used to execute the test task to obtain a test result.

2. The model testing method according to claim 1, characterized in that: The method further comprises: After the test task is executed, the test result is returned to the master node.

3. The model testing method according to claim 1, characterized in that: After extracting the feature extraction module in the model to be tested, the method further includes: Converting floating point numbers in the feature extraction module into integers to form a binary module of the feature extraction module; The sending the feature extraction module to the fixed node in the distributed cluster includes: The binary module of the feature extraction module is sent to the fixed node.

4. A model testing method, characterized in that: Applicable to any fixed node in a distributed cluster, including: Receiving a feature extraction module in the model to be tested from a first working node in the distributed cluster; wherein the first working node is a working node that receives the model to be tested assigned by a master node in the distributed cluster; Acquire a test image, and perform feature extraction on the test image according to the feature extraction module, so that the second working node in the distributed cluster can execute the test task based on the extracted features; wherein, the second working node is a working node that has received the test task assigned by the master node, and when the second working node monitors that the features corresponding to the test task have met the requirements of the test task, the features corresponding to the test task are used to execute the test task and obtain the test results.

5. The model testing method according to claim 4, characterized in that: According to the feature extraction module, feature extraction is performed on the test image, including: Embedding the feature extraction module into the embedding position of the feature extraction code; The feature extraction code embedded in the feature extraction module is run to extract features from the test image.

6. The model testing method according to claim 4 or 5, characterized in that: The method further comprises: The extracted features are exported to the preset database for storage.

7. A model testing method, characterized in that: Applied to the master node of the distributed cluster, including: When receiving the model to be tested and the test task, the model to be tested and the test task are respectively assigned to each working node in the distributed cluster, so that the working node assigned with the model to be tested can extract the feature extraction module of the model to be tested, and send the feature extraction module to the fixed node in the distributed cluster, so that the fixed node can extract features from the acquired test image, so that the working node assigned with the test task can use the features corresponding to the test task to execute the test task and obtain the test result when monitoring that the features corresponding to the test task have met the requirements of the test task; Receive a test result of the test task returned by a working node that executes the test task.

8. The model testing method according to claim 7, characterized in that: Allocating the model to be tested and the test task to the working node of the distributed cluster includes: Obtaining the idle state of each working node in the distributed cluster, and respectively allocating the model to be tested and the test task to the idle working nodes in the distributed cluster; Or, the load condition of each working node in the distributed cluster is obtained, and the model to be tested and the test task are sequentially allocated to each working node with the lowest load in the distributed cluster.

9. The model testing method according to claim 8, characterized in that: The method further comprises: Monitor the execution of work tasks of each working node and fixed node in the distributed cluster; When the execution of a work task on any working node or fixed node fails, a task restart instruction is sent to the working node or fixed node to restart the failed work task on the working node or fixed node.

10. A model testing device, characterized in that: Applied to any working node of a distributed cluster, comprising: a first receiving module, a monitoring module and a processing module; The first receiving module is used to receive the test task assigned by the master node in the distributed cluster; The monitoring module is used to monitor whether the features corresponding to the test task in the extracted features have met the requirements of the test task when the first receiving module receives the test task assigned by the master node; the extracted features are obtained by extracting the test image by the fixed node according to the feature extraction module in the model to be tested; The processing module is used to execute the test task using the feature corresponding to the test task to obtain a test result when the feature corresponding to the test task meets the requirements of the test task; The first receiving module is also used to receive the model to be tested assigned by the master node, and the processing module is also used to extract a feature extraction module in the model to be tested when the first receiving module receives the model to be tested assigned by the master node, and send the feature extraction module to the fixed node, so that the fixed node can perform feature extraction on the test image.

11. A model testing device, characterized in that: Applied to any fixed node of a distributed cluster, comprising: a second receiving module and an extracting module; The second receiving module is used to receive the feature extraction module in the model to be tested transmitted by the first working node in the distributed cluster; and to obtain the test image; wherein the first working node is a working node that receives the model to be tested assigned by the master node in the distributed cluster; The extraction module is used to extract features from the test image according to the feature extraction module, so that the second working node in the distributed cluster can perform the test task based on the extracted features; wherein, the second working node is a working node that has received the test task assigned by the master node. When the second working node monitors that the features corresponding to the test task have met the requirements of the test task, the features corresponding to the test task are used to execute the test task and obtain the test results.

12. A model testing device, characterized in that: Applied in the master node of the distributed cluster, comprising: an allocation processing module and a third receiving module; The allocation processing module is used to allocate the model to be tested and the test task to each working node in the distributed cluster respectively when receiving the model to be tested and the test task, so that the working node assigned with the model to be tested can extract the feature extraction module of the model to be tested, and send the feature extraction module to the fixed node in the distributed cluster, so that the fixed node can extract features from the acquired test image, so that the working node assigned with the test task can use the features corresponding to the test task to execute the test task and obtain the test result when monitoring that the features corresponding to the test task have met the requirements of the test task; The third receiving module is used to receive the test result of the test task returned by the working node that executes the test task.

13. A distributed cluster, characterized in that: include: Master nodes, working nodes and fixed nodes; among them: The master node is configured to, upon receiving the model to be tested and the test task, assign the model to be tested to a first working node among the working nodes, and assign the test task to a second working node among the working nodes; The first working node is configured to extract a feature extraction module in the model to be tested when receiving the model to be tested assigned by the master node, and send the feature extraction module to the fixed node; The fixed node is used to obtain a test image and perform feature extraction on the test image according to the feature extraction module; The second working node is used to monitor whether the features corresponding to the test task have met the requirements of the test task among the extracted features when receiving the test task assigned by the main node, and when the features corresponding to the test task have met the requirements of the test task, use the features corresponding to the test task to execute the test task and obtain the test results.

14. An electronic device, characterized in that: include: Processor, memory and communication bus; The communication bus is used to realize the connection and communication between the processor and the memory; The processor is used to execute one or more programs stored in the memory to implement the model testing method according to any one of claims 1 to 9.

15. A readable storage medium, characterized in that: The readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the model testing method according to any one of claims 1 to 9.