Model determination method and device, equipment and storage medium
By parallel training and evaluation of multiple detection models, the problem of inefficient detection models in the existing technology is solved, efficient training and evaluation of multiple detection models is achieved, and the efficiency of industrial-grade defect detection is improved.
Patent Information
- Application Number
- CN202410920874.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-09
- Publication Date
- 2024-11-19
AI Technical Summary
The existing object detection technology cannot batch train multiple detection models, resulting in inefficient production efficiency of detection technology models, and industrial-grade defect detection tools do not have batch evaluation functions.
The pre-built model registration of the custom detection model is carried out through the pre-built model register, multiple training processes are called for parallel training, multiple trained detection models are obtained and evaluated, the object detection model is determined, and batches of multiple defect detection models are realized Training and evaluation.
It improves model training efficiency and the production efficiency of detection models, provides a method for calculating the evaluation index of defect detection data, and can quickly determine the target detection model with performance in line with needs.
Smart Images

Figure CN118981635A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to, but is not limited to, the field of computer technology, and in particular to a model determination method, apparatus, device, and storage medium. Background Art
[0002] In recent years, the quality inspection business in industrial manufacturing has developed rapidly, which has led to higher and higher requirements for the R&D efficiency and safety of industrial manufacturing business. At the same time, with the continuous breakthroughs in computer vision technology, target detection technology, as an important research direction in the field of computer vision, has made great contributions to industrial manufacturing business and safety monitoring business. At present, target detection technology has been widely used in industrial fields such as automobile manufacturing, robot arm control, equipment surface defect detection and component missing detection.
[0003] However, the relevant target detection technology is unable to perform batch training on multiple detection models, resulting in low production efficiency of the detection technology model, and the existing industrial-grade defect detection tools do not have batch evaluation capabilities. Summary of the invention
[0004] The embodiments of the present application at least provide a model determination method, apparatus, device and storage medium.
[0005] The technical solution of the embodiment of the present application is implemented as follows:
[0006] The present application provides a model determination method, the method comprising:
[0007] Get a custom detection model;
[0008] Registering the custom detection model through a pre-built model registrar to obtain at least two detection models to be trained;
[0009] Calling at least two training processes to perform parallel training on the at least two detection models to be trained, and correspondingly obtaining at least two trained detection models and detection results of each of the trained detection models; wherein each training process implements training of one detection model to be trained;
[0010] Based on the detection results of the at least two trained detection models, performing evaluation processing on the at least two trained detection models to obtain model evaluation results;
[0011] Based on the model evaluation results, a target detection model is determined from the at least two trained detection models.
[0012] The present application provides a model determination device, including:
[0013] Acquisition module, used to obtain custom detection models;
[0014] A model registration module, used to register the custom detection model through a pre-built model registrar to obtain at least two detection models to be trained;
[0015] A parallel training module, used to call at least two training processes to perform parallel training on the at least two detection models to be trained, and correspondingly obtain at least two trained detection models and the detection results of each of the trained detection models; wherein each training process implements the training of one detection model to be trained;
[0016] An evaluation module, configured to perform evaluation processing on the at least two trained detection models based on the detection results of the at least two trained detection models to obtain model evaluation results;
[0017] A determination module is used to determine a target detection model from the at least two trained detection models based on the model evaluation results.
[0018] An embodiment of the present application provides a model determination device, including a processor and a memory, wherein the memory stores a computer program that can be run on the processor, and the processor implements the above-mentioned model determination method when executing the computer program.
[0019] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the above-mentioned model determination method when executed by a processor.
[0020] An embodiment of the present application provides a computer program product, which includes executable instructions stored in a computer-readable storage medium; when a processor of a model determination device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the above-mentioned model determination method is implemented.
[0021] The model determination method, apparatus, device and storage medium provided in the embodiments of the present application, firstly, registers the custom detection model through a pre-built model registrar to obtain at least two detection models to be trained, and calls at least two training processes to perform parallel training on the at least two detection models to be trained, correspondingly obtaining at least two trained detection models and the detection results of each trained detection model, each training process implements the training of a detection model to be trained, so that the embodiments of the present application can realize batch training of multiple defect detection models through a multi-process manner, greatly improving the model training efficiency; secondly, the embodiments of the present application evaluate the at least two trained detection models based on the detection results of the at least two trained detection models to obtain the model evaluation results, which not only provides a method for calculating the evaluation index of defect detection data, but also can realize the batch evaluation of the model detection results; finally, based on the model evaluation results, the target detection model is determined from the at least two trained detection models, so that the embodiments of the present application can quickly determine the target detection model whose performance meets the requirements among the multiple detection models to be trained according to the evaluation results of the multiple detection models to be trained, thereby improving the production efficiency of the detection model.
[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.
[0024] Figure 1 It is a schematic diagram of an application scenario of the model determination method provided in an embodiment of the present application;
[0025] Figure 2 It is a schematic diagram of an implementation flow of a model determination method provided in an embodiment of the present application;
[0026] Figure 3 It is a schematic diagram of an implementation flow of a model determination method provided in an embodiment of the present application;
[0027] Figure 4 It is a schematic diagram of an implementation flow of a model determination method provided in an embodiment of the present application;
[0028] Figure 5 It is a schematic diagram of an implementation flow of a model determination method provided in an embodiment of the present application;
[0029] Figure 6 It is a schematic diagram of a multi-process startup experiment provided in an embodiment of the present application;
[0030] Figure 7 is an example diagram of the calculation of the recall rate and the false alarm rate provided in the embodiment of the present application;
[0031] Figure 8 It is a schematic diagram of the data correctness check operation flow provided by the embodiment of the present application;
[0032] Fig. 9 It is a schematic diagram of the structure of a model determination device provided in an embodiment of the present application;
[0033] Fig.10 It is a hardware entity schematic diagram of a model determination device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0034] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.
[0035] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0036] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0038] In the related technologies, the industrial manufacturing and quality inspection fields have successfully applied defect detection technology to the production process, but there are many problems in the process of use.
[0039] Based on the problems existing in the related art, an embodiment of the present application provides a model determination method. On the one hand, a custom detection model is registered through a pre-built model registrar to obtain at least two detection models to be trained, and at least two training processes are called to perform parallel training on the at least two detection models to be trained, correspondingly obtaining at least two trained detection models and the detection results of each trained detection model. Each training process implements the training of a detection model to be trained, so that the embodiment of the present application can realize batch training of at least two defect detection models through a multi-process manner, greatly improving the model efficiency; secondly, based on the detection results of at least two trained detection models, the embodiment of the present application evaluates and processes at least two trained detection models to obtain model evaluation results, which not only provides a method for calculating evaluation indicators of defect detection data, but also can realize batch evaluation of model detection results; finally, based on the model evaluation results, a target detection model is determined from at least two trained detection models, so that the embodiment of the present application can quickly determine the target detection model whose performance meets the requirements among multiple detection models to be trained according to the evaluation results of multiple detection models to be trained, thereby improving the production efficiency of the detection model.
[0040] The model determination method provided in the embodiment of the present application can be executed by an electronic device such as a model determination device, wherein the electronic device can be a laptop, a tablet computer, a desktop computer, a set-top box, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device) and other types of terminals, and can also be implemented as a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.
[0041] Below, an exemplary application of the model determination device implemented as a server will be described, and the technical solution in the embodiment of the present application will be clearly and completely described in conjunction with the drawings in the embodiment of the present application.
[0042] Figure 1: is a schematic diagram of an application scenario of the model determination method provided in an embodiment of the present application. The model determination system 10 provided in an embodiment of the present application includes a terminal 100, a network 200 and a server 300. When determining the model, the server 300 can adopt the method provided in an embodiment of the present application, obtain a custom detection model through the network 200, register the custom detection model through a pre-built model register, obtain at least two detection models to be trained, call at least two training processes to train at least two detection models to be trained in parallel, and obtain at least two trained detection models and the detection results of each trained detection model, based on the detection results of at least two trained detection models, evaluate the at least two trained detection models, obtain model evaluation results, determine the target detection model from the at least two trained detection models based on the model evaluation results, and the server 300 pushes the target detection model to the terminal 100 through the network 200. Here, the terminal 100 can be a defect detection device, and the defect detection device performs defect detection or component missing detection on the device surface based on the target detection model.
[0043] Figure 2 is a schematic diagram of an implementation flow of a model determination method provided in an embodiment of the present application, such as Figure 2 As shown, the method is implemented through steps S201 to S205:
[0044] Step S201: Obtain a custom detection model.
[0045] In some embodiments, a custom detection model refers to a detection model that can implement custom functions, for example, data enhancement functions. The custom detection model can perform abnormal area detection, defect detection, or other types of target detection. When the existing target detection tool cannot achieve the required detection function, the existing defect detection tool can be connected with the custom management model through a model management tool (such as git submodule) to obtain a target detection model that meets the requirements.
[0046] Step S202: register the custom detection model through a pre-built model registrar to obtain at least two detection models to be trained.
[0047] In some embodiments, the custom detection model includes at least custom configuration files such as model information and model training information. The pre-built model registrar is used to read the custom configuration file of the custom detection model and the standard configuration file of the existing target detection tool, connect the custom configuration file and the standard configuration file to obtain the configuration file of the detection model to be trained, perform model instantiation processing on the configuration file of the detection model to be trained, adjust the model parameters of the model, and obtain at least two detection models to be trained.
[0048] In some embodiments, the model information in the custom configuration file may be the network structure, model parameters, classification loss function, regression loss function, data set configuration, graphics processing unit (GPU) configuration, image normalization parameters, image preprocessing parameters and other configuration information of the model itself; the model training information may be the configuration information of parameters related to the training process such as the optimizer, learning rate, and number of iterations. The pre-built model register reads the configuration information to construct the model to be detected. Here, by modifying the parameters in the configuration information, multiple different detection models to be trained can be obtained.
[0049] In some embodiments, a pre-built model registry can complete the conversion from strings to classes, so that you only need to write the model information of the custom model and the parameter information during training into a configuration file, and then use the registry to instantiate the configuration file to obtain the detection model to be trained.
[0050] Step S203, calling at least two training processes to perform parallel training on the at least two detection models to be trained, and correspondingly obtaining at least two trained detection models and the detection results of each of the trained detection models; wherein each training process implements the training of one detection model to be trained.
[0051] In some embodiments, a sample data set may be input into at least two detection models to be trained for parallel training, or different sample data sets may be input into at least two detection models to be trained for parallel training.
[0052] In an embodiment of the present application, a multi-process approach is adopted to perform parallel training for at least two detection models to be trained simultaneously, and each training process implements the training of one detection model to be trained, thereby obtaining at least two trained detection models and the detection results of each trained detection model.
[0053] In some embodiments, the detection model to be trained includes at least one model parameter, and the model parameter may be a task name of the detection model to be trained, the number of GPUs required for model training, and the model training task priority type and other parameters. For each detection model to be trained, the model parameters such as the number of GPUs and the task priority type of the detection model to be trained may be modified to obtain at least two experimental models corresponding to each detection model to be trained. The embodiment of the present application may use multiple processes to simultaneously train at least two experimental models corresponding to each detection model to be trained, thereby obtaining each trained experimental model and the training results corresponding to each trained experimental model.
[0054] In some embodiments, step S203 may be implemented by steps S2031 to S2033:
[0055] Step S2031, setting the model parameters to obtain at least two experimental models corresponding to each of the detection models to be trained and the number of experiments of the experimental models; wherein the model parameters corresponding to each experimental model are different from the model parameters corresponding to other experimental models.
[0056] Step S2032: calling training processes having the same number as the number of experiments, and performing parallel training on the at least two experimental models based on the sample data set to obtain at least two trained experimental models and experimental results corresponding to each experimental model.
[0057] In an embodiment of the present application, by modifying model parameters such as the number of GPUs required for model training and the model training task priority type, at least two experimental models corresponding to each detection model to be trained can be obtained. For example, a custom detection model is registered through a pre-built model registrar to obtain six detection models to be trained. The model parameters of each detection model to be trained are modified to obtain five experimental models corresponding to each detection model to be trained. In this way, the number of experimental models is thirty, and thirty training processes are called. The thirty experimental models are trained in parallel based on the sample data set to obtain thirty trained experimental models and experimental results corresponding to each experimental model.
[0058] Step S2033: Determine the at least two trained experimental models and the experimental results corresponding to each of the experimental models as the detection results of the at least two trained detection models and each of the trained detection models.
[0059] In an embodiment of the present application, at least two trained experimental models corresponding to each model to be detected are determined as detection models after each model to be detected is trained, and the experimental results of at least two trained experimental models corresponding to each model to be detected are determined as detection results of each of the trained detection models.
[0060] Step S204: Based on the detection results of the at least two trained detection models, the at least two trained detection models are evaluated to obtain model evaluation results.
[0061] In some embodiments, performing evaluation processing on the trained detection model means evaluating the detection results of the trained detection model. Here, the evaluation methods for performing evaluation processing may include evaluation methods at the instance level, image level, evaluation methods for converting instance level detection results to image level detection results, and evaluation methods for supporting datasets with semantic labels.
[0062] In some embodiments, an image sample is obtained, the image sample includes image data with abnormal regions and abnormal region types marked, the image data is formatted to obtain a sample data set, and multiple experimental models corresponding to the detection model to be detected are trained based on the sample data set to obtain the detection results of the trained detection model, and the detection results can be sample data with abnormal regions detected by each experimental model. The recall rate of each trained detection model can be determined based on the number of abnormal data samples detected by the trained detection model and the number of abnormal images with abnormal regions in the image samples; the false alarm rate of each trained detection model can be determined based on the number of non-abnormal data samples detected by the trained detection model and the number of non-abnormal images without abnormal regions in the image samples; and at least two trained detection models are evaluated based on the recall rate and the false alarm rate to obtain model evaluation results.
[0063] In some embodiments, the evaluation method provided by the embodiments of the present application can select the abnormal area with the largest defect score (score) in the same type of abnormal area on each acquired picture as the abnormal score of this type of abnormal area on this picture. When performing the evaluation, the recall rate @ false alarm rate (recall@fpr) of each type of abnormal area is counted, where @ is used to connect the two indicators of recall rate (recall) and false alarm rate (fpr), and recall@fpr indicates the recall index of the experimental results under the fpr index. For example: when recall@fpr is 99%@1%, it means that when fpr reaches 1%, recall reaches 99%.
[0064] In some embodiments, the recall rate is calculated by formula (1), and the false alarm rate is calculated by formula (2):
[0065]
[0066] Among them, m is the number of defective samples reported correctly; n is the total number of defective samples; a is the number of non-defective samples reported falsely; and b is the total number of non-defective samples.
[0067] In some embodiments, based on the detection results of at least two trained detection models, multiple processes can be used to evaluate at least two trained detection models to obtain model evaluation results.
[0068] Step S205: Based on the model evaluation result, determine the target detection model from the at least two trained detection models.
[0069] In an embodiment of the present application, standard deviation calculation or mean calculation can be performed on the model evaluation results of all experimental models corresponding to each trained detection model, and the trained detection model that meets the standard deviation condition or mean condition is determined as the target detection model. The target detection model refers to a detection model whose detection results obtained after testing the sample data set are within the error range of the abnormal area results marked in the sample data set, that is, a trained detection model that meets the detection requirements.
[0070] The custom detection model is registered through a pre-built model registrar to obtain at least two detection models to be trained, and at least two training processes are called to perform parallel training on the at least two detection models to be trained, correspondingly obtaining at least two trained detection models and the detection results of each trained detection model. Each training process implements the training of a detection model to be trained, so that the embodiment of the present application can realize batch training of at least two defect detection models through a multi-process manner, greatly improving the model efficiency; secondly, based on the detection results of at least two trained detection models, the embodiment of the present application evaluates and processes at least two trained detection models to obtain model evaluation results, which not only provides a method for calculating evaluation indicators of defect detection data, but also can realize batch evaluation of model detection results; finally, based on the model evaluation results, a target detection model is determined from at least two trained detection models, so that the embodiment of the present application can quickly determine the target detection model whose performance meets the requirements among multiple detection models to be trained based on the evaluation results of multiple detection models to be trained, thereby improving the production efficiency of the detection model.
[0071] Figure 3 is a schematic diagram of an implementation flow of a model determination method provided in an embodiment of the present application, such as Figure 3 As shown, obtaining at least two detection models to be trained can be achieved through steps S301 to S303:
[0072] Step S301: Obtain model information and training information of the custom model.
[0073] Step S302: Determine a custom configuration file of the custom model according to the model information and the training information.
[0074] In some embodiments, the model information may be the network structure, model parameters, classification loss function, regression loss function, data set configuration, GPU configuration, image normalization parameters, image preprocessing parameters, and other configuration information of the model itself; the model training information may be the configuration information of parameters related to the training process, such as optimizer, learning rate, and number of iterations. These configuration information are determined as the custom configuration file of the custom model.
[0075] Step S303: read the custom configuration file through the pre-built model registrar, and register the custom detection model based on the custom configuration file to obtain the at least two detection models to be trained.
[0076] In some embodiments, the existing target detection tool may have a standard configuration file, and the standard configuration file of the existing target detection tool may be connected with the custom configuration file of the custom model to obtain a new configuration file, so that the detection model to be trained obtained can have both the functions of the existing target detection tool and the customized functions.
[0077] In some embodiments, the existing target detection tool can be managed through a submodule (submodule module), and the custom model is connected with the existing target detection tool to obtain a detection model to be trained.
[0078] In some embodiments, the model determination method provided in the embodiments of the present application may also obtain a standard configuration file of the standard detection model. Based on the obtained standard configuration file of the standard detection model, step S303 may be implemented through steps S3031 to S3032:
[0079] Step S3031: Connect the custom configuration file and the standard configuration file to obtain a configuration file.
[0080] Step S3032: Based on the custom detection model, the configuration file is instantiated to obtain at least two detection models to be trained; wherein the model parameters of each detection model to be trained are different from the model parameters of other detection models to be trained.
[0081] In some embodiments, the configuration file is a text file composed of a series of variables, and the custom configuration file and the standard configuration file are connected to obtain a configuration file corresponding to the detection model to be trained. The configuration file corresponding to the detection model to be trained is read based on a pre-built model register, and the configuration file is instantiated to obtain the detection model to be trained. Here, the model training parameters such as the number of GPUs and the task priority type in the configuration file corresponding to the detection model to be trained can be modified to obtain at least two detection models to be trained, and the model parameters of each detection model to be trained are different from the model parameters of other detection models to be trained.
[0082] The embodiment of the present application determines the detection model to be trained by adding a custom model to the target detection tool managed by the submodule module. By connecting the third-party target detection tool and the custom model, not only can different third-party target detection tools be used efficiently and conveniently, but also custom operations and requirements can be added, so that the detection model to be trained meets the target detection requirements.
[0083] In some embodiments, when at least two to-be-trained detection models are trained in parallel, the training is performed based on a sample data set. Based on the above embodiments, Figure 4 is a schematic diagram of an implementation flow of a model determination method provided in an embodiment of the present application, such as Figure 4 As shown, determining the sample data set can be achieved through steps S401 to S402:
[0084] Step S401: Obtain a grayscale mask sample image, where the grayscale mask sample image includes pixel information of an abnormal area.
[0085] In some embodiments, when training the detection model to be trained, the input data is data in polygon mask format (i.e., mask in polygons data format). The content stored in the mask in polygons format is the contour information of each type of abnormal area in the image. The data format is a list, and each element in the list stores the contour information of each abnormal area. For example, the list is [x 0 ,y 0 , x 1 ,y 1 ,...x n-1 ,y n-1 , x n ,y n ], where x and y represent the horizontal and vertical coordinates of the abnormal area contour, respectively. However, the mask in polygons data format cannot be obtained directly, and needs to be converted from image format data. Therefore, it is necessary to obtain a grayscale mask sample image that has been labeled with the abnormal area type and abnormal area location (such as a mask in grayscale image data format, which stores the pixel-level labels of each type of abnormal area in a single-channel image). By converting the grayscale mask sample image, you can get the data in polygonal mask format.
[0086] In some embodiments, the obtained grayscale mask sample image with the abnormal region type and abnormal region position marked may have data defects or the resolution is not a resolution that the model can recognize. Therefore, it is necessary to perform data preprocessing on the obtained grayscale mask sample image. Data preprocessing can be achieved through steps S4011 to S4016:
[0087] Step S4011, obtaining at least two initial grayscale mask images.
[0088] Here, the initial grayscale mask image may be a grayscale mask image in which the abnormal region type and the abnormal region position have been marked.
[0089] Step S4012: performing data reading processing on the at least two initial grayscale mask images to obtain a data reading result.
[0090] Step S4013: in response to the data reading result indicating that part of the initial grayscale mask image cannot be read, correct the part of the initial grayscale mask image to obtain at least two readable grayscale mask images; data of the readable grayscale mask image can be read.
[0091] Step S4014: determine the resolution of each readable grayscale mask image.
[0092] Step S4015: In response to the resolution of a partially readable grayscale mask image in each readable grayscale mask image being different from a preset resolution, performing resolution correction on the partially readable grayscale mask image to obtain at least two grayscale mask images; the resolution of the grayscale mask image is the same as the preset resolution.
[0093] Step S4016: Determine the at least two grayscale mask images as grayscale mask sample images.
[0094] In some embodiments, performing data reading processing on the initial grayscale mask image means detecting whether the grayscale mask data can be read according to the specified address. The imread function provided by Open CV can be used to read the data. If the read result is not None, it means that the initial grayscale mask image is readable. Here, it is detected whether the data address of each grayscale mask image can be read to obtain the data reading result of each grayscale mask image. The data reading result can be "can be read" or "cannot be read".
[0095] When the data reading result of part of the initial grayscale mask image shows that the data is unreadable, the reading address of the unreadable data can be modified so that the unreadable data can be read according to the specified address, thereby making each initial grayscale mask image readable according to the specified address, and obtaining at least two readable grayscale mask images that can be read.
[0096] In some embodiments, when training the model to be detected, the grayscale mask image corresponding to the input sample data set has a preset resolution. Therefore, the embodiment of the present application needs to detect the resolution of the readable grayscale mask image. When the resolution of some readable grayscale mask data among at least two readable grayscale mask data is different from the preset resolution, the resolution of the part of the readable grayscale mask data is corrected so that the resolution of all the readable grayscale mask data is the same as the preset resolution, and at least two grayscale mask images are obtained, and the two grayscale mask images are determined as grayscale mask sample images.
[0097] In some embodiments, when the number of initial grayscale mask images is too large, the initial mask images may be grouped to obtain multiple initial grayscale mask image groups, and multiple initial grayscale mask image groups may be preprocessed using a multi-process method to improve the efficiency of data preprocessing. In some embodiments, determining the grayscale mask sample image may also be implemented through steps S41 to S47:
[0098] Step S41, obtaining at least two initial grayscale mask images.
[0099] Step S42: performing grouping processing on the at least two initial grayscale mask images to obtain at least one initial grayscale mask image group and the number of groups of the initial grayscale mask image group.
[0100] Step S43: calling data reading processes having the same number as the number of the groups, performing data reading processing on the initial grayscale mask images in each initial grayscale mask image group, and obtaining data reading results.
[0101] Step S44: In response to the data reading result indicating that the data of part of the initial grayscale mask image cannot be read, correction processing is performed on the part of the initial grayscale mask image to obtain a readable grayscale mask image group with the same number as the number of groupings; the data of the readable grayscale mask image in the readable grayscale mask image group can be read.
[0102] Step S45: for each readable grayscale mask image group, determine the resolution of each readable grayscale mask image in the readable grayscale mask image group.
[0103] Step S46: In response to the fact that the resolution of some readable grayscale mask images in the readable grayscale mask image group is different from the preset resolution, a resolution correction process is called to perform resolution correction on the some readable grayscale mask images to obtain grayscale mask image groups with the same number as the number of groupings, and the resolution of the grayscale mask images in the grayscale mask image group is the same as the preset resolution.
[0104] Step S47: Determine the grayscale mask image in the grayscale mask image group as a grayscale mask sample image.
[0105] In some embodiments, the initial grayscale mask image can be grouped to obtain at least one initial grayscale mask image group and a corresponding number of groups. The same number of processes as the number of groups are used to perform data readability detection and resolution detection on each group of initial grayscale mask image groups to obtain at least one group of grayscale mask image groups. The grayscale mask data in at least one group of grayscale mask images is determined as a grayscale mask sample image. In this way, the embodiment of the present application uses multiple processes to implement data preprocessing, efficiently checks the readability and resolution of image data, and improves the efficiency of data processing.
[0106] Step S402: Perform data conversion processing on the grayscale mask sample image according to the pixel information to obtain a sample data set.
[0107] In some embodiments, data conversion processing of the grayscale mask sample image refers to converting the obtained grayscale mask sample image into a data sample that can be input into and out of the model. Here, the data sample format that can be input into the model is polygonal mask data (i.e., mask in polygons data format), so the embodiment of the present application provides a data conversion method to convert the grayscale mask sample image into polygonal mask data. In some embodiments, data conversion is implemented by steps S4021 to S4023:
[0108] Step S4021: Determine contour information of the abnormal area in the grayscale mask sample image according to pixel information of the abnormal area in the grayscale mask sample image.
[0109] Step S4022: Determine the coordinate information of each contour point of the abnormal area from the contour information.
[0110] Step S4023: Generate the sample data set according to the coordinate information; the sample data set at least includes polygon mask data.
[0111] In some embodiments, the preprocessed grayscale mask sample image stores pixel information of different types of abnormal regions, and the contour points of the abnormal region and the coordinates of each contour point on the image can be determined based on the pixel information of the abnormal region, wherein the coordinates of each contour point can be [x 0 ,y 0 ], the abnormal area contour coordinates on each image form an abnormal area contour coordinate list, and multiple grayscale mask sample images are correspondingly converted into multiple abnormal area contour coordinate lists, that is, multiple sample data, to form a sample data set input to the model to be detected.
[0112] In some embodiments, the conversion logic between the grayscale mask image and the polygonal mask data is: extracting the contour of the grayscale mask image can obtain the contour coordinates of the polygonal mask data, or filling the range defined by the contour coordinates of the polygonal mask data to obtain the grayscale mask image. Here, after the polygonal mask data restores the grayscale mask image, there will be some pixel differences with the original grayscale image. The embodiment of the present application will count the number of difference pixels and judge the quality of the conversion between the two masks according to actual needs.
[0113] The embodiment of the present application adopts multiple processes to implement data preprocessing of sample images, efficiently checks the readability and resolution of image data, and improves the efficiency of data processing; at the same time, it provides a data format conversion method to realize the conversion between grayscale mask data format and polygon mask data format, providing a basis for model training.
[0114] In some embodiments, the recall rate of each trained detection model can be determined according to the number of abnormal data samples detected by the trained detection model and the number of abnormal images with abnormal areas in the image samples; the false alarm rate of each trained detection model can be determined according to the number of non-abnormal data samples detected by the trained detection model and the number of non-abnormal images without abnormal areas in the image samples; at least two trained detection models are evaluated according to the recall rate and the false alarm rate to obtain model evaluation results. Based on the above embodiments, Figure 5 is a schematic diagram of an implementation flow of a model determination method provided in an embodiment of the present application, such as Figure 5 As shown, step S204 can be implemented through steps S501 to S505:
[0115] Step S501: Determine the number of abnormal images of grayscale mask images with abnormal regions and the number of normal images of grayscale mask images without abnormal regions in the grayscale mask sample images.
[0116] In some embodiments, the acquired grayscale mask sample image is an image with annotated abnormal region type and position, that is, the image with abnormal region is certain. Therefore, the number of abnormal images of grayscale mask images with abnormal regions and the number of non-abnormal images of grayscale mask images without abnormal regions in the grayscale mask sample image can be obtained based on the annotated data.
[0117] In an embodiment of the present application, the number of abnormal images of grayscale mask images with abnormal areas and the number of non-abnormal images of grayscale mask images without abnormal areas in the grayscale mask sample images can also be determined based on the pixel information of the abnormal area in at least one grayscale mask image.
[0118] Step S502: Determine the number of abnormal data samples with abnormal areas and the number of non-abnormal data samples without abnormal areas output by each trained detection model according to the detection results of the at least two trained detection models.
[0119] In an embodiment of the present application, a sample data set is input into at least two detection models to be trained. After training, each trained detection model outputs the number of abnormal data samples with abnormal areas and the number of non-abnormal data samples without abnormal areas in the sample data set.
[0120] Step S503: Determine the recall rate of each trained detection model according to the number of abnormal data samples and the number of abnormal images.
[0121] Step S504: Determine the false alarm rate of each trained detection model according to the number of non-abnormal data samples and the number of non-abnormal images.
[0122] In the embodiment of the present application, when performing the evaluation, the recall rate @ false alarm rate (recall@fpr) of each type of abnormal area in the statistical sample set can be calculated, wherein the recall rate is calculated by formula (3) and the false alarm rate is calculated by formula (4):
[0123]
[0124] Among them, m is the number of defective samples reported correctly; n is the total number of defective samples; a is the number of non-defective samples reported falsely; and b is the total number of non-defective samples.
[0125] Here, the number of correctly reported defective samples and the number of falsely reported non-defective samples can be obtained by comparing the number of abnormal images of the grayscale mask image with abnormal areas marked in the grayscale mask image and the number of non-abnormal images of the grayscale mask image without abnormal areas.
[0126] Step S505: According to the recall rate and the false alarm rate, the at least two trained detection models are evaluated to obtain a model evaluation result.
[0127] In some embodiments, each model to be detected corresponds to multiple experimental models, and the evaluation processing of the trained detection model may refer to evaluating the multiple experimental models corresponding to the trained detection model, calculating the mean or standard deviation of the evaluation results of the multiple experimental models, and obtaining the model evaluation result of the trained detection model.
[0128] Step S5051: According to the recall rate and false alarm rate of each experimental model, the experimental model is evaluated and processed to obtain the experimental evaluation results of each experimental model.
[0129] In the embodiment of the present application, the experimental evaluation result of each experimental model may be the recall rate@false alarm rate (recall@fpr) of each type of abnormal region.
[0130] Step S5052: Calculate the standard deviation of the experimental evaluation results of at least two groups of experimental models corresponding to each of the trained detection models to obtain the standard deviation evaluation results of each of the trained detection models.
[0131] Step S5053: determine the standard deviation evaluation result of each trained detection model as the model evaluation result of the trained detection model.
[0132] In some embodiments, the standard deviation of the experimental evaluation results of the multiple experimental models corresponding to each trained detection model can be calculated to determine the degree of discreteness between the detection results of the multiple experimental models corresponding to each trained detection model. In some embodiments, among at least two trained detection models, the trained detection model whose standard deviation evaluation result satisfies a preset standard deviation condition is determined as the target detection model. For example, the preset standard deviation condition can be a preset standard deviation threshold, and the trained detection model whose standard deviation evaluation result is less than the preset standard deviation threshold is determined as the target detection model.
[0133] The embodiment of the present application provides a model evaluation method, which can evaluate the trained detection model through the detection results of the trained detection model, and also adopts a multi-process method to evaluate multiple detection models at the same time, and compares the advantages and disadvantages of multiple detection models according to the evaluation results to determine the target detection model that meets the requirements. In this way, the embodiment of the present application trains and evaluates the model through multiple processes, improves the model production efficiency, and can also train a detection model that meets the requirements.
[0134] The embodiment of the present application further provides an application of a model determination method in an actual scenario.
[0135] Defect detection technology is a more refined research field of target detection technology, which refers to the location and classification of defects such as flaws, breakages and missing parts in industrial parts. The industrial manufacturing and quality inspection fields have successfully applied defect detection technology to production processes, but there are many problems in the process of use. Therefore, the embodiment of this application proposes an efficient and scalable industrial-grade defect detection technology and tool to optimize the problems of defect detection tools currently being used in the industrial field.
[0136] In order to solve the problem that the official defect detection tool code base version switching is not convenient enough and the custom extension module cannot be used in conjunction with the code version switching function of the official defect detection tool at the same time, the embodiment of the present application adopts a model management tool (for example, git submodule) and a custom extension model to manage the defect detection tool, so as to achieve efficient switching of the official defect detection tool, and at the same time, the custom function can be implanted in a friendly manner to make the defect detection tool convenient to manage, so as to solve the problem of clumsy and inefficient tool management in the prior art. Among them, git is a code version control tool, and git introduces submodules (submodule modules) for managing third-party code bases.
[0137] The embodiment of the present application connects the official defect detection tool with the custom extension module through a model management tool (for example, git submodule) for training the defect detection model. Here, the official defect detection tool and the custom extension module managed by the submodule module can be used to maintain the defect detection tool, so as to achieve efficient switching of the official defect detection tool. At the same time, custom functions can be implanted on the basis of the official defect detection tool to facilitate the management of the defect detection tool.
[0138] In some embodiments, in the defect detection tools required by industrial vision, general defect detection methods are implemented using official defect detection tools, and model management tools (for example, git submodule) can easily switch the detection tool versions, so that different versions of official tools can be used concisely and efficiently. For defect detection methods not supported by the official defect detection tools, custom extension modules are used to implement functions that are not supported by the official defect detection tools. In this way, the connection between different versions of official defect detection tools and custom extension modules can be achieved. The submodule module is used to manage the official defect detection tools, and the official defect detection tools are connected to the custom extension modules to form an industrial-grade defect detection tool for training defect detection models. The submodule module and the custom extension module are two relatively independent modules, and these two modules simultaneously implement their respective functions in the industrial-grade defect detection tool.
[0139] In some embodiments, a registrar can be constructed using the characteristics of a computer programming language (e.g., Python). In a custom extension module, a registrar is used to register custom functions. The registered functions can be used in a defect detection tool. For example, when adding a custom DetNet network to a custom extension module, a model registrar must first be defined. The functions of the registrar essentially use the syntax characteristics of a Python decorator. In some embodiments, a simple model registrar Python language implementation example is as follows:
[0140]
[0141]
[0142] Secondly, you need to build a model registry (model registry), for example, model_registry = Registry ("model"). Next, use model_registry to register the custom DetNet, and the Python language is implemented as follows:
[0143]
[0144] After registering the custom DetNet, when you need to use DetNet in a custom extension module, you can get an instance of the DetNet network through det_net = model_registry.build("DetNet").
[0145] In some embodiments, in an industrial-grade defect detection tool, two registrars can be implemented in a custom extension module, namely: a data enhancement registrar (AUGMENTATION_REGISTRY) and an evaluation function registrar (EVALUATOR_REGISTRY), which are respectively used to register the data enhancement class and the evaluation function class implemented in the custom extension model.
[0146] In some embodiments, the related technology cannot perform batch training on multiple models, resulting in low production efficiency of defect detection technology models. The embodiments of the present application implement batch training on multiple models to solve the problem of low model production efficiency in the prior art.
[0147] In the embodiment of the present application, in order to realize the batch training function, multiple data sets in the same database (such as the benchmark database) can be trained at the same time, multiple groups of experiments can be started at the same time for the same data set, and the task name (Job-name) of the customized experiment is supported. Tasks with different priorities are supported, such as normal priority (normal) and hungry priority (urgent-kill). When the task is of normal priority, if no computing resources are found after the task is submitted, the task is arranged in the queue for queuing. When the task is of hungry priority, if no computing resources are found after the task is submitted, the computing resources of normal priority are preempted.
[0148] The embodiment of the present application uses multiple processes to implement batch training of the model. Starting batch training can be implemented through steps S1 to S3:
[0149] Step S1: Determine the configuration file for each experiment.
[0150] Step S2: Determine the batch training task according to the configuration file of each experiment.
[0151] Step S3: Start the batch training task.
[0152] In some embodiments, the configuration file may be a markup language file (e.g., a file in YAML format). The embodiment of the present application adopts a multi-process approach, and experiments can be started for multiple models at the same time, and multiple groups of experiments can be started for the same model at the same time. Here, the batch training task has multiple functions, for example, the training of multiple models can be started for the same data set at the same time; multiple groups of experiments can be started for the same model at the same time; the task name (Job-name) of the customized experiment can be supported; and multiple training task types can be supported.
[0153] In some embodiments, the date of the batch training task and the name of the database (e.g., benchmark dataset) can be specified, where the benchmark dataset contains two datasets. A group of experiments is started for each of the two datasets, and the name of each experiment, the experiment training YAML, the task name (Job-name), the number of GPUs required for the experiment, and the experiment task priority type are specified.
[0154] For example, you can start 5 experiments for 6 data sets. The experimental configurations of these 6 data sets are as follows:
[0155] name:C4_dianlianjiesongchi
[0156] train_yaml:configs / bm_phase1 / BADv0.5.0 / pod4.2.0 / C4_dianlianjiesongchi / pod.yaml
[0157] job_name:R-SC210079.001_dljsch
[0158] GPUs:8
[0159] urgent_kill:False
[0160] repeat:5
[0161] -name:C4_diaoxiansunshang_yingwan
[0162] train_yaml:configs / bm_phase1 / BADv0.5.0 / pod4.2.0 / C4_diaoxiansunshang_yingwan / pod.ya ml
[0163] job_name:R-SC210079.001_dxsshyw
[0164] gpus:4
[0165] urgent_kill:False
[0166] repeat:5
[0167] -name:C4_jueyuanzi_defect4
[0168] train_yaml:configs / bm_phase1 / BADv0.5.0 / pod4.2.0 / C4_jueyuanzi_defect4 / pod.yaml
[0169] job_name:R-SC210079.001_jyzdef4
[0170] gpus:4
[0171] urgent_kill:False
[0172] repeat:5
[0173] -name:C4_dianlianjiesunshang_duangu
[0174] train_yaml:
[0175] configs / bm_phase1 / BADv0.5.0 / pod4.2.0 / C4_dianlianjiesunshang_duangu / pod.yaml
[0176] job_name:R-SC210079.001_dljsshdg
[0177] gpus:8
[0178] urgent_kill:False
[0179] repeat:5
[0180] -name:C4_dianqilianjietiaoxiantuoluo
[0181] train_yaml:
[0182] configs / bm_phase1 / BADv0.5.0 / pod4.2.0 / C4_dianqilianjietiaoxiantuoluo / pod.yaml
[0183] job_name:R-SC210079.001_dqljtxtl
[0184] gpus:4
[0185] urgent_kill:False
[0186] repeat:5
[0187] -name:C4_fangfenglaxiansunshang
[0188] train_yaml:configs / bm_phase1 / BADv0.5.0 / pod4.2.0 / C4_fangfenglaxiansunshang / pod.yamljob_name:R-SC210079.001_fflxssh
[0189] GPUs:8
[0190] urgent_kill:False
[0191] repeat:5
[0192] Based on the above experimental configuration, 30 (5*6=30) processes will be started after the batch training script is executed. Each process is responsible for starting one experiment, and finally 30 experiments are started. Figure 6 As shown, Figure 6 is a schematic diagram of a multi-process startup experiment provided in an embodiment of the present application, Figure 6 There are 5 experimental groups (models to be tested), each experimental group corresponds to 5 experiments (experimental models), and each experiment calls a process to train the experiment. For example, experimental group 1 corresponds to five experiments, namely, experiments 11, 12, 13, 14 and 15. Each experiment corresponds to a training process, process 11 corresponds to experiment 11, process 12 corresponds to experiment 12, process 13 corresponds to experiment 13, process 14 corresponds to experiment 14, and process 15 corresponds to experiment 15, thereby realizing multi-process training.
[0193] In some embodiments, in response to the problem that industrial-grade defect assessment indicators in current industrial target detection tools are incomplete, embodiments of the present application provide defect detection assessment indicators for multiple business requirements to solve the problem of incomplete industrial-grade defect assessment indicators in the prior art.
[0194] The embodiments of the present application provide four evaluation indicators and methods, which can respectively provide support for the evaluation of on-site business detection results and the evaluation of model effects on the research side. Among them, the first is an evaluation method that supports the conversion of defect detection instance-level detection results into image-level detection results; the second is an evaluation method that supports target detection, defect detection instance-level and image-level evaluation required by the high-speed rail business side; the third is an evaluation method that supports target detection, defect detection instance-level and image-level evaluation required by the proof of concept (POC) and automated optical inspection (AOI) business side; the fourth is an evaluation method that supports data sets with semantic labels.
[0195] In some embodiments, the evaluation method for converting the detection results of the defect detection instance level to the detection results of the image level can be performed by selecting the abnormal area with the largest defect score (score) in the same type of defects (i.e., abnormal area) on each acquired image as the abnormal score of this type of abnormal area on this image. During the evaluation, the recall rate @ false alarm rate (recall@fpr) of each type of defect is calculated, where the recall rate is calculated by formula (5) and the false alarm rate is calculated by formula (6):
[0196]
[0197] Among them, m is the number of defective pictures reported correctly; n is the total number of defective pictures; a is the number of non-defective pictures reported falsely; and b is the total number of non-defective pictures.
[0198] In the embodiment of the present application, the target detection, defect detection instance level and image level evaluation methods required by the high-speed rail business side redefine the calculation method of recall and the calculation method of fpr in the target detection and defect detection instance level evaluation, where recall is calculated by formula (7) and fpr is calculated by formula (8):
[0199]
[0200] Among them, m1 is the number of correctly reported instances; n1 is the number of actually labeled instances; a1 is the number of falsely reported instances; and b1 is the number of unlabeled instance images + the number of actually labeled instances.
[0201] Figure 7 is an example diagram of the recall and fpr calculation provided in the embodiment of the present application, such as Figure 7As shown, dti is a defect instance, sorted from large to small by score field, i = 1, ..., n; tps is an array of length 10, with a value range of {0, 1}, tps[i] is 0 for dti prediction error, tps[i] is 1 for dti prediction correct; fps is an array of length 10, with a value range of {0, 1}, fps[i] is 0 for dti prediction correct, fps[i] is 1 for dti prediction error; tp is the prefix sum of the tps array, and fp is the prefix sum of the fps array. recall is calculated by formula (9), and fpr is calculated by formula (10):
[0202]
[0203] Among them, tp is the prefix sum of the tps array; m tps is the length of the tps array; fp is the prefix sum of the fps array; m fps is the array length of the fps array.
[0204] In some embodiments, the target detection and defect detection instance-level evaluation indicators recall and fpr required by the POC and AOI business side are consistent with the definitions in machine learning. Recall is the number of correctly detected instances of such defects / the number of all labeled instances of such defects; fpr is the number of incorrectly detected instances of such defects / the number of all labeled instances of non-such defects.
[0205] In some embodiments, whether it is a data set for landing business or a data set on the research side, it usually contains some semantic labels, that is, pixel-level labels of defects. At this time, it is necessary not only to determine the specific location and category of the defect, but also to determine which pixels belong to which category of defect. For the evaluation of detection results containing semantic labels, there are 2 types of evaluation indicators: (Pixel Accuracy, Pixel Acc.) and mean Intersection over Union (mIoU). Among them, the calculation method of Pixel Acc. is the total number of pixels predicted correctly for all categories / the total number of pixels labeled for all categories.
[0206] In the embodiment of the present application, before calculating mIoU, it is necessary to calculate the intersection-over-union ratio (IoUi) of each class (classi). The calculation method of IoUi is shown in formula (11):
[0207]
[0208] Among them, x i y is the pixel predicted by classi; i The real pixels for classi.
[0209] In some embodiments, mIoU is the mean of the IoUs of all categories, as shown in formula (12):
[0210]
[0211] Here, n is the number of categories.
[0212] In some embodiments, in order to solve the problem that the detection results of multiple models on the test set cannot be uniformly evaluated at the same time, the embodiments of the present application implement a unified batch evaluation of the detection results of multiple models on the test set to solve the problem of incomplete evaluation indicators in the prior art.
[0213] The batch evaluation function provided in the embodiment of the present application can evaluate the defect detection results of multiple data sets at the same time, and can evaluate multiple groups of experimental results of the same data set at the same time and calculate the minimum, maximum, mean and standard deviation of the multiple groups of experimental results.
[0214] After the batch training task is completed, the detection results of each experimental model can be evaluated by the four methods in the above embodiments.
[0215] In some embodiments, starting a batch evaluation task may be achieved through steps S10 to S30:
[0216] Step S10: Obtain the detection results of each batch of training models.
[0217] Step S20: Determine the batch evaluation task according to the detection results of each batch training model.
[0218] Step S30: Start the batch evaluation task.
[0219] The batch evaluation method provided in the embodiment of the present application adopts a multi-process approach to uniformly evaluate the test results of experiments on multiple data sets in the same data set; it can uniformly evaluate the test results of multiple groups of experiments on the same data set, and count the minimum, maximum, mean and standard deviation of the evaluation accuracy of multiple groups of experiments; when evaluating, it does not focus on the evaluation indicators of all categories of defects in the data set, and only needs to count the evaluation indicators of a part of the defect categories. At this time, it is only necessary to specify the target class field (such as the target_classes field) in the batch evaluation Yaml file as the defect category that needs to be evaluated to achieve this requirement. For example, when it is necessary to count the evaluation accuracy of the "missing bolt" defect, the content of the target class field in the batch evaluation task Yaml file can be set to "missing bolt".
[0220] The embodiment of the present application provides a data collection tool to solve the problem that the Benchmark data collection function in the prior art is not complete. The data collection tool can efficiently check the correctness of the data to correct the erroneous data problems existing in the existing data, and can also realize the conversion between Mask and polygons.
[0221] In an embodiment of the present application, the data collection tool includes at least a data correctness checking tool and a format conversion tool, wherein the data correctness checking tool is implemented using multiple processes and can efficiently check the correctness of the data; the format conversion tool implements a tool for mutual conversion between grayscale image mask data format and polygon mask data format.
[0222] In some embodiments, the mask grayscale image is the acquired data format, and during model training, the polygon mask data format is required as input, so the conversion between the grayscale image mask data format and the polygon mask data format needs to be implemented during data collection.
[0223] In some embodiments, the data to be checked is divided into multiple groups, each group of data is checked by a process and erroneous data is corrected, and finally the erroneous data corrected by each process are merged into one group. Figure 8 is a schematic diagram of the data correctness check operation flow provided by the embodiment of the present application, such as Figure 8 As shown, the data correctness check operation process includes dividing the data to be checked 801 into at least one data group, for example, data group 1 to data group n, using the process corresponding to each data group to perform data correctness check on data group 1 to data group n, and obtaining at least one corrected data group, for example, corrected data group 1 to corrected data group n, and merging all corrected data groups to obtain all corrected data 802.
[0224] In some embodiments, data readability refers to whether the data can be read according to the specified address. If it can be read, it means that the data is readable; if it cannot be read, it means that the data is unreadable. Here, the imread function (used to read data in the image file) provided by OpenCV can be used to read the data. If the read result is not None, it means that the data is readable. Resolution correctness means that if the data is readable, it will check whether the resolution of the read image is consistent with the collected resolution. If it is inconsistent, the resolution will be corrected.
[0225] In some embodiments, the grayscale mask data format stores the pixel-level labels of each type of defect, and the storage format is a single-channel image. The polygon mask data format stores the contour information of each type of defect, and the data format is a list. Each element in the list stores the contour information of each defect. For example, the list is [x 0 ,y 0 , x 1 ,y 1 ,...x n-1 ,y n-1 , x n ,y n ], where x and y represent the horizontal and vertical coordinates of the defect contour respectively. A conversion tool between these two data formats is implemented in this patent.
[0226] In some embodiments, the conversion logic between the grayscale mask data format and the polygon mask data format is: extracting the outline of the grayscale mask data format can obtain the polygon mask data, or filling the range enclosed by the polygon mask data to obtain the grayscale mask data format. Here, after the polygon mask data is restored to the grayscale mask data format, there will be some pixel differences with the original grayscale mask data format. The embodiment of the present application will count the number of difference pixels and judge the quality of the conversion between the two masks according to actual needs.
[0227] The embodiment of the present application adopts a submodule module plus a custom expansion module to realize the management of defect detection tools. The submodule module manages different versions of official detection tools, and the custom module can realize customized operations and requirements. In this way, the connection between official tools of different versions and custom modules can be realized. For developers and maintainers, industrial-grade defect detection tools can be efficiently and conveniently managed and used. The embodiment of the present application realizes the batch training function of industrial-grade defect detection tools, and multiple groups of experiments can be started for multiple data sets at the same time, which greatly improves the efficiency of experimental deployment. The embodiment of the present application provides a complete evaluation index calculation method for industrial-grade defect detection data, which can provide calculation method support for different indicators for business-side data sets and research-side data sets at the same time. The embodiment of the present application actually provides a batch evaluation function of industrial-grade defect detection tools, which can simultaneously evaluate the detection results of multiple data set experiments in the same Benchmark, and count the minimum, maximum, mean and standard deviation of multiple groups of experimental results of the same data set. The embodiment of the present application provides an industrial-grade defect detection data collection tool, which can efficiently check the readability and resolution correctness of data, can realize the conversion between grayscale data format mask and polygons data format mask, and check the quality of the conversion results between the two.
[0228] Based on the above embodiments, the present application provides a model determination device. Fig. 9 is a model determination device provided in an embodiment of the present application, such as Fig. 9 As shown, the model determination device 90 includes an acquisition module 901, a model registration module 902, a parallel training module 903, an evaluation module 904 and a determination module 905.
[0229] Among them, the acquisition module 901 is used to obtain a custom detection model; the model registration module 902 is used to register the custom detection model through a pre-built model registrar to obtain at least two detection models to be trained; the parallel training module 903 is used to call at least two training processes to perform parallel training on the at least two detection models to be trained, and correspondingly obtain at least two trained detection models and the detection results of each of the trained detection models; wherein each training process implements the training of a detection model to be trained; the evaluation module 904 is used to evaluate the at least two trained detection models based on the detection results of the at least two trained detection models to obtain model evaluation results; the determination module 905 is used to determine the target detection model from the at least two trained detection models based on the model evaluation results.
[0230] In some embodiments, the model registration module 902 is also used to obtain model information and training information of the custom model; determine a custom configuration file of the custom model based on the model information and the training information; read the custom configuration file through the pre-built model registrar, and register the custom detection model based on the custom configuration file to obtain the at least two detection models to be trained.
[0231] In some embodiments, the model determination device also includes: a first acquisition module, used to obtain a standard configuration file of a standard detection model; the model registration module 902 is also used to connect the custom configuration file and the standard configuration file to obtain a configuration file; based on the custom detection model, the configuration file is model instantiated to obtain at least two detection models to be trained; wherein the model parameters of each detection model to be trained are different from the model parameters of other detection models to be trained.
[0232] In some embodiments, the model determination device also includes: a second acquisition module, used to acquire a grayscale mask sample image, wherein the grayscale mask sample image includes pixel information of the abnormal area; a data conversion module, used to perform data conversion processing on the grayscale mask sample image according to the pixel information to obtain a sample data set; correspondingly, a parallel training module, also used to call at least two training processes to perform parallel training on the at least two detection models to be trained based on the sample data set.
[0233] In some embodiments, the data conversion module is also used to determine the contour information of the abnormal area in the grayscale mask sample image based on the pixel information of the abnormal area in the grayscale mask sample image; determine the coordinate information of each contour point of the abnormal area from the contour information; generate the sample data set based on the coordinate information; the sample data set includes at least polygonal mask data.
[0234] In some embodiments, the model determination device also includes: a third acquisition module, used to acquire at least two initial grayscale mask images; a first data reading module, used to perform data reading processing on the at least two initial grayscale mask images to obtain data reading results; a first correction module, used to perform correction processing on the part of the initial grayscale mask images in response to the data reading results indicating that part of the initial grayscale mask images cannot be read, to obtain at least two readable grayscale mask images; the data of the readable grayscale mask images can be read; a first determination module, used to determine the resolution of each readable grayscale mask image; a first resolution correction module, used to perform resolution correction on the part of the readable grayscale mask images in response to the resolution of the part of the readable grayscale mask images in each readable grayscale mask image being different from the preset resolution, to obtain at least two grayscale mask images; the resolution of the grayscale mask images is the same as the preset resolution; a second determination module, used to determine the at least two grayscale mask images as grayscale mask sample images.
[0235] In some embodiments, the model determination device further includes: a fourth acquisition module, used to acquire at least two initial grayscale mask images; a grouping module, used to group the at least two initial grayscale mask images to obtain at least one initial grayscale mask image group and the number of groups of the initial grayscale mask image group; a second data reading module, used to call a data reading process with the same number as the number of groups, perform data reading processing on the initial grayscale mask images in each initial grayscale mask image group, and obtain a data reading result; a second correction module, used to correct the data of part of the initial grayscale mask images in response to the data reading result indicating that the data of part of the initial grayscale mask images cannot be read, and obtain a readable grayscale mask image with the same number as the number of groups. image group; the data of the readable grayscale mask images in the readable grayscale mask image group can be read; a third determination module, used to determine the resolution of each readable grayscale mask image in the readable grayscale mask image group for each readable grayscale mask image group; a second resolution correction module, used to call a resolution correction process in response to the resolution of some readable grayscale mask images in the readable grayscale mask image group being different from a preset resolution, and perform resolution correction on the some readable grayscale mask images to obtain grayscale mask image groups with the same number as the number of groups, and the resolution of the grayscale mask images in the grayscale mask image group is the same as the preset resolution; a fourth determination module, used to determine the grayscale mask images in the grayscale mask image group as grayscale mask sample images.
[0236] In some embodiments, the detection model to be trained includes at least one model parameter; the parallel training module is also used to set the model parameters to obtain at least two experimental models corresponding to each of the detection models to be trained and the number of experiments of the experimental models; wherein the model parameters corresponding to each experimental model are different from the model parameters corresponding to other experimental models; calling a training process with the same number as the number of experiments, and performing parallel training on the at least two experimental models based on the sample data set to obtain at least two trained experimental models and experimental results corresponding to each experimental model; the experimental results corresponding to the at least two trained experimental models and each of the experimental models are correspondingly determined as the detection results of the at least two trained detection models and each of the trained detection models.
[0237] In some embodiments, the evaluation module is also used to determine the number of abnormal images of grayscale mask images with abnormal areas, and the number of non-abnormal images of grayscale mask images without abnormal areas in the grayscale mask sample images; determine the number of abnormal data samples with abnormal areas, and the number of non-abnormal data samples without abnormal areas output by each trained detection model according to the detection results of the at least two trained detection models; determine the recall rate of each trained detection model according to the number of abnormal data samples and the number of abnormal images; determine the false alarm rate of each trained detection model according to the number of non-abnormal data samples and the number of non-abnormal images; and evaluate the at least two trained detection models according to the recall rate and the false alarm rate to obtain model evaluation results.
[0238] In some embodiments, the grayscale mask sample image includes at least one grayscale mask image; the model determination device also includes: a fifth determination module, which is used to determine the number of abnormal images of grayscale mask images with abnormal areas and the number of non-abnormal images of grayscale mask images without abnormal areas in the grayscale mask sample images based on pixel information of abnormal areas in the at least one grayscale mask image.
[0239] In some embodiments, each of the trained detection models corresponds to at least two groups of experimental model evaluation modules, and is also used to evaluate the experimental models according to the recall rate and false alarm rate of each experimental model, and obtain the corresponding experimental evaluation results of each experimental model; the standard deviation of the experimental evaluation results of at least two groups of experimental models corresponding to each of the trained detection models is calculated to obtain the standard deviation evaluation results of each of the trained detection models; the standard deviation evaluation results of each of the trained detection models are determined as the model evaluation results of the trained detection model.
[0240] In some embodiments, the determination module is further used to determine, among the at least two trained detection models, a trained detection model whose standard deviation evaluation result satisfies a preset standard deviation condition as the target detection model.
[0241] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.
[0242] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.
[0243] It should be noted that in the embodiment of the present application, if the above-mentioned model determination method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.
[0244] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the above-mentioned model determination method when executing the computer program.
[0245] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned model determination method is implemented. The computer-readable storage medium can be transient or non-transient.
[0246] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.
[0247] It should be noted that Fig.10 is a hardware entity diagram of a model determination device provided in an embodiment of the present application, such as Fig.10 As shown, the hardware entities of the model determination device 10 include: a processor 11, a communication interface 12 and a memory 13, wherein:
[0248] The processor 11 generally controls the overall operation of the model determination device 10 .
[0249] The communication interface 12 may enable the model determination device 10 to communicate with other terminals or servers through a network.
[0250] The memory 13 is configured to store instructions and applications executable by the processor 11, and can also cache data to be processed or processed by the processor 11 and each module in the model determination device 10 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM). Data can be transmitted between the processor 11, the communication interface 12 and the memory 13 through the bus 14.
[0251] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.
[0252] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.
[0253] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0254] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0255] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0256] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately configured as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0257] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.
[0258] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a model determination device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.
[0259] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.
Claims
1. A model determination method, characterized in that: The method comprises: Get a custom detection model; Registering the custom detection model through a pre-built model registrar to obtain at least two detection models to be trained; Calling at least two training processes to perform parallel training on the at least two detection models to be trained, and correspondingly obtaining at least two trained detection models and detection results of each of the trained detection models; wherein each training process implements training of one detection model to be trained; Based on the detection results of the at least two trained detection models, performing evaluation processing on the at least two trained detection models to obtain model evaluation results; Based on the model evaluation results, a target detection model is determined from the at least two trained detection models.
2. The method according to claim 1, characterized in that The custom detection model is registered by a pre-built model registrar to obtain at least two detection models to be trained, including: Obtaining model information and training information of the custom model; Determining a custom configuration file of the custom model according to the model information and the training information; The custom configuration file is read by the pre-built model registrar, and the custom detection model is registered based on the custom configuration file to obtain the at least two detection models to be trained.
3. The method according to claim 1, characterized in that The method further comprises: Get the standard configuration file of the standard detection model; The step of registering the custom detection model based on the custom configuration file to obtain the at least two detection models to be trained includes: Connecting the custom configuration file and the standard configuration file to obtain a configuration file; Based on the custom detection model, the configuration file is instantiated to obtain at least two detection models to be trained; wherein the model parameters of each detection model to be trained are different from the model parameters of other detection models to be trained.
4. The method according to claim 1, characterized in that: The method further comprises: Acquire a grayscale mask sample image, wherein the grayscale mask sample image includes pixel information of an abnormal area; Performing data conversion processing on the grayscale mask sample image according to the pixel information to obtain a sample data set; Correspondingly, the calling of at least two training processes to perform parallel training on the at least two detection models to be trained includes: At least two training processes are called to perform parallel training on the at least two detection models to be trained based on the sample data set.
5. The method according to claim 4, characterized in that The step of performing data conversion processing on the grayscale mask image according to the pixel information to obtain a sample data set includes: Determining contour information of the abnormal area in the grayscale mask sample image according to pixel information of the abnormal area in the grayscale mask sample image; Determine coordinate information of each contour point of the abnormal area from the contour information; The sample data set is generated according to the coordinate information; the sample data set at least includes polygon mask data.
6. The method according to claim 4, characterized in that The method further comprises: Obtain at least two initial grayscale mask images; Performing data reading processing on the at least two initial grayscale mask images to obtain a data reading result; In response to the data reading result indicating that part of the initial grayscale mask image cannot be read, performing correction processing on the part of the initial grayscale mask image to obtain at least two readable grayscale mask images; data of the readable grayscale mask images can be read; determining a resolution of each readable grayscale mask image; In response to the resolution of a partially readable grayscale mask image in each of the readable grayscale mask images being different from a preset resolution, performing resolution correction on the partially readable grayscale mask image to obtain at least two grayscale mask images; the resolution of the grayscale mask image being the same as the preset resolution; The at least two grayscale mask images are determined as grayscale mask sample images.
7. The method according to claim 4, characterized in that The method further comprises: Obtain at least two initial grayscale mask images; performing grouping processing on the at least two initial grayscale mask images to obtain at least one initial grayscale mask image group and the number of groups of the initial grayscale mask image group; Calling data reading processes having the same number as the number of the groups, performing data reading processing on the initial grayscale mask image in each initial grayscale mask image group, and obtaining data reading results; In response to the data reading result indicating that data of a part of the initial grayscale mask images cannot be read, performing correction processing on the part of the initial grayscale mask images to obtain a number of readable grayscale mask image groups having the same number as the number of the groupings; data of the readable grayscale mask images in the readable grayscale mask image groups can be read; For each readable grayscale mask image group, determining a resolution of each readable grayscale mask image in the readable grayscale mask image group; In response to the resolution of some of the readable grayscale mask images in the readable grayscale mask image group being different from the preset resolution, calling a resolution correction process to perform resolution correction on the some of the readable grayscale mask images, so as to obtain grayscale mask image groups having the same number as the number of the groups, and the resolution of the grayscale mask images in the grayscale mask image groups being the same as the preset resolution; The grayscale mask images in the grayscale mask image group are determined as grayscale mask sample images.
8. The method according to any one of claims 4 to 7, characterized in that: The detection model to be trained includes at least one model parameter; The calling of at least two training processes to perform parallel training on the at least two detection models to be trained, and correspondingly obtaining at least two trained detection models and a detection result of each of the trained detection models, includes: The model parameters are set to obtain at least two experimental models corresponding to each of the detection models to be trained and the number of experiments of the experimental models; wherein the model parameters corresponding to each experimental model are different from the model parameters corresponding to other experimental models; Calling training processes having the same number as the number of experiments, and training the at least two experimental models in parallel based on the sample data set to obtain at least two trained experimental models and experimental results corresponding to each experimental model; The at least two trained experimental models and the experimental results corresponding to each of the experimental models are correspondingly determined as the detection results of the at least two trained detection models and each of the trained detection models.
9. The method according to any one of claims 1 to 8, characterized in that: The step of performing evaluation processing on the at least two trained detection models based on the detection results of the at least two trained detection models to obtain model evaluation results includes: Determine the number of abnormal images of grayscale mask images having abnormal regions and the number of non-abnormal images of grayscale mask images not having abnormal regions in the grayscale mask sample images; Determine, according to the detection results of the at least two trained detection models, the number of abnormal data samples with abnormal areas and the number of non-abnormal data samples without abnormal areas output by each trained detection model; Determining the recall rate of each trained detection model according to the number of abnormal data samples and the number of abnormal images; Determine the false alarm rate of each trained detection model according to the number of non-abnormal data samples and the number of non-abnormal images; According to the recall rate and the false alarm rate, the at least two trained detection models are evaluated to obtain a model evaluation result.
10. The method according to claim 9, characterized in that The grayscale mask sample image includes at least one grayscale mask image; the method further includes: According to the pixel information of the abnormal area in the at least one grayscale mask image, the number of abnormal images of the grayscale mask images with abnormal areas and the number of non-abnormal images of the grayscale mask images without abnormal areas in the grayscale mask sample images are determined.
11. The method according to claim 9, characterized in that Each of the trained detection models corresponds to at least two groups of experimental models; The step of evaluating the at least two trained detection models according to the recall rate and the false alarm rate to obtain a model evaluation result includes: According to the recall rate and false alarm rate of each experimental model, the experimental model is evaluated and processed, and the experimental evaluation results of each experimental model are obtained accordingly; Calculating the standard deviation of the experimental evaluation results of at least two groups of experimental models corresponding to each of the trained detection models to obtain the standard deviation evaluation results of each of the trained detection models; The standard deviation evaluation result of each of the trained detection models is determined as the model evaluation result of the trained detection model.
12. The method according to any one of claims 1 to 11, characterized in that: The step of determining a target detection model from the at least two trained detection models based on the model evaluation results includes: Among the at least two trained detection models, the trained detection model whose standard deviation evaluation result satisfies a preset standard deviation condition is determined as the target detection model.
13. A model determination device, characterized in that: The device comprises: Acquisition module, used to obtain custom detection models; A model registration module, used to register the custom detection model through a pre-built model registrar to obtain at least two detection models to be trained; A parallel training module, used to call at least two training processes to perform parallel training on the at least two detection models to be trained, and correspondingly obtain at least two trained detection models and the detection results of each of the trained detection models; wherein each training process implements the training of one detection model to be trained; An evaluation module, configured to perform evaluation processing on the at least two trained detection models based on the detection results of the at least two trained detection models to obtain model evaluation results; A determination module is used to determine a target detection model from the at least two trained detection models based on the model evaluation results.
14. A model determination device, comprising a processor and a memory, wherein the memory stores a computer program executable on the processor, characterized in that: When the processor executes the computer program, the model determination method according to any one of claims 1 to 12 is implemented.
15. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the model determination method according to any one of claims 1 to 12 is implemented.
