Task processing method, device, equipment and storage medium based on defect detection

Abstract

A method and apparatus for task processing based on defect detection, a computer-readable storage medium and a task processing device based on defect detection, the method comprising: receiving a detection task and determining the task type of the detection task (S110); if the task type is a target task type, storing the detection task in a task queue (S120); when the processor is idle, executing the detection task in a preset order and generating a feedback signal (S130); wherein the detection task of the target task type includes an inference task and a training task; executing the training task comprises: modifying configuration information according to preset rules based on product information in the detection task; obtaining training data and an initial model based on the product information; and using the training data to train the initial model according to the configuration information to obtain a target model and store it.

Description

Background Art

[0002] In the prior art, the detection of display screen defects is performed manually, which is slow and prone to cause unnecessary losses due to human errors.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] According to a first aspect of the present disclosure, a task processing method based on defect detection is provided, comprising:

[0005] Receiving a detection task and determining a task type of the detection task;

[0006] If the task type is a target task type, storing the detection task in a task queue;

[0007] When the processor is idle, the detection tasks are executed in a preset order and a feedback signal is generated;

[0008] The detection task of the target task type includes an inference task and a training task; and executing the training task includes:

[0009] Modify configuration information according to preset rules based on product information in the inspection task;

[0010] Acquire training data and an initial model according to the product information;

[0011] The initial model is trained using the training data according to the configuration information to obtain a target model and store it.

[0012] In an exemplary embodiment of the present disclosure, the task type includes the target task type and a reference task type;

[0013] If the task type is a reference task type, the processor directly executes the detection task.

[0014] In an exemplary embodiment of the present disclosure, executing the detection tasks in a preset order includes:

[0015] The detection tasks are executed in the order in which they are stored in the task queue.

[0016] In an exemplary embodiment of the present disclosure, the step of modifying the configuration information according to the product information in the detection task according to the preset rules includes:

[0017] Extracting the product information field in the detection task to obtain product information;

[0018] The training image size, training times, test times, defect threshold and learning rate reduction strategy are set according to the product information and preset rules.

[0019] In an exemplary embodiment of the present disclosure, the training task includes a training data acquisition address and model data information; the step of acquiring training data and an initial model according to the product information includes:

[0020] Download the training data according to the training data acquisition address;

[0021] Obtain model data information according to the model data information field;

[0022] The initial model is acquired according to the model data information.

[0023] In an exemplary embodiment of the present disclosure, the training data includes a label file and an image file. Before using the training data to train the initial model according to the configuration information to obtain the target model and storing it, the method further includes:

[0024] Preprocessing the training data;

[0025] The preprocessing of the training data includes:

[0026] Delete abnormal images in image files;

[0027] Delete image files that do not need to match label files;

[0028] Match image files with label files;

[0029] The training data is divided into a training set, a validation set and a test set according to a preset ratio.

[0030] In an exemplary embodiment of the present disclosure, the using the training data to train the initial model according to the configuration information to obtain the target model and storing the target model includes:

[0031] Training the initial model according to the training set, the validation set, and the number of training times;

[0032] The test set and the test times are used to complete the test of the trained initial model to obtain the target model and store it.

[0033] In an exemplary embodiment of the present disclosure, the detection task includes a task type tag, and determining the task type of the detection task includes:

[0034] A task type tag in the detection task is identified to determine the task type of the detection task.

[0035] In an exemplary embodiment of the present disclosure, executing the training task further includes:

[0036] Upload the target model and training results to a preset address;

[0037] The training results include the accuracy, recall and F1 score of the target model.

[0038] In an exemplary embodiment of the present disclosure, executing the reasoning task includes:

[0039] Acquire the target model and image information of the product according to the product information;

[0040] The image information is input into the target model to obtain the defect location and defect type of the product.

[0041] In an exemplary embodiment of the present disclosure, the reasoning task includes an image information acquisition address, and acquiring the image information of the product according to the product information includes:

[0042] The image information is downloaded according to the image information acquisition address.

[0043] In an exemplary embodiment of the present disclosure, acquiring the target model according to the product information includes:

[0044] Detecting whether there is a target model corresponding to the product information in the local repository;

[0045] If the target model corresponding to the product information exists in the local repository, then the target model is loaded into the local repository;

[0046] If the target model corresponding to the product information does not exist in the local repository, the local repository downloads the target model corresponding to the product information according to the product information.

[0047] In an exemplary embodiment of the present disclosure, the local repository includes a preset number of target models. When the number of target models in the local repository is greater than the preset number, the local repository is updated according to the chronological order in which the target models are used so that the number of target models in the local repository remains at the preset number.

[0048] In an exemplary embodiment of the present disclosure, the performing of the reasoning task further includes:

[0049] The feedback signal is generated according to the defect location and the defect type of the product.

[0050] According to one aspect of the present disclosure, there is provided a task processing device based on defect detection, comprising:

[0051] A communication node, used for receiving a detection task and identifying a task type of the detection task;

[0052] The task management node is used to store the detection task in a task queue when the task type is a target task type; and to execute the detection task in a preset order and generate a feedback signal when the processor is idle.

[0053] In an exemplary embodiment of the present disclosure, the device further includes:

[0054] The task agent node is used to receive the feedback signal and send the feedback signal through the communication module.

[0055] According to one aspect of the present disclosure, there is provided a task processing device based on defect detection, comprising:

[0056] Processor; and

[0057] A memory is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned task processing methods based on defect detection.

[0058] In an exemplary embodiment of the present disclosure, the task type includes the target task type and the reference task type; the processor includes:

[0059] A first processor, the first processor comprising at least one graphics processor, configured to execute the reasoning task and the training task;

[0060] A second processor, the second processor comprising at least one graphics processor, is configured to execute a detection task of a reference task type.

[0061] According to one aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the task processing method based on defect detection as described in any one of the above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0063] Figure 1 A flowchart schematically illustrates a task processing method based on defect detection in an exemplary embodiment of the present disclosure;

[0064] Figure 2 A schematic diagram schematically illustrates a server in an exemplary embodiment of the present disclosure;

[0065] Figure 3 A flowchart schematically illustrates a training task performed in an exemplary embodiment of the present disclosure;

[0066] Figure 4 A schematic diagram schematically illustrates the interaction of multiple segments when performing a training task in an exemplary embodiment of the present disclosure;

[0067] Figure 5 A flowchart schematically illustrates an exemplary embodiment of the present disclosure in which a reasoning task is performed;

[0068] Figure 6 A schematic diagram schematically illustrates the interaction of multiple segments when performing a reasoning task in an exemplary embodiment of the present disclosure;

[0069] Figure 7 A schematic diagram of the architecture deployment of the server in an exemplary embodiment of the present disclosure is schematically shown;

[0070] Figure 8 A schematic diagram schematically illustrates task scheduling on a server in an exemplary embodiment of the present disclosure;

[0071] Fig. 9 A schematic diagram schematically shows a system architecture diagram of the interaction between the server and the backend in an exemplary embodiment of the present disclosure;

[0072] Fig.10 A schematic diagram schematically shows the composition of a task processing device based on defect detection in an exemplary embodiment of the present disclosure;

[0073] Fig.11 A schematic diagram schematically shows the composition of another task processing device based on defect detection in an exemplary embodiment of the present disclosure;

[0074] Fig.12 A schematic diagram of the structure of a computer system suitable for implementing a task processing device based on defect detection according to an exemplary embodiment of the present disclosure is shown;

[0075] Fig.13 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0076] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0077] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0078] In this exemplary embodiment, a task processing method based on defect detection is first provided, which can be applied to defect detection of a display screen. Figure 1 As shown in , the above-mentioned task processing method based on defect detection may include the following steps:

[0079] S110, receiving a detection task and determining a task type of the detection task;

[0080] S120, if the task type is a target task type, storing the detection task in a task queue;

[0081] S130, when the processor is idle, executing the detection tasks in a preset order and generating a feedback signal;

[0082] The detection task of the target task type includes an inference task and a training task; and executing the training task includes:

[0083] Modify configuration information according to preset rules based on product information in the inspection task;

[0084] Acquire training data and an initial model according to the product information;

[0085] The initial model is trained using the training data according to the configuration information to obtain a target model and store it.

[0086] According to the task processing method based on defect detection provided in this exemplary embodiment, compared with the existing technology, the detection task is accepted and the detection task of the target task type is stored in the task queue, the detection task is processed when the processor is idle, and the training task and the reasoning task are completed to realize the detection of the product. A computer training model is used and the model is used for reasoning. There is no need to complete the detection of the product manually, which speeds up the detection efficiency, prevents losses caused by human errors, and improves the detection accuracy.

[0087] In the following, each step of the task processing method based on defect detection in this exemplary embodiment will be described in more detail with reference to the accompanying drawings and embodiments.

[0088] Step S110, receiving a detection task and determining the task type of the detection task.

[0089] In an exemplary embodiment of the present disclosure, referring to Figure 2 As shown, the server receives the detection task, which can be issued by the background 210. The task type of the detection task can include a target task type and a reference task type, wherein the detection task of the target task type can include a training task, a reasoning task, etc., and the detection task of the reference task type can include a training status query task, an early termination training task, a node information query task (querying the unique identification number and type of the current GPU node), a task queue 202 clearing task, and a task status query task. In this example implementation, the server can be a gpu server.

[0090] In this example implementation, the above detection task includes a task type tag, and the server can determine the task type of the above detection task based on the above task type tag. The task format of the detection task consists of a task header and a task body, wherein the task header can be used to distinguish the type of task. In this example, real-time party verification can include three types of task headers, namely transact (transaction request task), reply (transaction return task) and ack (acknowledgement task). The "type" key can be included in the task body of the Transact type to indicate the type of the detection task.

[0091] In this example implementation, the server can establish a socket network connection with the background 210 through the communication node 201, and the background 210 can send the detection task to the communication node 201 of the server through the TCP protocol. The server receives the above-mentioned detection task through the above-mentioned communication node 201, and determines the task type of the detection task through the above-mentioned task type label.

[0092] In step S120, if the task type is a target task type, the detection task is stored in a task queue;

[0093] In one embodiment of the present disclosure, referring to Figure 2 As shown, the number of various detection tasks received by the above-mentioned communication node 201 may include multiple ones. When the task type of the above-mentioned detection task is a target task type, the detection tasks in the target task type, such as training tasks and reasoning tasks, all require a longer time to execute, so the above-mentioned detection tasks can be stored in the task queue 202.

[0094] In this example implementation, when the detection task is of a reference task type, the detection task of the reference task type may be directly executed by the processor because the execution time of the detection task is relatively short.

[0095] In step S130, when the processor is idle, the detection tasks are executed in a preset order and a feedback signal is generated;

[0096] In this example implementation, the server can try to detect the status of the processor, and when the processor completes all tasks before the detection task, it executes the above detection task in a preset order. The above preset order can be the order of storage in the task queue, or it can be based on the execution difficulty of the task, which is not specifically limited in this example implementation.

[0097] Specifically, refer to Figure 3 As shown, when executing the training task, the following steps are included:

[0098] Step S310, modifying the configuration information according to the product information in the detection task according to the preset rules;

[0099] Step S320, obtaining training data and an initial model according to the product information;

[0100] Step S330: Use the training data to train the initial model according to the configuration information to obtain a target model and store it.

[0101] In this example implementation, refer to Figure 4 As shown, the above-mentioned training task includes basic information required for training, such as the training data acquisition address, model data information, training model upload address, training result upload address and relevant configuration of the FTP server. Step S401 can be executed first, and the communication node 201 receives the training task from the background 210. After receiving the training task, the communication node executes step S402 to put the training task into the task queue 202; the task management node 203 monitors the status of the processor, and when the processor is idle, it extracts the training task from the queue, and obtains the training data and the initial model in turn, and starts to execute step S403 to execute the training task.

[0102] In this example implementation, when acquiring training data, first, steps S404 and S405 can be executed, and a large amount of training data can be downloaded from a remote FTP server (FTP SERVER) 206 for training the initial model, and the training data must be stored in the format set by the algorithm. Since the amount of training data is large, which may include hundreds of thousands of images, the (FTP client) FTP client can be used to load the training data using the multi-threaded FTP protocol 205. After the acquisition of the training data is completed, steps S406 and S407 are executed to start the training algorithm, and the training algorithm can be called from the training algorithm repository 208 to train the above initial model using the training data. When the preset number of training times or the training accuracy, recall rate and other indicators are reached, the training algorithm can be exited, and step S412 is executed to complete the above training task. Then execute steps S408, S409, S410 and S411 to upload the trained model and training result details (such as training accuracy, recall rate, loss value curve graph, defect confusion matrix and other charts) to the specified directory of the FTP server, where the above directory can be extracted from the above training task.

[0103] Then, by executing step S413 and step S414, the final training results (accuracy, recall, F1 score and MD5 value of the model) can be packaged into a TCP message in a specified format, and the training task is completed. Then, the processor sends the training results to the task agent node 204, which is then fed back to the communication node 201, and the communication node 201 finally sends it to the background 210, and the entire training process is completed.

[0104] The above steps are described in detail below.

[0105] In step S310, the configuration information is modified according to the product information in the detection task according to the preset rules;

[0106] In this example implementation, according to actual business needs, three types of images may be involved, namely, the final site image (SD_final site), the intermediate site image (mask site) and the TDI grayscale image; among which the models that can be applied to the SD_final final site image are the subcode model (subcode model) and the maincode model (main code model)), the image of the intermediate site can apply the subcode model (subcode model); and the TDI grayscale image can apply the subcode model (subcode model).

[0107] In this example implementation, the final site image divides the models by product type. Therefore, different products can be distinguished by different model names. The naming format of the model used in the final site image can be product_product name_date, and the naming format of the main code model (maincode model) is maincode_product name_date; the naming of the model name can also be customized according to needs, and is not specifically limited in this example implementation.

[0108] In this example implementation, the intermediate site image divides the model by site, and the naming rule of the model name of the sub-code model used by the intermediate site image can be mask_site name_date; the naming of the model name can also be customized according to needs and is not specifically limited in this example implementation.

[0109] In this example implementation, the tdi grayscale image divides the model into conventional and vehicle-mounted models, and the naming rule of the model used in the tdi grayscale image can be tdi_1 / 2_date, where the model of conventional products uses 1 and the model of vehicle-mounted products uses 2. Since different model types correspond to a set of different data processing (image size, image and label matching method, etc.), and different products or different products at the same site correspond to different training parameters, the server needs to clearly define the training model and product or site information before calling the training algorithm.

[0110] In this example implementation, due to the large differences in the occurrence rate and background morphology of the same defect code for different products or different products at the same site, the amount of training data and the difficulty of recognition for the same defect code for different products or the same products at different sites are different. Therefore, it is necessary to set different training code lists, special defect code thresholds, image sizes, learning rate reduction strategies, training times, test times and other configuration information for individual products or the same products at different sites.

[0111] In this example implementation, before training is started, the user can set the above parameters on the interface according to the training task to be submitted, and then the training system will write the interface parameters to the configuration file of the server. After the training algorithm is started, the server will read the corresponding configuration information according to the model type and product information of the training task.

[0112] In step S320, training data and an initial model are acquired according to the product information;

[0113] Step S330: Use the training data to train the initial model according to the configuration information to obtain a target model and store it.

[0114] In an example implementation of the present disclosure, the server receives a training task sent by the training system and parses the task. In order to determine the model type and product name / site name through the training task, the training task contains the "modelPath" (model field) and "trainType" fields (product information field). By parsing these two fields, the server can clearly understand the model type, site name, and product name to be trained this time. For example, the task contains the "modelPath": / XXX / XXX / XXX / product_ak_20200422084814916" and "trainType":"subCodeTraining" fields. By parsing these two fields, the server processor will clarify the sub-code model of the ak product, the final site image, to be trained this time.

[0115] In this example implementation, the training data is the core of the training model. The training data can be manually uploaded to a specified address. Before executing the training algorithm, the server will download the data from the specified address to the local file of the server via FTP for subsequent data processing.

[0116] In this example implementation, before training begins, the server can parse the model data information in this training task, where the model data information may include model type, product name, site name, and corresponding configuration information. The initial model is obtained based on the above model data information. After obtaining the training data and the initial model, the server calls and starts the training algorithm in the training algorithm repository, where the training algorithm may include preprocessing of the training data, a training module, and a testing module, where the training module is used to train the initial model, and the testing module is used to test the trained initial model.

[0117] In this example implementation, when preprocessing the training data, specifically, first, you can check the two folders image and xml respectively, delete abnormal code folders, such as other, unknown folders, etc., as well as folders that do not match image and xml; then, delete the xml folder corresponding to the code that does not need a label, and match the image file with the label file, and delete the code folder that does not need xml: according to the specific training data type, delete the corresponding code folder in the xml folder. Finally, match the image and xml files: match all images and label files in the image image and label xml folders one by one, delete the pictures that do not correspond to the label files, or delete the label files that do not correspond to the pictures. The image file is an image file, and the xml file is a label file.

[0118] In this example implementation, the training data can be allocated according to a preset ratio to obtain a training set, a validation set, and a test set, wherein the preset ratio can be 8:1:1. It can also be 6:2:2, which is not specifically limited in this example implementation.

[0119] Specifically, a txt file can be generated, and the above data set can be divided into a training set, a validation set, and a test set according to 8:1:1, and written into three files, train.txt, val.txt, and test.txt, respectively. Among them, train.txt, val.txt, and test.txt correspond to the training set, the validation set, and the test set, respectively. At the same time, in the process of generating the txt file, the file can be checked, and the problematic images can be deleted and not written into the txt file.

[0120] In this example implementation, after the training data is preprocessed, the initial model is trained using the training data to obtain the target model. According to the "image_scales" in the configuration information, the images in the training set will be modified to the corresponding size and then sent to the constructed network to participate in the learning of the initial model network parameters. The "learningRate_stepSize" in the configuration information sets the learning rate reduction strategy, that is, after the initial model is trained to a specified number of rounds, the learning rate will be reduced to about 1 / 9 to 1 / 10 of the original, so that the initial model can converge as soon as possible. "max_step" sets the number of training times, that is, when the model is trained to a specified number of times, the training task ends.

[0121] In this example implementation, after the initial model training is completed, the server can use the test data set (test.txt file) to perform a preset number of model tests on the trained initial model according to the "test_model_iter" in the configuration file, and obtain the model effect (accuracy, recall rate and F1 score) of each test number. The user can select the best model as the target model for the reasoning task based on the accuracy and recall rate of the test model.

[0122] In this example implementation, after completing the training of the initial model to obtain the target model, the server may store the target model to a specified address, that is, upload it to the above-mentioned FTP server (FTP SERVER) 206 through the above-mentioned FTP client (FTP client) 205.

[0123] In this example implementation, when the model is tested, a confusion matrix of the model is generated. The server uploads the confusion matrix file corresponding to the target model to the path specified in the message, that is, uploads it to the FTP server 206 through the FTP client 205. Then the server sends feedback information to the background 210 through the task proxy node 204 and the communication node to complete the training task.

[0124] In an exemplary embodiment of the present disclosure, referring to Figure 5 As shown, executing the reasoning task may include the following steps:

[0125] Step S510, acquiring the target model and image information of the product according to the product information;

[0126] Step S520: input the image information into the target model to obtain the defect location and defect type of the product.

[0127] In this example implementation, refer to Figure 6 As shown, the above target model can be used to perform real-time reasoning on the product to obtain the defect type of the product. The image information of the product can be obtained first, and the image information can be stored in an XML file, where the image information can include site information, product model, image storage address and other information.

[0128] In this example implementation, the inference system will send the XML file address of the production line data set to the backend 210 in the form of a TCP message. The backend 210 can obtain the XML file according to the file address, parse the information of the data set in the XML file, and organize it into an inference task, where the inference task includes the inference data set loading address, the inference model loading address, the inference model md5 value, and the relevant configuration of the FTP server, and send it to the communication node; then steps S501 and S502 can be executed, and the communication node 201 receives the inference task and puts the inference task into the task queue. When the task management node 203 is idle, it will obtain the inference task object from the queue, and execute step S503 to start executing the inference task.

[0129] In this example implementation, for the reasoning task, step S504, step S505, step S506, step S507 and step S508 may be first executed. First, the image information required for the reasoning task may be loaded from the remote FTP server (FTP SERVER) 206 according to the product information, and the target model matching the data set may be loaded through the model loader 207. The model loading may first be loaded from the locally cached model. If the local loading fails, the target model may be downloaded from the remote FTP server through the FTP protocol (FTP clint) 205. After the image information and the target model are downloaded, step S509 may be executed, and the processor may call the reasoning algorithm from the reasoning algorithm repository 209 for reasoning. After all the image reasoning in the image information is completed, step S510 may be executed to exit the reasoning algorithm and end the reasoning task. Finally, step S511, step S512, and step S513 can be executed to package the reasoning result message (the maincode class of each reasoning image and the confidence of the maincode class, the subcode class and the confidence of the subcode class and the position of the defect box, etc.) into a TCP message according to the specified format, and the reasoning task is completed; the task management node 203 will send the packaged message to the task agent node 204, which will then feed back to the communication node 201, and the communication node 201 will send it to the background 210.

[0130] The above steps are described in detail below.

[0131] In this example implementation, different reasoning tasks may correspond to different product information, so each reasoning task may call a different target model. If the target model of the corresponding task is loaded after each task is received, it will be very time-consuming and may cause the task processing to time out. Therefore, in order to save reasoning time, after the server is started, it will first complete the loading of the target model in advance, and directly call the loaded target model for each received reasoning task.

[0132] In this example implementation, since there are too many types of products and sites involved in the business, and each product can correspond to a different target model, if all models are loaded into the video memory, it may cause the video memory to overflow and fail to complete the inference task. Therefore, when the server is started for the first time, only the target models corresponding to the more commonly used products can be loaded into the video memory. When the target models corresponding to the products need to be loaded later, they will be loaded accordingly.

[0133] In this example implementation, each time a target model is loaded, the loading time is bound to each target model as a timestamp. And each time the target model is called, the timestamp is changed to the calling time. In addition, the corresponding code list, general threshold, and special threshold in the configuration information set before training the initial model are bound to the target model. Subsequent configuration information modification will not affect the inference task.

[0134] In step S510, the target model and image information of the product are acquired according to the product information;

[0135] In this example implementation, after receiving the inference task message, the server will first parse the inference task to obtain the image list of this inference task (the absolute path of each image), the called subcode model, maincode model and tdi model and other information.

[0136] In this example implementation, the image information of this reasoning task is downloaded to the local server according to the address obtained by the image information in the reasoning task message, and then the target model can be loaded. When loading the target model, it can be first determined whether the target model specified in the reasoning task has been loaded. If there is a target model corresponding to the image information in the list of loaded target models, the loaded target model can be directly called to complete the reasoning; if there is no target model corresponding to the image information in the list of loaded target models, it is determined whether the product name and site name in the specified model exist in the loaded product information list. If it exists, the model is a target model to be updated, so the new target model needs to be reloaded and the old model needs to be released from the video memory; if it does not exist, the model is a model to be newly launched, and the new target model needs to be loaded first. The newly loaded target model can be completed online and will be added to the list of loaded target models.

[0137] In this example implementation, in order to prevent video memory overflow caused by constantly loading new models, when the models loaded into the video memory reach a preset number, the target model that has not been called for a long time will be deleted according to the above timestamp, and the video memory will be released. The preset number can be 30, 40, etc., and can also be set according to the server configuration used by the user, which is not specifically limited in this example implementation.

[0138] Step S520: input the image information into the target model to obtain the defect location and defect type of the product.

[0139] In this example implementation, after the target model is loaded, the image information in the reasoning task will be read into the memory in sequence and sent to the prepared target model for reasoning. Based on the set threshold, each image will get the defect category, defect location and its confidence after passing through the model. After the image information in this task has obtained the defect category, defect location and its confidence, the reasoning algorithm is exited to complete the reasoning task.

[0140] In this example implementation, after obtaining the defect categories, defect locations and confidence levels of all image information, the reasoning results of all images in the entire task are integrated and then returned to the reasoning system through the task proxy node and the communication node.

[0141] In this example embodiment, the server may be one or more devices capable of performing reasoning tasks and training tasks using GPU computing resources. The reasoning task based on artificial intelligence may be a defect identification task based on a target model, which is mainly a neural network model based on deep learning. For example, the target model may be based on a feedforward neural network. The feedforward network may be implemented as an acyclic graph in which nodes are arranged in layers. Typically, the feedforward network topology includes an input layer and an output layer, which are separated by at least one hidden layer. The hidden layer transforms the input received by the input layer into a representation useful for generating outputs in the output layer. The network nodes are fully connected to the nodes in the adjacent layers via edges, but there are no edges between the nodes in each layer. The data received at the nodes of the input layer of the feedforward network is propagated (i.e., "feedforward") to the nodes of the output layer via an activation function, which calculates the state of the nodes of each continuous layer in the network based on coefficients ("weights"), and the coefficients are associated with each of the edges connecting these layers. The output of the target model may take various forms, and the present disclosure is not limited thereto. The target model may also include other neural network models, such as a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, and a generative adversarial network (GAN) model, but is not limited thereto. Other neural network models known to those skilled in the art may also be used.

[0142] The target model model usually needs to be obtained through training. The above-mentioned initial training using a training algorithm may include the following steps: selecting a network topology; using a set of training data representing the problem modeled by the network; and adjusting the weights until the network model exhibits the minimum error for all instances of the training data set. For example, during the supervised learning training process for a neural network, the output generated by the network in response to the input representing an instance in the training data set is compared with the "correct" labeled output of the instance; an error signal representing the difference between the output and the labeled output is calculated; and when the error signal is propagated backward through the layers of the network, the weights associated with the connection are adjusted to minimize the error. When the error of each output generated from the instance of the training data set is minimized, the initial model is considered to be "trained" and defined as the target model, and can be used for artificial intelligence reasoning tasks.

[0143] The reasoning and training process of neural network models involves a large number of parameters, activation values, and gradient value buffers, each of which must be completely updated in each training iteration, which places very high demands on computing and throughput capabilities. GPUs have strong parallel capabilities and high video memory bandwidth, and are extremely efficient in processing calculations associated with training deep neural networks. Therefore, using a GPU cluster that integrates multiple GPUs can effectively improve the training and reasoning speed of neural network models.

[0144] In the embodiment of the present disclosure, based on the reasoning task, the implementation method of identifying the product defect content in the product image through the target model is as follows: first, the product image is scaled to a fixed pixel size MxN (or not scaled), and then the MxN image is sent to the target model (VGG / Resnet / MobileNet, etc.); through multiple layers of convolutional layers, activation layers, and pooling layers, the feature map of the entire image is obtained; the feature map is input into the screening region network (ZF / SSD / RPN, etc.), and after calculation, the proposal region is obtained; then, convolution and pooling operations are performed on the proposal region to obtain the regional feature map (proposal feature) of the proposal region, and then the regional feature map (proposal feature) is sent to the subsequent full connection and softmax network for classification (classification, that is, what kind of defect the proposal is), and the defect category with the highest probability is obtained as the final classification result, and the category and probability are recorded. In addition, the coordinates and size of the proposal region (proposal region) represent the location and size of the defect. The method for identifying product defect content based on the defect model may adopt similar variations of the above method or other methods known to those skilled in the art, and the present disclosure does not limit this.

[0145] In an example implementation of the present disclosure, the backend and the server may be a set of distributed system architectures, which are interconnected through a network to transmit messages and communicate and coordinate their behaviors to form a system. Components interact with each other to achieve a common goal. The network may be an Internet of Things (IoT) based on the Internet and / or telecommunications network, which may be a wired network or a wireless network, for example, it may be an electronic network that can realize information exchange functions, such as a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a cellular data communication network, etc. A distributed computing system may have software components, such as software objects or other types of individually addressable isolated entities, such as distributed objects, agents, actors, virtual components, etc. Typically, each such component is individually addressable and has a unique identity (such as an integer, GUID, string, or opaque data structure, etc.) in a distributed system that allows geographic distribution, applications may be deployed and reside in a cluster. There are various systems, components, and network configurations that support distributed computing environments. For example, computing systems may be connected together through a wired or wireless system, through a local network or a widely distributed network. Currently, many networks are coupled to the Internet, which provides the infrastructure for widely distributed computing and includes many different networks, although any network infrastructure may be used for communications such as those that occur in the systems described in the various examples.

[0146] The above-mentioned backend, TMS system (reasoning system) and server provide sharing of computer resources and services through communication exchanges between computing devices and systems. These resources and services include information exchange for objects (such as files), cache storage devices and disk storage devices. These resources and services also include sharing of processing power across multiple processing units for load balancing, resource expansion, specialization of processing, etc.

[0147] In this example implementation, refer to Figure 7As shown, the server 710 receives multiple reasoning tasks sent by the reasoning system 730 and multiple training tasks sent by the training system 740, and assigns them to different nodes of the processor 720 for processing, completes the scheduling of tasks, and can ensure the timely execution of multiple reasoning tasks and training tasks. For example, the first node, the second node, the third node, the fourth node, the fifth node, the sixth node and the seventh node in the processor 720 are all used to perform reasoning tasks, then the first node, the second node, the third node, the fourth node, the fifth node, the sixth node and the seventh node can all access the reasoning algorithm repository 209. The eighth node is used to perform training tasks, and the eighth node can interact with the training algorithm repository 208 so that the eighth node can complete the relevant training tasks. In this example implementation, the first node, the second node, the third node, the fourth node, the fifth node, the sixth node and the seventh node can all be GPU nodes for performing training tasks and reasoning tasks.

[0148] In this example implementation, refer to Figure 8 As shown, the server 710 can complete the scheduling of training tasks and inference tasks according to the type of node and the task volume of the node, wherein the nodes of the inference task can call the inference algorithm from the inference algorithm repository 209, and the training tasks can call the training algorithm from the training algorithm repository 208.

[0149] In this example implementation, refer to Fig. 9 As shown, the reasoning system 730 and the training system 740 can communicate with the server 710 through the gateway 750, send the reasoning tasks and the training tasks to the server 710, and the server 710 will place and execute the tasks according to the status of each node in the processor 720.

[0150] The following describes an embodiment of the device disclosed in the present invention, which can be used to execute the above-mentioned task processing method based on defect detection in the present invention. In addition, in an exemplary embodiment of the present invention, a task processing device based on defect detection is also provided. Fig.10 As shown, the task processing device 1000 based on defect detection includes: a communication node 1010 and a task management node 1020 .

[0151] Among them, the communication node 1010 can be used to receive the detection task and identify the task type of the detection task; the task management node 1020 can be used to store the detection task in the task queue when the task type is the target task type; and when the processor is idle, execute the detection task in a preset order and generate a feedback signal.

[0152] In this example implementation, refer to Fig.11As shown, the above-mentioned device may further include a task agent node 1030, and the task agent node 1030 may be used to receive the feedback signal and send the feedback signal through the communication module.

[0153] Since the various functional modules of the task processing device based on defect detection in the example embodiment of the present disclosure correspond to the steps of the example embodiment of the task processing method based on defect detection described above, for details not disclosed in the embodiment of the device of the present disclosure, please refer to the embodiment of the task processing method based on defect detection described above in the present disclosure.

[0154] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0155] Furthermore, in an exemplary embodiment of the present disclosure, a defect detection-based task processing device capable of implementing the above-mentioned defect detection-based task processing is also provided.

[0156] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware embodiments, complete software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0157] Refer to the following Fig.12 1 and 2 to describe a task processing device 1200 based on defect detection according to such an embodiment of the present disclosure. Fig.12 The displayed defect detection-based task processing device 1200 is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0158] like Fig.12 As shown, the task processing device 1200 based on defect detection is presented in the form of a general-purpose computing device. The components of the task processing device 1200 based on defect detection may include, but are not limited to: the at least one processing unit 1210, the at least one storage unit 1220, a bus 1230 connecting different system components (including the storage unit 1220 and the processing unit 1210), and a display unit 1240.

[0159] The storage unit stores a program code, which can be executed by the processing unit 1210, so that the processing unit 1210 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 1210 can perform the following steps: Figure 1 Step S120 shown in: receiving a detection task and determining a task type of the detection task; S120: if the task type is a target task type, storing the detection task in a task queue; S130: when the processor is idle, executing the detection task in a preset order and generating a feedback signal.

[0160] The storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 1221 and / or a cache memory unit 1222 , and may further include a read-only memory unit (ROM) 1223 .

[0161] The storage unit 1220 may also include a program / utility 1224 having a set (at least one) of program modules 1225, such program modules 1225 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0162] Bus 1230 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0163] The defect detection-based task processing device 1200 may also communicate with one or more external devices 1270 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the defect detection-based task processing device 1200, and / or may communicate with any device that enables the defect detection-based task processing device 1200 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1250. Furthermore, the defect detection-based task processing device 1200 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 1260. As shown, the network adapter 1260 communicates with other modules of the defect detection-based task processing device 1200 via a bus 1230. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in conjunction with the defect detection-based task processing device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0164] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0165] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible embodiments, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to perform the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0166] In this example implementation, the processor may include a first processor and a second processor, wherein the first processor is used to perform the above-mentioned reasoning tasks and training tasks, and the first processor includes a plurality of nodes, which may be divided into reasoning task nodes and training task nodes, the reasoning task nodes are used to perform reasoning tasks, and the training task nodes are used to perform training tasks. The second processor is used to perform detection tasks of reference task types, which may specifically be training status query tasks, early termination of training tasks, node information query tasks (querying the unique identification number and type of the current GPU node), task queue clearing tasks clear, and task status query tasks, etc. The execution process of the multiple tasks will not conflict with each other, and the timeliness of task execution can be guaranteed. Each of the first processors may include at least one graphics processor.

[0167] Reference Fig.13 , describes a program product 1300 for implementing the above method according to an embodiment of the present disclosure, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, apparatus, or device.

[0168] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. Examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0169] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0170] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0171] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0172] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0173] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0174] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A task processing method based on defect detection, wherein: include: Receiving a detection task and determining a task type of the detection task; If the task type is a target task type, storing the detection task in a task queue; When the processor is idle, the detection tasks are executed in a preset order and a feedback signal is generated; The detection task of the target task type includes an inference task and a training task; and executing the training task includes: Modifying the configuration information according to the preset rules based on the product information in the detection task, wherein the configuration information is modified according to the preset rules based on the product information in the detection task, including: extracting the product information field in the detection task to obtain the product information; setting the training image size, the number of training times, the number of tests, the defect threshold and the learning rate reduction strategy according to the preset rules based on the product information; Acquire training data and an initial model according to the product information; Modify the image size in the training data to the size corresponding to the configuration information; The modified training data is used to train the initial model according to the configuration information to obtain a target model and store it.

2. The method according to claim 1, wherein: The task type includes the target task type and the reference task type; If the task type is a reference task type, the processor directly executes the detection task.

3. The method according to claim 1, wherein: The performing of the detection tasks in a preset order includes: The detection tasks are executed in the order in which they are stored in the task queue.

4. The method according to claim 1, wherein: The training task includes a training data acquisition address and model data information; the acquisition of training data and an initial model according to the product information includes: Download the training data according to the training data acquisition address; Obtain model data information according to the model data information field; The initial model is acquired according to the model data information.

5. The method according to claim 1, wherein: The training data includes a label file and an image file. Before the initial model is trained using the training data according to the configuration information to obtain a target model and store it, the method further includes: Preprocessing the training data; Wherein, the preprocessing of the training data includes: Delete abnormal images in image files; Delete image files that do not need to match label files; Match image files with label files; The training data is divided into a training set, a validation set and a test set according to a preset ratio.

6. The method according to claim 5, wherein: The using the training data to train the initial model according to the configuration information to obtain the target model and storing the target model comprises: Training the initial model according to the training set, the validation set, and the number of training times; The test set and the test times are used to complete the test of the trained initial model to obtain the target model and store it.

7. The method according to claim 1, wherein: The detection task includes a task type tag, and determining the task type of the detection task includes: A task type tag in the detection task is identified to determine the task type of the detection task.

8. The method according to claim 1, wherein: The performing of the training task further comprises: Upload the target model and training results to a preset address; The training results include the accuracy, recall and F1 score of the target model.

9. The method according to claim 1, wherein: Executing the reasoning task includes: Acquire the target model and image information of the product according to the product information; The image information is input into the target model to obtain the defect location and defect type of the product.

10. The method according to claim 8, wherein: The reasoning task includes an image information acquisition address, and acquiring the image information of the product according to the product information includes: The image information is downloaded according to the image information acquisition address.

11. The method according to claim 8, wherein: Acquiring the target model according to the product information includes: Detecting whether there is a target model corresponding to the product information in the local repository; If the target model corresponding to the product information exists in the local repository, loading the target model into the local repository; If the target model corresponding to the product information does not exist in the local repository, the local repository downloads the target model corresponding to the product information according to the product information.

12. The method according to claim 11, wherein: The local repository includes a preset number of target models. When the number of target models in the local repository is greater than the preset number, the local repository is updated according to the time sequence in which the target models are used so that the number of target models in the local repository remains at the preset number.

13. The method according to claim 9, wherein: Executing the reasoning task also includes: The feedback signal is generated according to the defect location and the defect type of the product.

14. A task processing device based on defect detection, wherein: include: A communication node, used for receiving a detection task and identifying a task type of the detection task; A task management node, used for storing the detection task in a task queue when the task type is a target task type; and for executing the detection task in a preset order and generating a feedback signal when the processor is idle; The task management node is also used for: Modifying the configuration information according to the preset rules based on the product information in the detection task, wherein the configuration information is modified according to the preset rules based on the product information in the detection task, including: extracting the product information field in the detection task to obtain the product information; setting the training image size, the number of training times, the number of tests, the defect threshold and the learning rate reduction strategy according to the preset rules based on the product information; Acquire training data and an initial model according to the product information; Modify the image size in the training data to the size corresponding to the configuration information; The modified training data is used to train the initial model according to the configuration information to obtain a target model and store it.

15. The device according to claim 14, wherein: The device also includes: The task agent node is used to receive the feedback signal and send the feedback signal through the communication module.

16. A task processing device based on defect detection, wherein: include: processor; as well as A memory for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the task processing method based on defect detection as described in any one of claims 1 to 13.

17. The task processing device based on defect detection according to claim 16, wherein: The task type includes the target task type and the reference task type; the processor includes: A first processor, the first processor comprising at least one graphics processor, configured to execute the reasoning task and the training task; The second processor is used to execute a detection task of a reference task type.

18. A non-volatile computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the task processing method based on defect detection as described in any one of claims 1 to 13 is implemented.

Images