A method and related equipment for evaluating iterative differences in inspection algorithms

Abstract

An embodiment of the present invention discloses a method for evaluating iterative differences in inspection algorithms and related equipment, the method comprising: calling an initial algorithm to analyze a preset inspection image data set to obtain a first result set, wherein the preset inspection image data set is a data set of defect images having the same defect characteristics; calling a target algorithm to analyze a preset inspection image data set to obtain a second result set, wherein the target algorithm is an algorithm after iteration of the initial algorithm; completing a single analysis and evaluation of the target algorithm based on the first result set and the second result set; classifying the first result set and the second result set according to preset classification evaluation indicators, respectively, to obtain first classification data and second classification data, and completing an overall evaluation of the target algorithm based on the first classification data and the second classification data; completing an evaluation of iterative differences in inspection algorithms based on the single analysis and evaluation and the overall evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of inspection and emergency dispatch, and in particular to a method for evaluating iterative differences in an inspection algorithm and related equipment. Background Art

[0002] With technological advancements, electronic devices are developing at an ever-increasing pace, and the corresponding software is also undergoing iterative upgrades. In the field of monitoring, an increasing number of algorithms are being used to troubleshoot various issues. As monitoring applications expand, intelligent analysis capabilities are becoming increasingly advanced, diverse, and complex. When the functions are limited and simple, manual inspection is inexpensive, efficient, and fast. However, as inspections require more and more functions, manual inspections fall far short of meeting the requirements for algorithm iteration. This makes it time-consuming and labor-intensive, leading to delayed algorithm improvements and a decline in the ability to meet demand. Summary of the Invention

[0003] In view of this, the present invention provides a method and related equipment for evaluating iteration differences of inspection algorithms, which are used to solve the problem in the prior art that verification of algorithm iteration direction is time-consuming and labor-intensive, resulting in untimely algorithm improvement.

[0004] To achieve one, some, or all of the above objectives, or other objectives, the present invention provides a method for evaluating iteration differences of an inspection algorithm. The method determines whether the iteration is effective by reasonably evaluating various properties after the algorithm iteration. The method comprises: calling an initial algorithm to analyze a preset inspection image dataset to obtain a first result set, wherein the preset inspection image dataset is a dataset of defect images having the same defect characteristics;

[0005] Invoking a target algorithm to analyze a preset inspection image data set to obtain a second result set, wherein the target algorithm is an iterative algorithm of the initial algorithm;

[0006] completing a single analysis and evaluation of the target algorithm based on the first result set and the second result set;

[0007] Classify the first result set and the second result set according to preset classification evaluation indicators to obtain first classification data and second classification data, and complete an overall evaluation of the target algorithm based on the first classification data and the second classification data;

[0008] The evaluation of iteration differences of the inspection algorithm is completed based on the single analysis evaluation and the overall evaluation.

[0009] Optionally, before the step of calling the initial algorithm to analyze the preset inspection image data set to obtain the first result set, the method further includes:

[0010] Acquire the preset defect image set, and identify target defect features common to all images in the preset defect image set;

[0011] Calling the initial algorithm to screen out a target image dataset that also has the target defect feature from an original inspection image dataset, wherein the original inspection image dataset includes all images acquired by an image acquisition device;

[0012] The preset defect image set and the target image data set are combined to obtain the preset inspection image data set.

[0013] Optionally, the step of completing a single analysis and evaluation of the target algorithm based on the first result set and the second result set includes:

[0014] Acquire first target data obtained by analyzing the target image using the initial algorithm in the first result set;

[0015] Acquire second target data obtained by analyzing the target image with the target algorithm in the second result set;

[0016] Calculate the mean absolute error, root mean square error, and mean absolute percentage error of each test score of the target image based on the first target data and the second target data;

[0017] A single analytical evaluation of the target algorithm is completed based on the mean absolute error, the root mean square error, and the mean absolute percentage error.

[0018] Optionally, the step of classifying the first result set and the second result set according to preset classification evaluation indicators to obtain first classification data and second classification data includes:

[0019] Classifying the first result set according to preset classification evaluation indicators to obtain the first classification data, wherein the preset classification evaluation indicators include true positive examples, false positive examples, true negative examples, and false negative examples;

[0020] The second result set is classified according to a preset classification evaluation index to obtain the second classification data.

[0021] Optionally, the step of completing an overall evaluation of the target algorithm based on the first classification data and the second classification data includes:

[0022] Calculating a first attribute set of the initial algorithm based on the first classification data, the first attribute set including the accuracy of the initial algorithm, the precision of the initial algorithm, and the recall of the initial algorithm;

[0023] Calculating a second attribute set of the target algorithm based on the second classification data, the second attribute set including the accuracy of the target algorithm, the precision of the target algorithm, and the recall rate of the target algorithm;

[0024] An overall evaluation of the initial algorithm and the target algorithm is completed according to the first attribute set and the second attribute set.

[0025] Optionally, before the step of completing the evaluation of the inspection algorithm iteration difference based on the single analysis evaluation and the overall evaluation, the method further includes:

[0026] Obtaining first time-consuming data of a process in which the initial algorithm completes the analysis of a preset inspection image data set;

[0027] Obtaining second time-consuming data of a process in which the target algorithm completes the analysis of a preset inspection image data set;

[0028] Completing an efficiency evaluation of the target algorithm based on the first time consumption data and the second time consumption data;

[0029] Obtaining maximum running memory data of the initial algorithm and the target algorithm under different operating systems respectively, and completing an operation evaluation of the target algorithm based on the maximum running memory data;

[0030] A computer knowledge evaluation of the target algorithm is completed according to the efficiency evaluation and the operation evaluation.

[0031] Optionally, the step of completing the evaluation of iteration differences of the inspection algorithm based on the single analysis evaluation and the overall evaluation includes:

[0032] The evaluation of iteration differences of the inspection algorithm is completed based on the computer knowledge evaluation, the single analysis evaluation and the overall evaluation.

[0033] On the other hand, the present application provides a device for evaluating iteration differences of a patrol algorithm, the device comprising:

[0034] An initial algorithm analysis module, configured to call an initial algorithm to analyze a preset inspection image data set to obtain a first result set, wherein the preset inspection image data set is a data set of defect images having the same defect characteristics;

[0035] A target algorithm analysis module is used to call a target algorithm to analyze a preset inspection image data set to obtain a second result set, wherein the target algorithm is an algorithm after iteration of the initial algorithm;

[0036] A single evaluation module, configured to complete a single analysis and evaluation of the target algorithm based on the first result set and the second result set;

[0037] an overall evaluation module, configured to classify the first result set and the second result set according to preset classification evaluation indicators, obtain first classification data and second classification data, and complete an overall evaluation of the target algorithm based on the first classification data and the second classification data;

[0038] An integration module is used to complete the evaluation of iterative differences of the inspection algorithm based on the single analysis evaluation and the overall evaluation.

[0039] In a third aspect, the present application provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for evaluating the iteration differences of the inspection algorithm as described above are performed.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for evaluating iteration differences of a patrol algorithm as described above are executed.

[0041] The implementation of the present invention will have the following beneficial effects:

[0042] The first result set is obtained by calling an initial algorithm to analyze a preset inspection image dataset, wherein the preset inspection image dataset is a dataset of defect images with the same defect characteristics; the second result set is obtained by calling a target algorithm to analyze the preset inspection image dataset, wherein the target algorithm is an algorithm after iteration of the initial algorithm; a single analysis and evaluation of the target algorithm is completed based on the first result set and the second result set; the first result set and the second result set are respectively classified according to preset classification evaluation indicators to obtain first classification data and second classification data, and an overall evaluation of the target algorithm is completed based on the first classification data and the second classification data; and an evaluation of the iteration difference of the inspection algorithm is completed based on the single analysis evaluation and the overall evaluation. By calculating various attributes of the target algorithm after iteration and evaluating various attributes, namely, the evaluation of the single analysis, the evaluation of the overall analysis, and the evaluation of computer knowledge, the correctness of the evolution direction of the target algorithm is judged based on the evaluation results of various attributes, thereby avoiding the inconvenience of manual detection and being able to timely determine whether the evolution direction of the algorithm is correct, thereby facilitating the iteration of the algorithm and ensuring that the iterated algorithm meets actual needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] in:

[0045] Figure 1 This is a flowchart of a method for evaluating iteration differences of an inspection algorithm provided by an embodiment of the present application;

[0046] Figure 2 1 is a schematic diagram of the structure of a device for evaluating iterative differences in an inspection algorithm provided in an embodiment of the present application;

[0047] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0048] Figure 4 It is a structural diagram of a storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0050] like Figure 1 As shown, the embodiment of the present application provides a method for evaluating iteration differences of an inspection algorithm, including:

[0051] S101, calling an initial algorithm to analyze a preset inspection image dataset to obtain a first result set, where the preset inspection image dataset is a dataset of defect images having the same defect characteristics;

[0052] S102, calling a target algorithm to analyze a preset inspection image data set to obtain a second result set, wherein the target algorithm is an iterative algorithm of the initial algorithm;

[0053] S103, completing a single analysis and evaluation of the target algorithm based on the first result set and the second result set;

[0054] S104: Classify the first result set and the second result set according to preset classification evaluation indicators to obtain first classification data and second classification data, and complete an overall evaluation of the target algorithm based on the first classification data and the second classification data;

[0055] S105 : Complete the evaluation of inspection algorithm iteration differences based on the single analysis evaluation and the overall evaluation.

[0056] The first result set is obtained by calling an initial algorithm to analyze a preset inspection image dataset, wherein the preset inspection image dataset is a dataset of defect images with the same defect characteristics; the second result set is obtained by calling a target algorithm to analyze the preset inspection image dataset, wherein the target algorithm is an algorithm after iteration of the initial algorithm; a single analysis and evaluation of the target algorithm is completed based on the first result set and the second result set; the first result set and the second result set are respectively classified according to preset classification evaluation indicators to obtain first classification data and second classification data, and an overall evaluation of the target algorithm is completed based on the first classification data and the second classification data; and an evaluation of the iteration difference of the inspection algorithm is completed based on the single analysis evaluation and the overall evaluation. By calculating various attributes of the target algorithm after iteration and evaluating various attributes, namely, the evaluation of the single analysis, the evaluation of the overall analysis, and the evaluation of computer knowledge, the correctness of the evolution direction of the target algorithm is judged based on the evaluation results of various attributes, thereby avoiding the inconvenience of manual detection and being able to timely determine whether the evolution direction of the algorithm is correct, thereby facilitating the iteration of the algorithm and ensuring that the iterated algorithm meets actual needs.

[0057] In a possible implementation, before the step of calling the initial algorithm to analyze the preset inspection image data set to obtain the first result set, the method further includes:

[0058] Acquire the preset defect image set, and identify target defect features common to all images in the preset defect image set;

[0059] Calling the initial algorithm to screen out a target image dataset that also has the target defect feature from an original inspection image dataset, wherein the original inspection image dataset includes all images acquired by an image acquisition device;

[0060] The preset defect image set and the target image data set are combined to obtain the preset inspection image data set.

[0061] Exemplarily, a portion of images with monitoring quality problems, i.e., defective images, such as images with no signal features, blur features, and missing monitoring screen features, are manually selected from all monitoring images to obtain a preset defective image set; an initial algorithm is used to screen out a target image data set that also has the target defect features, i.e., no signal features, blur features, and missing monitoring screen features, from the original inspection image data set. The original inspection image data set includes all images captured by the image acquisition device, and false alarm images are not deleted; the preset defective image set and the target image data set are merged to obtain the preset inspection image data set.

[0062] In one possible implementation, the step of completing a single analysis and evaluation of the target algorithm based on the first result set and the second result set includes:

[0063] Acquire first target data obtained by analyzing the target image using the initial algorithm in the first result set;

[0064] Acquire second target data obtained by analyzing the target image with the target algorithm in the second result set;

[0065] Calculate the mean absolute error, root mean square error, and mean absolute percentage error of each test score of the target image based on the first target data and the second target data;

[0066] A single analytical evaluation of the target algorithm is completed based on the mean absolute error, the root mean square error, and the mean absolute percentage error.

[0067] Exemplarily, the first target data obtained by analyzing the target image by the initial algorithm in the first result set is shown in the following table:

[0068] Image ID Vague Color cast brightness 1 70 80 90

[0069] The second target data obtained by analyzing the target image by the target algorithm in the second result set is shown in the following table:

[0070] Image ID Vague Color cast brightness 1 75 85 95

[0071] Calculate the mean absolute error (MAE) as:

[0072] [(75-70)+(85-80)+(95-90)] / 3

[0073] Calculate the root mean square error (RMSE) as:

[0074] sqrt{[(75-70)+(85-80)+(95-90)]^2 / 3}

[0075] Calculate the mean absolute percentage error (MAPE) as:

[0076] [(75-70) / 75+(85-80) / 85+(95-90) / 95] / 3*100

[0077] The mean absolute error, the root mean square error and the mean absolute percentage error are compared with the standard value. If the difference between the mean absolute error, the root mean square error and the mean absolute percentage error and the standard value is greater than 5% times the standard value, it is considered that the difference of the single analysis of the target algorithm is too large, that is, the single analysis evaluation of the target algorithm is completed, and the standard value is the first target data obtained by the initial algorithm analyzing the target image in the first result set.

[0078] In a possible implementation, the step of classifying the first result set and the second result set according to preset classification evaluation indicators to obtain first classification data and second classification data includes:

[0079] Classifying the first result set according to preset classification evaluation indicators to obtain the first classification data, wherein the preset classification evaluation indicators include true positive examples, false positive examples, true negative examples, and false negative examples;

[0080] The second result set is classified according to a preset classification evaluation index to obtain the second classification data.

[0081] Exemplarily, the first result set is classified according to a preset classification evaluation index to obtain the first classification data, and the preset classification evaluation index includes:

[0082] True positive TP: Positive samples are marked as positive examples. Taking image detection as an example, blurred images are detected as true positive TP;

[0083] False positive FP: A false positive sample is marked as a positive example. Taking image detection as an example, a normal image is detected as a blurred image or a color-cast image, which is a false positive FP.

[0084] True negative example TN: false example samples are marked as false examples. Taking image detection as an example, a normal image detected as a normal image is a true negative example TN;

[0085] False negatives (FN): Positive samples are marked as false positives. For example, in image detection, blurred images or color-shifted images are detected as normal images, which are false negatives (FN).

[0086] Similarly, the second result set is classified according to the preset classification evaluation index to obtain the second classification data.

[0087] In a possible implementation, the step of completing an overall evaluation of the target algorithm based on the first classification data and the second classification data includes:

[0088] Calculating a first attribute set of the initial algorithm based on the first classification data, the first attribute set including the accuracy of the initial algorithm, the precision of the initial algorithm, and the recall of the initial algorithm;

[0089] Calculating a second attribute set of the target algorithm based on the second classification data, the second attribute set including the accuracy of the target algorithm, the precision of the target algorithm, and the recall rate of the target algorithm;

[0090] An overall evaluation of the initial algorithm and the target algorithm is completed according to the first attribute set and the second attribute set.

[0091] For example, the accuracy is calculated as the ratio of the number of correct predictions to the total number of predictions. Accuracy = Number of correct predictions / Total number of predictions =

[0092] Correct predictions / total predictions

[0093] Where TP = true positive, TN = true negative, FP = false positive, and FN = false negative, we get:

[0094] Accuracy=(TP+TN) / (TP+TN+FP+FN)

[0095] Calculate the precision:

[0096] The proportion of samples identified as positive is the positive class. Taking image detection as an example: if 10 blurred images are detected, of these 10, 6 are truly blurred and 4 are normal images. Substituting this into the following equation, we get: 6 / (6+4)=60%. The algorithm's accuracy in predicting blurred images is 60%, which means that the probability of judging a normal image as blurred is low. The formula for calculating the accuracy is:

[0097] Precision = TP / (TP / FP)

[0098] Calculate the recall rate (Recall):

[0099] The recall rate is the proportion of all positive samples that are correctly identified as positive. For example, in image detection, if 10 blurred images are detected, 6 of them are truly blurred and 4 are normal images. However, 2 of the images that are judged to be normal are blurry. The recall rate is 6 / (6+2)=75%, and the percentage of blurred images that the algorithm can correctly identify is 75%. The formula for calculating the recall rate is:

[0100] Recall = TP / (TP+FN)

[0101] In a possible implementation, before the step of completing the evaluation of the inspection algorithm iteration difference based on the single analysis evaluation and the overall evaluation, the method further includes:

[0102] Obtaining first time-consuming data of a process in which the initial algorithm completes the analysis of a preset inspection image data set;

[0103] Obtaining second time-consuming data of a process in which the target algorithm completes the analysis of a preset inspection image data set;

[0104] Completing an efficiency evaluation of the target algorithm based on the first time consumption data and the second time consumption data;

[0105] Obtaining maximum running memory data of the initial algorithm and the target algorithm under different operating systems respectively, and completing an operation evaluation of the target algorithm based on the maximum running memory data;

[0106] A computer knowledge evaluation of the target algorithm is completed according to the efficiency evaluation and the operation evaluation.

[0107] For example, the time taken to complete the same sample detection is calculated, that is, the time taken for each algorithm to complete the analysis process for the same data set; the analysis process includes the time taken to perform all analyses such as blur, color cast, brightness anomaly, and screen distortion on each image in the same data set;

[0108] Exemplarily, the maximum running memory during the running of the initial algorithm and the target algorithm is recorded:

[0109] The maximum running memory of the initial algorithm and the target algorithm during the execution of the analysis using a static analysis tool under Linux;

[0110] The maximum running memory used by the initial algorithm and the target algorithm during execution when using a static analysis tool for analysis under Windows.

[0111] In a possible implementation, the step of completing the evaluation of inspection algorithm iteration differences based on the single analysis evaluation and the overall evaluation includes:

[0112] The evaluation of iteration differences of the inspection algorithm is completed based on the computer knowledge evaluation, the single analysis evaluation and the overall evaluation.

[0113] For example, after the algorithm is iterated, there may be problems in handling certain abnormal situations, but there is no overall impact. Therefore, the target algorithm is evaluated by computer knowledge and the single analysis evaluation is performed to make the evaluation of the algorithm after iteration, that is, the target algorithm, more accurate and reliable.

[0114] In one possible implementation, Figure 2 As shown, the present application provides a device for evaluating iteration differences of a patrol algorithm, the device comprising:

[0115] An initial algorithm analysis module 201 is configured to call an initial algorithm to analyze a preset inspection image dataset to obtain a first result set, wherein the preset inspection image dataset is a dataset of defect images having the same defect characteristics;

[0116] A target algorithm analysis module 202 is configured to call a target algorithm to analyze a preset inspection image data set to obtain a second result set, wherein the target algorithm is an iterative algorithm of the initial algorithm;

[0117] A single evaluation module 203, configured to complete a single analysis and evaluation of the target algorithm based on the first result set and the second result set;

[0118] An overall evaluation module 204 is configured to classify the first result set and the second result set according to a preset classification evaluation index to obtain first classification data and second classification data, and to complete an overall evaluation of the target algorithm based on the first classification data and the second classification data;

[0119] The integration module 205 is configured to complete the evaluation of iteration differences of the inspection algorithm based on the single analysis evaluation and the overall evaluation.

[0120] In one possible implementation, Figure 3As shown, an embodiment of the present application provides an electronic device 300, comprising: a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it implements: calling an initial algorithm to analyze a preset inspection image data set to obtain a first result set, where the preset inspection image data set is a data set of defect images having the same defect characteristics; calling a target algorithm to analyze a preset inspection image data set to obtain a second result set, where the target algorithm is an algorithm after iteration of the initial algorithm; completing a single analysis and evaluation of the target algorithm based on the first result set and the second result set; classifying the first result set and the second result set according to preset classification evaluation indicators, respectively, to obtain first classification data and second classification data, and completing an overall evaluation of the target algorithm based on the first classification data and the second classification data; and completing the step of evaluating the iterative differences of the inspection algorithm based on the single analysis and evaluation and the overall evaluation.

[0121] In one possible implementation, Figure 4 As shown, an embodiment of the present application provides a computer-readable storage medium 400, on which a computer program 411 is stored, which, when executed by a processor, implements: calling an initial algorithm to analyze a preset inspection image data set to obtain a first result set, wherein the preset inspection image data set is a data set of defect images having the same defect characteristics; calling a target algorithm to analyze a preset inspection image data set to obtain a second result set, wherein the target algorithm is an algorithm after iteration of the initial algorithm; completing a single analysis and evaluation of the target algorithm based on the first result set and the second result set; classifying the first result set and the second result set according to preset classification evaluation indicators, respectively, to obtain first classification data and second classification data, and completing an overall evaluation of the target algorithm based on the first classification data and the second classification data; completing the step of evaluating the iterative differences of the inspection algorithm based on the single analysis and evaluation and the overall evaluation.

[0122] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0123] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0124] Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0125] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0126] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computing device. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they can be implemented using program code executable by a computer device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

[0127] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

[0128] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for evaluating iterative differences in an inspection algorithm, characterized in that: include: Invoking an initial algorithm to analyze a preset inspection image dataset to obtain a first result set, where the preset inspection image dataset is a dataset of defect images having the same defect characteristics; Invoking a target algorithm to analyze a preset inspection image data set to obtain a second result set, wherein the target algorithm is an iterative algorithm of the initial algorithm; completing a single analysis and evaluation of the target algorithm based on the first result set and the second result set; Classify the first result set and the second result set according to preset classification evaluation indicators to obtain first classification data and second classification data, and complete an overall evaluation of the target algorithm based on the first classification data and the second classification data; Completing an evaluation of inspection algorithm iteration differences based on the single analysis evaluation and the overall evaluation; The step of completing a single analysis and evaluation of the target algorithm based on the first result set and the second result set includes: Acquire first target data obtained by analyzing the target image using the initial algorithm in the first result set; Acquire second target data obtained by analyzing the target image with the target algorithm in the second result set; Calculate the mean absolute error, root mean square error, and mean absolute percentage error of each test score of the target image based on the first target data and the second target data; The mean absolute error, the root mean square error and the mean absolute percentage error are compared with the standard value. If the difference between the mean absolute error, the root mean square error and the mean absolute percentage error and the standard value is greater than 5% times the standard value, it is considered that the difference of the single analysis of the target algorithm is too large, that is, the single analysis evaluation of the target algorithm is completed.

2. The method for evaluating iteration differences of inspection algorithms according to claim 1, wherein: Before the step of calling the initial algorithm to analyze the preset inspection image data set to obtain the first result set, the method further includes: Acquire the preset defect image set, and identify target defect features common to all images in the preset defect image set; Calling the initial algorithm to screen out a target image dataset that also has the target defect feature from an original inspection image dataset, wherein the original inspection image dataset includes all images acquired by an image acquisition device; The preset defect image set and the target image data set are combined to obtain the preset inspection image data set.

3. The method for evaluating iteration differences of inspection algorithms according to claim 1, wherein: The step of classifying the first result set and the second result set according to preset classification evaluation indicators to obtain first classification data and second classification data includes: Classifying the first result set according to preset classification evaluation indicators to obtain the first classification data, wherein the preset classification evaluation indicators include true positive examples, false positive examples, true negative examples, and false negative examples; The second result set is classified according to a preset classification evaluation index to obtain the second classification data.

4. The method for evaluating iteration differences of inspection algorithms according to claim 1, wherein: The step of completing an overall evaluation of the target algorithm based on the first classification data and the second classification data includes: Calculating a first attribute set of the initial algorithm based on the first classification data, the first attribute set including the accuracy of the initial algorithm, the precision of the initial algorithm, and the recall of the initial algorithm; Calculating a second attribute set of the target algorithm based on the second classification data, the second attribute set including the accuracy of the target algorithm, the precision of the target algorithm, and the recall rate of the target algorithm; An overall evaluation of the initial algorithm and the target algorithm is completed according to the first attribute set and the second attribute set.

5. The method for evaluating iteration differences of inspection algorithms according to claim 1, wherein: Before the step of completing the evaluation of the iteration difference of the inspection algorithm according to the single analysis evaluation and the overall evaluation, the method further includes: Obtaining first time-consuming data of a process in which the initial algorithm completes the analysis of a preset inspection image data set; Obtaining second time-consuming data of a process in which the target algorithm completes the analysis of a preset inspection image data set; Completing an efficiency evaluation of the target algorithm based on the first time consumption data and the second time consumption data; Obtaining maximum running memory data of the initial algorithm and the target algorithm under different operating systems respectively, and completing an operation evaluation of the target algorithm based on the maximum running memory data; A computer knowledge evaluation of the target algorithm is completed according to the efficiency evaluation and the operation evaluation.

6. The method for evaluating iteration differences of inspection algorithms according to claim 5, wherein: The step of completing the evaluation of the iteration difference of the inspection algorithm according to the single analysis evaluation and the overall evaluation includes: The evaluation of iteration differences of the inspection algorithm is completed based on the computer knowledge evaluation, the single analysis evaluation and the overall evaluation.

7. A device for evaluating iterative differences in an inspection algorithm, characterized in that: The device comprises: An initial algorithm analysis module, configured to call an initial algorithm to analyze a preset inspection image data set to obtain a first result set, wherein the preset inspection image data set is a data set of defect images having the same defect characteristics; A target algorithm analysis module is used to call a target algorithm to analyze a preset inspection image data set to obtain a second result set, wherein the target algorithm is an algorithm after iteration of the initial algorithm; A single evaluation module, configured to complete a single analysis and evaluation of the target algorithm based on the first result set and the second result set; an overall evaluation module, configured to classify the first result set and the second result set according to preset classification evaluation indicators, obtain first classification data and second classification data, and complete an overall evaluation of the target algorithm based on the first classification data and the second classification data; An integration module, configured to complete an evaluation of iteration differences of an inspection algorithm based on the single analysis evaluation and the overall evaluation; The single assessment module is specifically used for: Acquire first target data obtained by analyzing the target image using the initial algorithm in the first result set; Acquire second target data obtained by analyzing the target image with the target algorithm in the second result set; Calculate the mean absolute error, root mean square error, and mean absolute percentage error of each test score of the target image based on the first target data and the second target data; The mean absolute error, the root mean square error and the mean absolute percentage error are compared with the standard value. If the difference between the mean absolute error, the root mean square error and the mean absolute percentage error and the standard value is greater than 5% times the standard value, it is considered that the difference of the single analysis of the target algorithm is too large, that is, the single analysis evaluation of the target algorithm is completed.

8. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for evaluating iterative differences of the inspection algorithm as described in any one of claims 1 to 6 are performed.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, performs the steps of the method for evaluating iteration differences of patrol algorithms according to any one of claims 1 to 6.

Images