Standard knowledge comparison method and device based on multi-source data fusion and storage medium

Abstract

The invention discloses a standard knowledge comparison method and device based on multi-source data fusion and a storage medium, and relates to the technical field of defect detection data processing. The standard knowledge comparison method based on multi-source data fusion comprises the following steps of comparison feature collection, defect feature comparison and comparison judgment and correction. According to the method, image quality judgment is carried out on the defect detection image set to judge whether defect feature comparison is carried out or not, if yes, the defect feature set is compared with the standard defect knowledge base, comparison reliability evaluation is carried out to obtain a comparison reliability result, if not, re-acquisition optimization is carried out, and finally, after defect feature comparison, the defect feature set is subjected to re-acquisition optimization. According to the output defect feature type, a comparison judgment correction measure is adopted to judge the defect feature type, so that the effect of improving the robustness of standard knowledge comparison is achieved; the problem that in the prior art, in the defect detection process through standard defect knowledge base comparison, due to the fact that the comparison information amount is limited, the comparison judgment robustness is low is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection data processing, and particularly relates to a standard knowledge comparison method and device based on multi-source data fusion and a storage medium. BACKGROUND

[0002] With the advent of the big data era, the rapid development of information technology has made various data sources increasingly rich. Multi-source data fusion technology can improve the accuracy of data analysis and the scientificity of decision-making by integrating data from different sources. In this context, the method based on standard knowledge comparison has emerged, which helps to optimize data processing and information extraction by comparing standard knowledge stored in different data sources, providing support for intelligent decision-making, automation systems and other fields. The research of related devices and storage media helps to improve the practical application efficiency of the technology.

[0003] Existing multi-source data fusion technology mainly focuses on integrating data from different sources to improve the accuracy and comprehensiveness of data. However, existing technologies still face challenges in standard knowledge comparison, such as data heterogeneity, timeliness, and consistency issues, resulting in unsatisfactory comparison results. Existing methods rely on manual rules or simple algorithms, lack automated and intelligent processing methods, and are difficult to cope with complex and variable data environments. Although there has been some progress, there is still room for improvement in the accuracy and efficiency of standard knowledge comparison.

[0004] For example, the patent application with publication number CN119783037A discloses a multi-source heterogeneous data fusion and processing method based on big data, which includes: data preprocessing, cleaning, completing and feature extraction of multi-source heterogeneous data to provide standardized and high-quality input for subsequent processing; dynamic knowledge graph modeling, constructing a dynamic knowledge graph based on the semantic relationship of multi-source data to generate a unified semantic representation; cross-modal alignment and deep fusion, performing semantic alignment on multi-modal data and generating a unified comprehensive representation through deep fusion of features; real-time stream processing, dynamically updating the real-time updated stream data to realize real-time fusion of data; adaptive optimization, dynamically adjusting the fusion strategy parameters.

[0005] For example, the invention patent with publication number CN117218079B discloses a vehicle body defect detection method based on machine vision, which includes: selecting an abnormal area through a corresponding comparison method, then selecting a center area from the abnormal area, and selecting a center point, a far point and a near point from the center area, and then constructing a far point line and a near point line by combining the corresponding points, confirming the corresponding K value, and comparing the K value with the corresponding standard area to analyze whether the K value is too large or too small, and then performing overall analysis according to the actual situation to determine whether there is a crack.

[0006] But in the process of implementing the technical scheme of the embodiments of the present application, the applicant finds that the above-mentioned technology at least has the following technical problems:

[0007] In the prior art, in the quality detection comparison link in the intelligent manufacturing field, when comparing defect features with a standard defect knowledge base, a data-driven distribution fitting method is used to set the similarity threshold of each defect feature type for determining the defect feature type. However, due to the fact that the weak influence of each step on the comparison result determination is easily ignored in the comparison process, there is a problem of low robustness of comparison determination caused by limited comparison information in the defect detection process through standard defect knowledge base comparison. SUMMARY

[0008] The embodiments of the present application provide a standard knowledge comparison method based on multi-source data fusion, a device and a storage medium, which solve the problem of low robustness of comparison determination caused by limited comparison information in the defect detection process through standard defect knowledge base comparison in the prior art, and realize the improvement of the robustness of standard knowledge comparison.

[0009] The embodiments of the present application provide a standard knowledge comparison method based on multi-source data fusion, which includes the following steps: acquiring defect detection image sets by comparing and collecting features of products to be compared and detected through multi-angle industrial cameras, judging whether to perform defect feature comparison by performing image quality determination on the defect detection image sets, and performing defect feature comparison for defect feature recognition of the products to be compared and detected; if the defect feature comparison is performed, defect feature sets are extracted from the image sets to compare with a standard defect knowledge base, and comparison reliability evaluation is performed to obtain corresponding comparison reliability results, otherwise, the corresponding products to be compared and detected are re-sampled and optimized, and comparison reliability evaluation is used to evaluate the reliability degree of the defect feature comparison process; after the defect feature comparison, corresponding comparison determination correction measures are taken according to the output results of the defect feature types to determine the defect feature types, and the comparison determination correction measures are used to correct the influence of the influence factors in the comparison process on the initial similarity threshold used for comparison judgment.

[0010] The embodiment of the application provides a standard knowledge comparison device based on multi-source data fusion, comprising a comparison feature acquisition module, a defect feature comparison module and a comparison judgment correction module: the comparison feature acquisition module is used for acquiring comparison features of a to-be-compared detection product by a multi-angle industrial camera to obtain a defect detection image set, performing image quality judgment on the defect detection image set to determine whether to perform defect feature comparison, and performing defect feature recognition on the to-be-compared detection product; the defect feature comparison module is used for, if the defect feature comparison is performed, performing image set feature extraction to obtain a defect feature set for comparison with a standard defect knowledge base, and performing comparison reliability evaluation to obtain a corresponding comparison reliability result, or otherwise, performing resampling optimization on the corresponding to-be-compared detection product; the comparison reliability evaluation is used for evaluating the reliability degree of the defect feature comparison process; and the comparison judgment correction module is used for, after the defect feature comparison, taking corresponding comparison judgment correction measures according to the output defect feature type result to perform defect feature type judgment, and the comparison judgment correction measures are used for correcting the influence of influence factors in the comparison process on the initial similarity threshold used in the comparison judgment.

[0011] The embodiment of the application provides a computer readable storage medium for storing a program, and the program is executed by a processor to realize a standard knowledge comparison method based on multi-source data fusion.

[0012] The one or more technical solutions provided in the embodiment of the application have at least the following technical effects or advantages:

[0013] 1. The defect detection image set obtained by comparison feature acquisition is subjected to image quality judgment to determine whether to perform defect feature comparison, if the defect feature comparison is performed, image set feature extraction is performed to obtain a defect feature set for comparison with a standard defect knowledge base, comparison reliability evaluation is performed to obtain a corresponding comparison reliability result, otherwise, resampling optimization is performed on the corresponding to-be-compared detection product, finally, after the defect feature comparison, corresponding comparison judgment correction measures are taken according to the output defect feature type result to perform defect feature type judgment, so that the corresponding defect type is more accurately output, and the robustness of the standard knowledge comparison is improved, and the problem of low comparison judgment robustness caused by limited comparison information in the defect detection process by comparison with the standard defect knowledge base in the prior art is effectively solved.

[0014] 2、By acquiring and normalizing comparison reliability data, and acquiring comparison reliability analysis proportion and reference comparison time range from a preset database, then range deviation quantization is performed on the comparison time and the reference comparison time range to obtain a corresponding comparison time deviation, and finally, after weighting operation based on the comparison time deviation, comparison sample size and comparison coverage and the corresponding comparison reliability analysis proportion, a comparison reliability evaluation value is obtained by coupling, thereby more accurately quantifying the reliability degree of the defect feature comparison process, and the improvement of the standard knowledge comparison robustness is realized.

[0015] 3、By acquiring and normalizing defect feature type reliability evaluation data corresponding to the candidate defect feature type, then acquiring a corresponding defect feature type evaluation proportion from a preset database, then weighting operation is performed on the defect feature type reliability evaluation data and the corresponding defect feature type evaluation proportion after coupling to obtain a defect feature type reliability initial quantization value, and finally, after compensation operation on the defect feature type reliability initial quantization value and the comparison reliability evaluation value, a defect feature type reliability quantization value is obtained, thereby more accurately quantifying the reliability degree of each type of candidate defect feature type output, and the improvement of the standard knowledge comparison robustness is realized. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A flowchart of a standard knowledge comparison method based on multi-source data fusion provided by the embodiment of the present application;

[0017] Figure 2 A flowchart of comparison reliability evaluation provided by the embodiment of the present application;

[0018] Figure 3 A flowchart of comparison determination correction measures provided by the embodiment of the present application. DETAILED DESCRIPTION

[0019] The embodiment of the application provides a standard knowledge comparison method and device based on multi-source data fusion and a storage medium, solves the problem of low comparison determination robustness caused by limited comparison information in the prior art, performs image quality determination on a defect detection image set obtained by comparison feature collection to determine whether to perform defect feature comparison, if the defect feature comparison is performed, performs feature extraction on the image set to obtain a defect feature set for comparison with a standard defect knowledge base, performs comparison reliability evaluation during comparison of the defect feature set and the standard defect knowledge base, obtains a corresponding comparison reliability evaluation value, if the comparison reliability evaluation value is not less than a reliability determination value obtained from a preset database, outputs a corresponding defect feature type, if the comparison reliability evaluation value is less than the reliability determination value obtained from the preset database, outputs the corresponding defect feature type, and performs difference quantization on the comparison reliability evaluation value and the reliability determination value to obtain a corresponding reliability difference amount, then maps the reliability difference amount in the preset database to obtain a similarity threshold first influence amount, otherwise, optimizes the corresponding to-be-compared detection product, finally, after defect feature comparison, takes corresponding comparison determination correction measures according to the output defect feature type result to determine the defect feature type, and the robustness of standard knowledge comparison is improved.

[0020] The technical solution in the embodiment of the application is used to solve the problem of low comparison determination robustness caused by limited comparison information in the prior art, and the general idea is as follows:

[0021] The obtained defect detection image set is subjected to image quality determination to determine whether to perform defect feature comparison, if yes, the defect feature set is compared with the standard defect knowledge base, comparison reliability evaluation is performed to obtain a comparison reliability result, otherwise, re-sampling optimization is performed, finally, after defect feature comparison, corresponding comparison determination correction measures are taken according to the output defect feature type result to determine the defect feature type, and the robustness of standard knowledge comparison is improved.

[0022] In order to better understand the above technical solution, the above technical solution will be described in detail in combination with the description of the drawings and the specific embodiments.

[0023] As Figure 1As shown, a flowchart of a standard knowledge comparison method based on multi-source data fusion provided by the embodiment of the application is provided. The method is applied to a standard knowledge comparison device based on multi-source data fusion. The method includes the following steps: acquiring a defect detection image set by comparing and collecting features of a product to be compared by a multi-angle industrial camera, and judging whether to perform defect feature comparison by judging the image quality of the defect detection image set. Defect feature comparison is used to identify defect features of the product to be compared.

[0024] If defect feature comparison is performed, defect feature set is extracted from the image set to compare with the standard defect knowledge base, and the corresponding comparison reliability result is obtained by performing comparison reliability evaluation. Otherwise, the corresponding product to be compared is re-sampled and optimized. Comparison reliability evaluation is used to evaluate the reliability degree of the defect feature comparison process.

[0025] After defect feature comparison, the corresponding comparison judgment correction measures are taken according to the output defect feature type result to perform defect feature type judgment. Comparison judgment correction measures are used to correct the influence of influencing factors on the initial similarity threshold used in the comparison judgment.

[0026] In this embodiment, the standard knowledge comparison based on multi-source data fusion means that data from different sources is integrated by a multi-source data fusion method, and then data from different sources is integrated. Then, the data is matched, compared and corrected by a standard knowledge comparison algorithm to ensure the consistency and accuracy of the information. Common application scenarios include intelligent manufacturing, medical health, financial risk control and public security, etc. In the quality detection comparison link in the field of intelligent manufacturing, when comparing defect features with a standard defect knowledge base, a data-driven distribution fitting method is usually used to set the similarity threshold of each defect feature type for defect feature type judgment. The standard defect knowledge base represents a database that integrates and stores various defect features for defect comparison. However, due to the limited number of comparison samples in the standard knowledge base and the unstable timeliness of the update of the standard defect knowledge base, some defect features may not have comparison data or limited comparison data in the standard defect knowledge base, which reduces the robustness of defect comparison judgment on feature data after multi-source data fusion. By considering the influence of image quality, comparison process reliability, etc. on the similarity threshold and correcting it, the identification of various defect features is more accurate, and the robustness of standard knowledge comparison of various defect feature types is improved.

[0027] It should be noted that before designing the standard knowledge comparison method based on multi-source data fusion, the preset personnel establishes a preset database for storing various setting data, which includes but is not limited to image quality reference data, camera shooting angle adjustment amount, reliability determination value, similarity threshold first influence amount, comparison reliability analysis proportion, reference comparison time range, similarity threshold second influence amount, defect feature type evaluation proportion and batch similarity threshold influence amount, etc. Among them, various numerical values are directly set by preset professionals, for example, the reliability determination value is obtained by substituting the comparison reliability data corresponding to the qualified comparison result of each defect type in the historical database into the specific limit expression of the comparison reliability evaluation value, and the mean value operation is performed on the data set to obtain the reliability determination value, which is stored in the preset database in advance.

[0028] Further, the specific steps of image quality judgment of the defect detection image set are as follows:

[0029] S1, obtaining image quality judgment data of each defect detection image in the defect detection image set, the image quality judgment data including image sharpness and image contrast.

[0030] It should be noted that the image sharpness is measured by calculating the gradient of the defect detection image, and the larger the gradient, the clearer the defect detection image; the image contrast is measured by calculating the brightness range (i.e. the difference between the maximum brightness and the minimum brightness) of the defect detection image.

[0031] S2, obtaining image quality reference data from the preset database and comparing with the image quality judgment data, the image quality reference data including image sharpness reference value and image contrast reference value; if all the image quality judgment data are not less than the corresponding image quality reference data, indicating that the corresponding defect detection image quality is qualified, then the defect feature comparison is performed; if there is image quality judgment data less than the corresponding image quality reference data, indicating that the image quality is unqualified, then the defect feature comparison is not performed and the corresponding detection product to be compared is transmitted to the re-inspection conveying belt, and S3 is executed.

[0032] Specifically, the image quality reference data is preset by the preset worker and stored in the preset database.

[0033] S3, obtaining image quality difference amount by deviation quantization of the difference between the image quality judgment data less than the corresponding image quality reference data, and mapping the corresponding camera shooting angle adjustment amount in the preset database based on the image quality difference amount, the camera shooting angle adjustment amount is used to adjust the shooting angle of the camera when collecting the next comparison feature of the corresponding detection product to be detected.

[0034] Specifically, deviation quantization represents the result of performing a ratio calculation between the difference between the image quality reference data and the image quality judgment data, and then comparing it with the corresponding image quality reference data.

[0035] Specifically, the process involves locating the industrial camera at the corresponding angle based on the defect detection image, and then performing a compensation calculation between the camera's shooting angle adjustment and the corresponding angle to obtain an optimized camera shooting angle. Based on the optimized camera shooting angle, defect detection images of the corresponding comparison products are acquired. The compensation calculation refers to adding the camera shooting angle adjustment to the corresponding angle.

[0036] It should be added that a shooting angle adjustment mapping set between image quality difference and camera shooting angle adjustment is constructed in the preset database. The real-time image quality difference is input into the shooting angle adjustment mapping set, and the corresponding camera shooting angle adjustment is output. The shooting angle adjustment mapping set represents the set of mapping relationships between image quality difference and camera shooting angle adjustment. When two image quality difference values ​​are obtained, the data after averaging the two image quality difference values ​​is input into the shooting angle adjustment mapping set, and the corresponding camera shooting angle adjustment is output.

[0037] In this embodiment, the initial quality assessment of the defect detection images helps to screen out defect detection images that do not meet the quality standards, reducing the possibility of errors during subsequent comparisons. At the same time, the qualified defect detection images are directly compared, which improves the comparison and recognition effect. Furthermore, the re-sampling optimization based on the current image quality assessment results helps to improve the quality of the defect detection images acquired subsequently, thereby improving the accuracy of comparison and recognition, and thus improving the robustness of the defect feature recognition process.

[0038] like Figure 2 The diagram shown is a flowchart of the comparison reliability assessment provided in this application embodiment. The specific logic is as follows: a comparison reliability assessment is performed during the comparison of the defect feature set with the standard defect knowledge base to obtain the corresponding comparison reliability assessment value. If the comparison reliability assessment value is not less than the reliability judgment value, the corresponding defect feature type is output. If the comparison reliability assessment value is less than the reliability judgment value, the corresponding defect feature type is output. The reliability difference is obtained by mapping the comparison reliability assessment value and the reliability judgment value to the reliability difference. The first influence value of the similarity threshold is obtained based on the reliability difference. The above process not only improves the reliability of the comparison process, but also improves the robustness of defect comparison judgment.

[0039] Further, the comparison reliability evaluation is performed to obtain a corresponding comparison reliability result, and the specific process is as follows: the comparison reliability evaluation is performed in the process of comparing the defect feature set with the standard defect knowledge base to obtain a comparison reliability evaluation value; the comparison reliability evaluation value is compared with the reliability determination value obtained from the preset database, and there are two cases, and the specific process is as follows:

[0040] In the first case, if the comparison reliability evaluation value is not less than the reliability determination value obtained from the preset database, it indicates that the reliability degree of the comparison process is high, and the corresponding defect feature type is output.

[0041] In the second case, if the comparison reliability evaluation value is less than the reliability determination value obtained from the preset database, it indicates that the reliability degree of the comparison process is low, and the corresponding defect feature type is output, and the difference between the comparison reliability evaluation value and the reliability determination value is quantified to obtain a corresponding reliability difference, and the similarity threshold first influence quantity is obtained by mapping the reliability difference in the preset database. The similarity threshold first influence quantity represents the quantitative value of the influence degree of the influence factor in the comparison process on the similarity threshold.

[0042] Specifically, the difference quantification means that the reliability evaluation index and the reliability determination value are subjected to a subtraction operation.

[0043] Specifically, the similarity threshold first influence mapping set between the reliability difference and the similarity threshold first influence quantity is constructed in the preset database, and the real-time reliability difference is input into the similarity threshold first influence mapping set to output the corresponding similarity threshold first influence quantity. The similarity threshold first influence mapping set represents a set of mapping relationships between the reliability difference and the similarity threshold first influence quantity.

[0044] In this embodiment, by monitoring and analyzing the comparison process of various defect feature types, it is not only beneficial to real-time understanding of the reliability degree of the comparison result of defect identification in quality detection, but also helpful to improve the accuracy of defect identification, so as to timely take corresponding defect comparison optimization, and reduce the influence of the finiteness of the standard defect knowledge base on the comparison result in subsequent analysis, thereby improving the robustness of the standard knowledge comparison.

[0045] Further, the specific acquisition method of the comparison reliability evaluation value is as follows:

[0046] First, the comparison reliability data is obtained and subjected to data normalization processing. The comparison reliability data includes the comparison sample quantity, the comparison time, and the comparison coverage. The comparison coverage represents the ratio of the sample quantity used after completing all defect feature identification to the total sample quantity of the corresponding defect feature type during the identification of the corresponding defect feature type.

[0047] It should be added that the comparison sample quantity is obtained by counting each sample used in the comparison by the counter in real time during the comparison; the comparison time is obtained by recording the duration corresponding to the start time and the end time of the comparison by the timer; and the comparison coverage is obtained by ratio operation of the comparison sample quantity and the total number of samples of the corresponding defect feature type in the standard defect knowledge base.

[0048] Next, the comparison reliability analysis proportion and the reference comparison time range are obtained from the preset database. The comparison reliability analysis proportion includes the comparison sample quantity analysis proportion, the comparison time analysis proportion, and the comparison coverage analysis proportion. The reference comparison time range represents the range corresponding to the minimum value and the maximum value of the comparison time.

[0049] Specifically, the comparison reliability analysis proportion is obtained from the preset database. The comparison reliability analysis proportion represents the influence degree of the comparison reliability data on the comparison reliability evaluation value. Each comparison reliability data and the comparison reliability analysis proportion have a unique mapping relationship, and the value range is between 0 and 1. For example, a mapping set of the comparison reliability data and the preset comparison reliability analysis proportion is constructed. The real-time comparison sample quantity, the comparison time, and the comparison coverage are input into the mapping set to obtain the corresponding comparison sample quantity analysis proportion, the comparison time analysis proportion, and the comparison coverage analysis proportion, which respectively represent the influence degree of the comparison sample quantity, the comparison time, and the comparison coverage on the comparison reliability evaluation value, and the sum of the three is 1.

[0050] Specifically, the reference comparison time range is pre-set by the preset staff based on the analysis of the comparison time in the historical data and stored in the preset database.

[0051] Then, the range deviation of the comparison time and the reference comparison time range is quantified to obtain the corresponding comparison time deviation.

[0052] It should be added that the specific expression of the time deviation is as follows:

[0053] CTD = |CT-CT min |+|CT-CT max |-(CT max -CT min );

[0054] CTD = |CT-CT min CTD = |CT-CT maxCTD represents the alignment time deviation amount corresponding to the defect feature type; and the alignment time has a positive correlation with the influence of the alignment sample amount and the alignment coverage, for example, the lower the alignment time is below the alignment time minimum value, the greater the corresponding time deviation amount is, indicating that the shorter the alignment time is, the less data can be aligned, and the smaller the corresponding alignment sample amount and the alignment coverage can be; on the contrary, the higher the alignment time is above the alignment time maximum value, the greater the corresponding time deviation amount is, indicating that the longer the alignment time is, which indicates that the efficiency of the alignment is abnormal, and the alignment sample amount in the standard defect knowledge base can be limited or the alignment coverage can be insufficient.

[0055] Finally, the alignment reliability evaluation value is obtained by coupling the weighting operation based on the alignment time deviation amount, the alignment sample amount and the alignment coverage, and the corresponding alignment reliability analysis proportion. The alignment reliability evaluation value is used to quantify the reliability degree of the defect feature alignment process.

[0056] The specific limit expression of the alignment reliability evaluation value is as follows:

[0057]

[0058] In the formula, CSS represents the alignment sample amount corresponding to the defect feature type, CCR represents the alignment coverage corresponding to the defect feature type, γ CSS represents the alignment sample amount analysis proportion, γ CCR represents the alignment coverage analysis proportion, and CRA represents the alignment reliability evaluation value corresponding to the defect feature type. S represents the alignment sample amount analysis proportion, γ T represents the alignment time analysis proportion, γ C represents the alignment coverage analysis proportion, and CRA represents the alignment reliability evaluation value corresponding to the defect feature type.

[0059] In the embodiment, the algorithm obtains the alignment reliability evaluation value by analyzing the alignment time deviation amount, the alignment sample amount and the alignment coverage, and the corresponding alignment reliability analysis proportion. In the formula, as the alignment sample amount and the alignment coverage increase, the more data referred to in the alignment process, the greater the corresponding alignment reliability evaluation value is; however, the greater the alignment time deviation amount is, the higher the degree of deviation of the alignment time from the reference alignment time range is, the lower the reliability degree of the comparison result is, and the smaller the corresponding alignment reliability evaluation value is. Through the analysis of the alignment reliability evaluation value, the reliability of the defect feature in the alignment process can be more accurately evaluated, the credibility of the alignment result is increased, and then the similarity threshold used in subsequent alignment is corrected based on the result of the alignment reliability evaluation, which is beneficial to improve the robustness of the defect alignment determination.

[0060] As Figure 3As shown, the flowchart provided by the embodiment of the present application takes the comparison determination correction measure, and the specific logic is: judging whether the number of the output candidate defect feature types is 1, if only one kind of candidate defect feature type is output, the single-class defect feature type determination is performed; if multiple candidate defect feature types are output, the similarity threshold second influence quantity of each candidate defect feature type is obtained by mapping the defect feature type reliability quantitative value, and the multi-class defect feature type determination is performed; wherein, the single-class defect feature type determination is firstly based on the similarity threshold first influence quantity and the similarity threshold of the corresponding defect feature type to obtain the first modified similarity threshold, and then the candidate defect feature type is judged according to the first modified similarity threshold whether it is output as a defect category, if it is output as a defect category, the comparison operation is ended, otherwise the artificial review request is sent; in addition, the multi-class defect feature type determination is firstly based on the similarity threshold first influence quantity to modify the initial similarity threshold of each candidate defect feature type to obtain the corresponding first candidate similarity threshold, then the modified similarity threshold is obtained by modifying the similarity threshold second influence quantity of each candidate defect feature type and the corresponding first candidate similarity threshold, and finally the candidate defect feature type is judged based on the modified similarity threshold whether it is output as a defect category, if it is output as a defect category, the comparison operation is ended, otherwise the second determination of the multi-class defect feature type is performed; through the above process, it is helpful to more accurately evaluate the reliability of the defect feature in the comparison process, and then the similarity threshold used in the subsequent comparison is modified through the result of the comparison reliability evaluation, which is beneficial to improve the robustness of the defect comparison determination.

[0061] Further, according to the output defect feature type result, the corresponding comparison determination correction measure is taken, and the specific steps are as follows:

[0062] Step one, the output defect feature type is recorded as a candidate defect feature type, if only one kind of candidate defect feature type is output, the single-class defect feature type determination is performed.

[0063] Wherein, the specific process of the single-class defect feature type determination is: based on the obtained similarity threshold first influence quantity and the similarity threshold of the corresponding defect feature type, the first modified similarity threshold is obtained by modifying operation, and then the candidate defect feature type is judged according to the first modified similarity threshold whether it is output as a defect category, if it is output as a defect category, the comparison operation is ended, otherwise the artificial review request is sent.

[0064] It needs to be explained that when the comparison reliability evaluation value is not less than the reliability determination value, it means that the comparison process has high reliability degree, so there is no need to modify the similarity threshold, that is, the similarity threshold first influence quantity will not be obtained, and the artificial review request is directly sent.

[0065] It should be noted that the correction operation represents a product operation of the first influence amount of the similarity threshold and the similarity threshold corresponding to the defect feature type.

[0066] In step two, if multiple candidate defect feature types are output, the second influence amount of the similarity threshold of each candidate defect feature type is obtained by mapping the defect feature type reliability quantization value of each candidate defect feature type in the preset database, and multi-defect feature type judgment is performed. Multi-defect feature type judgment is used to filter the defect categories that can be output from more than one candidate defect feature type to reduce the probability of manual intervention.

[0067] Specifically, a second influence of similarity threshold mapping set between the defect feature type reliability quantization value and the second influence of similarity threshold is constructed in the preset database. The real-time defect feature type reliability quantization value is input into the second influence of similarity threshold mapping set, and the corresponding second influence of similarity threshold is output. The second influence of similarity threshold mapping set represents a set of mapping relationships between the defect feature type reliability quantization value and the second influence of similarity threshold.

[0068] In this embodiment, by taking corresponding comparison and judgment correction measures for the results of different defect feature types, the accuracy of correcting the similarity threshold can be improved. When multi-defect feature type judgment is performed, not only the automation and intelligence level can be improved, but also manual intervention can be avoided as much as possible, the intelligence of the mark comparison method based on multi-source data fusion is improved, and the corresponding robustness is ensured.

[0069] Further, the specific acquisition process of the defect feature type reliability quantization value is as follows:

[0070] In the first step, the defect feature type reliability evaluation data corresponding to the candidate defect feature type is obtained and data normalization processing is performed. The defect feature type reliability evaluation data includes defect category confidence and defect comparison sample size.

[0071] It should be noted that the defect category confidence is output by the comparison model when more than one candidate defect feature type is output; and the defect comparison sample size is obtained by counting each sample used in the comparison process of the corresponding category of candidate defect feature type.

[0072] In the second step, the corresponding defect feature type evaluation proportion is obtained from the preset database. The defect feature type evaluation proportion includes defect category confidence evaluation proportion and defect comparison sample size evaluation proportion.

[0073] Specifically, the defect feature type evaluation proportion is obtained from a preset database, and the defect feature type evaluation proportion represents an influence degree of the defect feature type reliability evaluation data on the defect feature type reliability quantitative value. Each defect feature type reliability evaluation data and the defect feature type evaluation proportion have a unique mapping relationship, and the value range is between 0 and 1; for example, a mapping set of the defect feature type reliability evaluation data and the preset defect feature type evaluation proportion is constructed, the real-time defect category confidence and the defect comparison sample quantity are input into the mapping set to obtain the corresponding defect category confidence evaluation proportion and the defect comparison sample quantity evaluation proportion, which respectively represent the influence degrees of the defect category confidence and the defect comparison sample quantity on the defect feature type reliability quantitative value, and the sum of the two is 1.

[0074] Thirdly, the defect feature type reliability evaluation data and the corresponding defect feature type evaluation proportion are coupled after weighting operation to obtain a defect feature type reliability initial quantitative value.

[0075] Fourthly, a defect feature type reliability quantitative value is obtained after compensation operation of the defect feature type reliability initial quantitative value and the comparison reliability evaluation value, and the defect feature type reliability quantitative value is used to quantify the reliability degrees of each type of candidate defect feature type output; wherein the compensation operation represents the product operation of the defect feature type reliability initial quantitative value and the comparison reliability evaluation value.

[0076] The specific limit expression of the defect feature type reliability quantitative value is as follows:

[0077]

[0078] In the formula, CRA represents the comparison reliability evaluation value of the corresponding candidate defect feature type, DFC represents the defect category confidence of the corresponding candidate defect feature type, DCS represents the defect comparison sample quantity of the corresponding candidate defect feature type, DFC represents the defect category confidence evaluation proportion, DCS represents the defect comparison sample quantity evaluation proportion, and DFQ represents the defect feature type reliability quantitative value of the corresponding candidate defect feature type.

[0079] In the embodiment, the algorithm combines the defect feature type reliability evaluation data, the defect feature type evaluation proportion, and the comparison reliability evaluation value to analyze to obtain the defect feature type reliability quantization value. In the formula, as the defect category confidence, the defect comparison sample quantity, and the comparison reliability evaluation value increase, the reliability degree of the corresponding defect feature type is higher, and the defect feature type reliability quantization value is larger. Through analysis of the defect feature type reliability quantization value, the reliability degree of each type of candidate defect feature type output is quantized more accurately, so that the candidate defect feature type with the highest reliability degree is screened out, and the robustness of the defect feature data and the corresponding standard defect knowledge base in defect comparison and determination is improved.

[0080] Further, the specific steps of the multi-type defect feature type determination are as follows:

[0081] L1, the initial similarity threshold of each type of candidate defect feature type is corrected based on the similarity threshold first influence quantity to obtain the corresponding first candidate similarity threshold.

[0082] It needs to be explained that when the comparison reliability evaluation value is not less than the reliability determination value, the reliability degree of the comparison process is high, and then the initial similarity threshold of each type of candidate defect feature type does not need to be corrected based on the similarity threshold first influence quantity, and the multi-type defect feature type second determination can be directly performed.

[0083] Specifically, the first candidate similarity threshold is obtained by multiplying the similarity threshold first influence quantity and the initial similarity threshold of each type of candidate defect feature type.

[0084] L2, the similarity threshold second influence quantity of each type of candidate defect feature type is corrected based on the corresponding first candidate similarity threshold to obtain the corrected similarity threshold.

[0085] Specifically, the corrected similarity threshold is obtained by multiplying the similarity threshold second influence quantity of each type of candidate defect feature type and the first candidate similarity threshold.

[0086] L3, whether the candidate defect feature type outputs the defect category is determined based on the corrected similarity threshold. If the defect category is output, the comparison operation is ended, otherwise the multi-type defect feature type second determination is performed.

[0087] In the embodiment, the similarity threshold and the similarity threshold second influence quantity are adjusted by the similarity threshold first influence quantity obtained in the foregoing, and the determination is performed accordingly. Not only is the similarity threshold corrected in time, but also the effect and accuracy of the defect feature type recognition are improved, and the robustness of the defect type comparison is further improved.

[0088] Further, the specific process of the second determination of the multiple types of defect feature types is as follows:

[0089] In one aspect, the identified number of each type of candidate defect feature type of the same production batch is obtained, and the batch similarity threshold influence quantity of each type of candidate defect feature type is obtained by mapping the identified number in the preset database.

[0090] It should be noted that the number of each type of defect feature type of the same production batch is recorded in real time by the counter, and the identified number of each type of candidate defect feature type of the current output is obtained therefrom.

[0091] Specifically, a batch similarity mapping set between the identified number and the batch similarity threshold influence quantity is constructed in the preset database, the identified number is input into the batch similarity mapping set, and the corresponding batch similarity threshold influence quantity is output, and the batch similarity mapping set represents a set of mapping relationships between the identified number and the batch similarity threshold influence quantity.

[0092] On the other hand, the batch similarity threshold influence quantity and the corrected similarity threshold are compensated to obtain the corresponding second corrected similarity influence quantity, and it is determined whether the corresponding candidate defect feature type is output as a defect category, if it is output as a defect category, the comparison operation is ended, otherwise an artificial review request is sent.

[0093] Among them, the compensation operation represents the product operation of the batch similarity threshold influence quantity and the corrected similarity threshold.

[0094] In this embodiment, by adjusting the similarity threshold by referring to the defect feature comparison situation of the same production batch, not only the accuracy of the defect type identification comparison is improved, but also the robustness of the defect feature type identification is improved, at the same time, the automation degree of the defect feature identification is also improved, the artificial participation is reduced, and the defect detection efficiency is improved.

[0095] The embodiment of the application provides a standard knowledge comparison device based on multi-source data fusion, which comprises a comparison feature acquisition module, a defect feature comparison module and a comparison determination correction module:

[0096] The comparison feature acquisition module is used for acquiring comparison features of a product to be compared and detected by a multi-angle industrial camera to obtain a defect detection image set, and performing image quality determination on the defect detection image set to determine whether to perform defect feature comparison, and the defect feature comparison is used for defect feature identification of the product to be compared and detected.

[0097] The defect feature comparison module is configured to, if defect feature comparison is performed, perform image set feature extraction to obtain a defect feature set for comparison with a standard defect knowledge base, and perform comparison reliability evaluation to obtain a corresponding comparison reliability result, or perform re-sampling optimization on the corresponding detection product to be compared.

[0098] The comparison decision correction module is configured to, after defect feature comparison, take corresponding comparison decision correction measures according to the output defect feature type result to perform defect feature type decision, and the comparison decision correction measures are configured to correct the influence of the influence factors in the comparison process on the initial similarity threshold used in the comparison judgment.

[0099] In the embodiment, through image quality pre-filtering (comparison feature collection module) and dynamic comparison decision correction (comparison decision correction module), it is beneficial to reduce false detection and missed detection of the detection product to be compared, and improve the robustness of defect detection; and through automatic triggering of re-sampling optimization and threshold adjustment, it is beneficial to reduce the need for manual review, and improve the automation degree of the device.

[0100] In the embodiment, the computer readable storage medium is also provided for storing a program, and the program is executed by a processor to implement the standard knowledge comparison method based on multi-source data fusion.

[0101] In summary, in the embodiment, the defect detection image set obtained by comparison feature collection is subjected to image quality judgment to determine whether to perform defect feature comparison, if defect feature comparison is performed, image set feature extraction is performed to obtain a defect feature set for comparison with a standard defect knowledge base, and comparison reliability evaluation is performed to obtain a corresponding comparison reliability result, or the corresponding detection product to be compared is subjected to re-sampling optimization, and finally, after defect feature comparison, corresponding comparison decision correction measures are taken according to the output defect feature type result to perform defect feature type decision, so as to more accurately output the corresponding defect type, and thus improve the robustness of defect comparison decision, and effectively solve the problem of low comparison decision robustness caused by limited comparison information in the prior art.

[0102] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems, or computer program products. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) having computer-usable program code embodied in the medium.

[0103] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0104] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0106] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those of skill in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, the attached claims are intended to embrace all such variations and modifications as fall within the scope of the application.

[0107] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A standard knowledge comparison method based on multi-source data fusion, characterized in that, Includes the following steps: A set of defect detection images is obtained by collecting comparison features of the product to be inspected using a multi-angle industrial camera. The image quality of the defect detection image set is judged to determine whether defect feature comparison should be performed. The defect feature comparison is used to identify defect features of the product to be inspected. If defect feature comparison is performed, the defect feature set is obtained by extracting features from the image set and comparing it with the standard defect knowledge base. The comparison reliability assessment is then performed to obtain the corresponding comparison reliability result. Otherwise, the corresponding product to be compared and tested is resampled and optimized. The comparison reliability assessment is used to evaluate the reliability of the defect feature comparison process. After the defect feature comparison, corresponding comparison judgment correction measures are taken according to the output defect feature type to determine the defect feature type. The comparison judgment correction measures are used to correct the influence of the comparison of influencing factors in the comparison process on the initial similarity threshold used for judgment.

2. The standard knowledge comparison method based on multi-source data fusion as described in claim 1, characterized in that: The specific steps for determining the image quality of the defect detection image set are as follows: S1, obtain image quality judgment data for each defect detection image in the defect detection image set, wherein the image quality judgment data includes image sharpness and image contrast; S2, If all image quality judgment data are not less than the corresponding image quality reference data, then perform defect feature comparison; If the image quality judgment data is less than the corresponding image quality reference data, then no defect feature comparison will be performed and the corresponding product to be compared will be transferred to the re-inspection conveyor belt, and S3 will be executed simultaneously. S3, the difference between the image quality judgment data and the corresponding image quality reference data is quantified to obtain the image quality difference amount. Based on the image quality difference amount, the corresponding camera shooting angle adjustment amount is mapped in the preset database. The camera shooting angle adjustment amount is used to adjust the shooting angle of the camera when the corresponding comparison product to be detected is collected for the next comparison feature.

3. The standard knowledge comparison method based on multi-source data fusion as described in claim 1, characterized in that: The process of performing a comparative reliability assessment to obtain the corresponding comparative reliability results is as follows: A comparison reliability assessment is performed during the comparison of the defect feature set with the standard defect knowledge base to obtain the corresponding comparison reliability assessment value. If the comparison reliability assessment value is not less than the reliability judgment value obtained from the preset database, the corresponding defect feature type will be output. If the reliability assessment value is less than the reliability judgment value obtained from the preset database, the corresponding defect feature type is output, and the reliability assessment value and the reliability judgment value are compared and the difference is quantified to obtain the corresponding reliability difference quantity. Based on the reliability difference quantity, the first influence quantity of the similarity threshold is obtained by mapping it in the preset database. The first influence quantity of the similarity threshold represents the quantified value of the influence of the influencing factors in the comparison process on the similarity threshold.

4. The standard knowledge comparison method based on multi-source data fusion as described in claim 3, characterized in that: The specific method for obtaining the comparison reliability assessment value is as follows: Acquire comparison reliability data and perform data normalization processing. The comparison reliability data includes the comparison sample size, comparison time, and comparison coverage. The comparison coverage represents the ratio of the number of samples used to complete the identification of all defect features to the total number of samples of the corresponding defect feature type when identifying the corresponding defect feature type. The comparison reliability analysis weight and reference comparison time range are obtained from the preset database. The comparison reliability analysis weight includes the comparison sample size analysis weight, the comparison time analysis weight, and the comparison coverage analysis weight. The reference comparison time range represents the range between the minimum comparison time and the maximum comparison time. The range deviation between the comparison time and the reference comparison time range is quantified to obtain the corresponding comparison time deviation. The comparison reliability assessment value is obtained by weighting and coupling the comparison time deviation, comparison sample size, comparison coverage and corresponding comparison reliability analysis weight. The comparison reliability assessment value is used to quantify the reliability of the defect feature comparison process.

5. The standard knowledge comparison method based on multi-source data fusion as described in claim 1, characterized in that: The specific steps for taking corresponding comparison and correction measures based on the output defect feature type are as follows: Step 1: Record the output defect feature type as the candidate defect feature type. If only one candidate defect feature type is output, then perform single-type defect feature type determination. Step 2: If multiple candidate defect feature types are output, the reliable quantification value of each candidate defect feature type is obtained and mapped to the preset database to obtain the second influence value of the similarity threshold of each candidate defect feature type. Then, multiple defect feature type determination is performed. The multiple defect feature type determination is used to filter out the defect category that can be output from more than one candidate defect feature type to reduce the probability of manual intervention. The specific process for determining the single-type defect feature type is as follows: The first corrected similarity threshold is obtained by performing a correction operation based on the first influence quantity of the obtained similarity threshold and the similarity threshold of the corresponding defect feature type. The first corrected similarity threshold is then used to determine whether the candidate defect feature type is output as a defect category. If the output is a defect category, the comparison operation ends; otherwise, a manual re-inspection request is sent.

6. The standard knowledge comparison method based on multi-source data fusion as described in claim 5, characterized in that: The specific process for obtaining the reliable metric value of the defect feature type is as follows: Obtain the reliability assessment data of the corresponding candidate defect feature types and perform data normalization processing. The reliability assessment data of the defect feature types includes the confidence level of the defect category and the sample size of the defect comparison. The corresponding defect feature type evaluation weight is obtained from the preset database. The defect feature type evaluation weight includes the defect category confidence evaluation weight and the defect comparison sample size evaluation weight. The initial quantitative value of the reliability of the defect feature type is obtained by coupling the reliability assessment data of the defect feature type with the corresponding defect feature type assessment weight after weighting operation; After performing a compensation operation on the initial quantified value of the reliability of the defect feature type and the comparison reliability evaluation value, a quantified value of the reliability of the defect feature type is obtained. The quantified value of the reliability of the defect feature type is used to quantify the reliability of various candidate defect feature types output.

7. The standard knowledge comparison method based on multi-source data fusion as described in claim 5, characterized in that: The specific steps for determining multiple defect feature types are as follows: L1, based on the first influence of the similarity threshold, corrects the initial similarity threshold of each candidate defect feature type to obtain the corresponding first candidate similarity threshold; L2, by modifying the second influence quantity of the similarity threshold of various candidate defect feature types with the corresponding first candidate similarity threshold, the modified similarity threshold is obtained; L3 determines whether to output a defect category based on the modified similarity threshold for the candidate defect feature type. If a defect category is output, the comparison operation ends; otherwise, a second determination of multiple defect feature types is performed.

8. The standard knowledge comparison method based on multi-source data fusion as described in claim 7, characterized in that: The specific process for performing the second determination of multiple defect feature types is as follows: The number of each candidate defect feature type identified in the same production batch is obtained, and the batch similarity threshold influence of each candidate defect feature type is obtained by mapping the number of identified defects in a preset database. The batch similarity threshold influence is compensated with the corrected similarity threshold to obtain the corresponding second corrected similarity influence. It is then determined whether the corresponding candidate defect feature type is output as a defect category. If the output is a defect category, the comparison operation ends; otherwise, a manual re-inspection request is sent.

9. A standard knowledge comparison device based on multi-source data fusion, characterized in that, It includes a comparison feature acquisition module, a defect feature comparison module, and a comparison judgment correction module: The comparison feature acquisition module is used to acquire the comparison features of the product to be compared and inspected through a multi-angle industrial camera to obtain a set of defect detection images. The defect detection image set is then used to determine the image quality to decide whether to perform defect feature comparison. The defect feature comparison is used to identify the defect features of the product to be compared and inspected. The defect feature comparison module is used to extract image features to obtain a defect feature set if defect feature comparison is to be performed, compare it with the standard defect knowledge base, and perform a comparison reliability assessment to obtain the corresponding comparison reliability result. Otherwise, the corresponding product to be compared and tested will be resampled and optimized. The comparison reliability assessment is used to evaluate the reliability of the defect feature comparison process. The comparison and judgment correction module is used to take corresponding comparison and judgment correction measures to determine the defect feature type based on the output defect feature type after the defect feature comparison. The comparison and judgment correction measures are used to correct the influence of the comparison of influencing factors in the comparison process on the initial similarity threshold used for judgment.

10. A computer-readable storage medium for storing a program that, when executed by a processor, implements the standard knowledge comparison method based on multi-source data fusion as described in any one of claims 1 to 8.

Images