Product yield detection method and device, terminal and storage medium
Through dynamic template matching and deep feature extraction, combined with multimodal data acquisition and difference analysis, the problems of low efficiency, insufficient accuracy and poor template adaptability of existing product detection methods are solved, and accurate detection and efficient identification of different products are achieved.
Patent Information
- Application Number
- CN202411994647.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-05-02
AI Technical Summary
The existing product inspection methods are inefficient, insufficient accuracy, and poor template adaptability, making it difficult to adapt to product changes and rapid testing needs.
Dynamic template matching technology is adopted, combined with deep feature extraction and matching, and through multimodal data acquisition and difference analysis, detection parameters and templates are dynamically adjusted to achieve accurate detection and efficient identification of different products.
It improves the flexibility and adaptability of detection, enhances the recognition ability of different products, and significantly improves the detection efficiency and accuracy.
Smart Images
Figure CN119917871A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of industrial detection technology, and in particular to a product yield detection method, device, terminal and storage medium. Background Art
[0002] With the rapid development of the manufacturing industry and the increasing variety of products, product inspection is facing unprecedented challenges. Traditional product inspection mainly relies on static templates for comparison. Although the static template matching method meets the basic inspection needs to a certain extent, it has many limitations. For example, when dealing with products with high diversity and complexity, the inspection efficiency and accuracy often cannot meet the requirements; or, every time the product is updated, a new inspection template needs to be remade, which not only increases the workload, but also the template production process is cumbersome and time-consuming, which can easily become a bottleneck in the production process. At the same time, most of the existing machine vision inspection systems rely on simple image processing algorithms, which are difficult to adapt to the rapidly changing product inspection needs, resulting in the accuracy of the inspection results being affected.
[0003] Therefore, it is particularly important to develop a new product detection method that can efficiently extract product features and perform accurate matching.
[0004] Therefore, it is particularly important to develop an efficient and accurate detection method that can adapt to different products. The method should be able to automatically adjust the detection parameters and templates to adapt to product changes; at the same time, it should have a high level of intelligence and automation to improve detection efficiency and accuracy. Summary of the invention
[0005] In order to solve the problems of low product detection efficiency, insufficient accuracy and poor template adaptability in the prior art, the present application provides a product yield detection method, device, terminal and storage medium.
[0006] This method achieves accurate detection and efficient identification of different products through dynamic template adjustment, deep feature extraction and matching, and intelligent decision optimization.
[0007] In the first aspect, the present application provides a product yield detection method, which adopts the following technical solution: A product yield detection method comprises the following steps: Acquire a product sample of the product to be detected, generate an initial matching template based on the product sample, collect multimodal data of the product to be detected, perform feature extraction on the multimodal data, and obtain a deep feature vector of the product to be detected; Performing an initial matching of the depth feature vector of the product to be inspected with the initial matching template and calculating a first matching degree, and obtaining a first matching result according to the first matching degree, wherein the first matching result includes a qualified result and a unqualified result; When the first matching result is unqualified, performing a difference analysis on the product to be tested and the initial matching template to obtain a difference analysis result, dynamically adjusting the parameters of the initial matching template according to the difference analysis result to obtain a final matching template; A second matching is performed on the product to be tested and the final matching template to obtain a second matching degree, and a product yield test result of the product to be tested is obtained according to the second matching degree.
[0008] By adopting the above technical means, through dynamic template matching technology, the template can be adjusted in real time to adapt to slight changes in products or the introduction of new varieties, without frequent manual template updates, thereby improving the flexibility and adaptability of detection. The application of deep learning algorithms enables the system to automatically learn and extract the deep features of products, further enhancing the system's ability to identify different products.
[0009] Preferably, the step of obtaining a product sample of the product to be tested and generating an initial matching template according to the product sample specifically includes the following steps: Obtaining a preset product sample of the product to be detected, wherein the product sample is a sample of the product to be detected collected in advance, and the types of the product sample include image samples, sound samples and infrared samples; Determine a feature extraction algorithm based on the type of the product sample, extract features from the corresponding product sample based on the determined feature extraction algorithm to obtain sample features, encode each sample feature and aggregate the encoded sample features to obtain a sample feature set, train a preset initial model based on the sample feature set to obtain an initial matching template.
[0010] By adopting the above-mentioned technical means and processing various types of samples, accurate and reliable initial templates can be generated, which can provide a more comprehensive understanding of the characteristics of the product and lay a solid foundation for subsequent feature extraction and matching.
[0011] Preferably, the collecting of multimodal data of the product to be detected, performing feature extraction on the multimodal data, and obtaining a deep feature vector of the product to be detected specifically comprises the following steps: Collect multimodal data of the product to be inspected in real time, the multimodal data is obtained based on a variety of sensors and data acquisition equipment deployed on the production line, and the data types of the multimodal data include image data, sound data and infrared data; The collected multimodal data is preprocessed and the data type of the multimodal data is determined, a feature extraction strategy is determined according to the data type, and features of the corresponding type of the multimodal data are extracted according to the feature extraction strategy to obtain a depth feature vector of the product to be inspected.
[0012] By adopting the above-mentioned technical means, multimodal data of the product to be tested can be collected from multiple aspects, and matching detection can be performed on the product to be tested from various aspects such as appearance, sound and infrared. Data of different modes can verify and supplement each other, thereby improving the accuracy and reliability of product quality detection, and also improving the applicability of this method.
[0013] Preferably, the performing of initial matching of the depth feature vector of the product to be detected with the initial matching template and calculating a first matching degree, and obtaining a first matching result according to the first matching degree, specifically comprises the following steps: Determine the type corresponding to the deep feature vector, analyze the matching requirements of the current product to be detected, determine a matching algorithm according to the data type and the matching requirements, and calculate a first matching degree and a first matching result between the deep feature vector and the initial matching template according to the matching algorithm; The first matching degree is compared with a preset matching threshold. When the first matching degree is higher than the matching threshold, the first matching result is qualified. When the first matching degree is not higher than the matching threshold, the first matching result is unqualified.
[0014] By adopting the above-mentioned technical means, through the initial matching between the features and the initial matching template, the product to be tested can be initially tested. The matching degree of the product can be known through the results of the preliminary test, which lays the foundation for further difference analysis and secondary matching, and is conducive to saving computing resources and time.
[0015] Preferably, performing difference analysis on the product to be detected and the initial matching template and obtaining a difference analysis result comprises the following steps: The difference data between the product to be detected and the initial matching template during the matching process are recorded, the difference data are quantified, and a difference analysis result is obtained. The difference analysis result is classified according to the data type, and the classification result includes at least one of image feature difference, sound feature difference and infrared feature difference. The image feature difference includes shape difference, texture difference and size difference, the sound feature difference includes frequency difference and amplitude difference, and the infrared feature difference includes absorption peak position difference and absorption peak intensity difference.
[0016] By adopting the above-mentioned technical means, the mismatches between data can be accurately identified through difference analysis, whether it is image, sound or infrared data, which helps to quickly locate the problem, thereby avoiding blind adjustment of the template, and helps the system optimize resource allocation, investing more computing resources and time in the features or parameters that have the greatest impact on the matching results, thereby improving the matching efficiency.
[0017] Preferably, dynamically adjusting the parameters of the initial matching template according to the difference analysis result to obtain the final matching template specifically includes the following steps: When the image feature difference exists in the difference analysis result, determining a first difference category which is the largest among the shape difference, the texture difference and the size difference, and determining an image parameter adjustment mechanism in a preset parameter adjustment strategy according to the first difference category; When the sound feature difference exists in the difference analysis result, determining a second difference category which is larger between the frequency difference and the amplitude difference, and determining a sound parameter adjustment mechanism in the parameter adjustment strategy according to the second difference category; When the infrared characteristic difference exists in the difference analysis result, determining a third difference category which is larger between the absorption peak position difference and the absorption peak intensity difference, and determining an infrared parameter adjustment mechanism in the parameter adjustment strategy according to the third difference category; The initial matching template is adjusted according to the determined parameter adjustment strategy to obtain a final matching template.
[0018] By adopting the above technical means, the parameters that need to be adjusted can be quickly found through difference analysis, avoiding the "one-size-fits-all" adjustment method. Targeted adjustments can more effectively solve matching problems and save a lot of time and computing resources.
[0019] Preferably, performing secondary matching on the product to be detected and the final matching template to obtain a second matching degree, and obtaining a product yield test result of the product to be detected according to the second matching degree, specifically comprises the following steps: Determining a matching algorithm according to the data type and the matching requirement, and calculating a second matching degree and a second matching result between the depth feature vector and the final matching template according to the matching algorithm; The second matching degree is compared with a preset matching threshold; when the second matching degree is higher than the matching threshold, the second matching result is qualified; when the second matching degree is not higher than the matching threshold, the second matching result is unqualified; and a product yield test result of the product to be tested is generated based on the second matching result and the second matching degree.
[0020] By adopting the above technical means, matching is performed again after performing difference analysis and adjusting the parameters of the initial template, which improves the adaptability and flexibility of the method and can significantly improve the accuracy of the matching process.
[0021] In the second aspect, the present application provides a product yield detection device, which adopts the following technical solution: A product yield detection device includes the following modules: A feature extraction module is used to obtain a product sample of the product to be detected, generate an initial matching template based on the product sample, collect multimodal data of the product to be detected, perform feature extraction on the multimodal data, and obtain a deep feature vector of the product to be detected; A primary matching module, used for performing a primary matching between the depth feature vector of the product to be detected and the initial matching template and calculating a first matching degree, and obtaining a first matching result according to the first matching degree, wherein the first matching result includes a qualified result and a unqualified result; A parameter adjustment module, configured to perform a difference analysis on the product to be detected and the initial matching template and obtain a difference analysis result when the first matching result is unqualified, and dynamically adjust the parameters of the initial matching template according to the difference analysis result to obtain a final matching template; The detection result generating module is used to perform secondary matching on the product to be detected and the final matching template to obtain a second matching degree, and obtain a product yield detection result of the product to be detected according to the second matching degree.
[0022] By adopting the above-mentioned technical means, a system has been built to detect product yield based on the multimodal data of the products to be tested, which provides the necessary software technical support for the quality assurance of products on the production line. It has the advantages of high matching accuracy and reliability, strong adaptability and wide application scenarios, optimized computing resource utilization, improved processing efficiency and promotion of technological innovation.
[0023] In a third aspect, the present application provides a smart terminal, which adopts the following technical solution: An intelligent terminal includes a memory and a processor, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the product yield detection method as described above.
[0024] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium, wherein at least one instruction, at least one program, code set or instruction set is stored in the computer-readable storage medium, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the product yield detection method as described above.
[0025] In summary, this application has at least the following beneficial effects: 1. This application uses dynamic template matching technology to adjust the template in real time to adapt to minor changes in products or the introduction of new varieties, without the need to frequently update the template manually, thereby improving the flexibility and adaptability of detection. The application of deep learning algorithms enables the system to automatically learn and extract the deep features of products, further enhancing the system's ability to identify different products.
[0026] 2. The multimodal data of this application provides various types of information such as images, sounds, infrared, etc., capturing the characteristics of the product in different dimensions. These data complement each other and provide a more comprehensive product description, thereby enhancing the accuracy and comprehensiveness of product matching and detection.
[0027] 3. This application captures subtle differences in products through deep feature extraction technology, improves matching accuracy, and sets up difference analysis based on the matching results to accurately judge and optimize the differences, further ensuring high accuracy of product testing. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 This is one of the flow charts of a product yield detection method according to an embodiment of the present application; Figure 2 This is the second flow chart of a product yield detection method according to an embodiment of the present application; Figure 3 It is an architectural diagram of a product yield detection device according to an embodiment of the present application. DETAILED DESCRIPTION
[0029] The present application provides a product yield detection method, device, terminal and storage medium. In order to make the purpose, technical solution and advantages of the present application clearer, the implementation method of the present application will be further described in detail below.
[0030] The following is a further detailed description of an embodiment of a product yield detection method of the present application in conjunction with the accompanying drawings.
[0031] A product yield detection method of the present application, such as Figure 1 As shown, the following steps are included: S1. Obtain product samples of the product to be detected, generate an initial matching template based on the product samples, collect multimodal data of the product to be detected, extract features from the multimodal data, and obtain a deep feature vector of the product to be detected. Specifically, the steps include: Figure 2 shown.
[0032] S11. Obtain preset product samples of the products to be tested. Product samples are samples of the products to be tested that are collected in advance. These samples should cover various possible forms and characteristics of the products to ensure the universality and accuracy of the template.
[0033] The types of product samples include image samples, sound samples and infrared samples. The characteristics of a product are not limited to its visual appearance. Therefore, in addition to image samples, this embodiment also provides sound samples and infrared samples.
[0034] Sound sample data provides important information about the internal mechanical condition or material quality of the product.
[0035] Likewise, infrared sample data reveals the thermal distribution of the product, helping to detect potential faults or anomalies.
[0036] S12. Determine a feature extraction algorithm based on the type of product sample. In this embodiment, the feature extraction algorithm includes a convolutional neural network (CNN) and a recurrent neural network (RNN).
[0037] For product samples whose types are image samples and infrared samples, a convolutional neural network is used for feature extraction; for product samples whose type is sound samples, a recurrent neural network is used for feature extraction.
[0038] According to the feature extraction algorithm, feature extraction is performed on the corresponding product samples to obtain sample features, each sample feature is encoded and the encoded sample features are summarized to obtain a sample feature set.
[0039] Each feature in the feature set is encoded for storage and matching in the template. The features in this embodiment are categorical features and are encoded using one-hot encoding. One-hot encoding can avoid the model's bias towards categorical variables by converting each type into an independent binary vector.
[0040] Each different type of product sample has its own set of sample characteristics.
[0041] S13. Train the preset initial model according to the sample feature set to obtain an initial matching template.
[0042] In a specific implementable method, different types of sample feature sets are fused to obtain a comprehensive sample feature set. By training the initial model with the comprehensive sample feature set, a more comprehensive and accurate feature representation can be extracted, thereby improving the accuracy and reliability of subsequent matching.
[0043] In this embodiment, a support vector machine (SVM) is used as a preset initial model. SVM can separate data of different categories by finding a hyperplane, thereby maximizing the interval between different categories and is suitable for processing unbalanced data sets.
[0044] The SVM model is trained using a comprehensive sample feature set to obtain an initial matching template, which includes product feature information and matching rules. These rules and feature information are learned based on the comprehensive sample feature set.
[0045] The initial matching template is a mathematical model that includes a support vector and a weight vector. The weight vector represents the importance of different sample features to the model decision.
[0046] S14. Collect multimodal data of the product to be inspected in real time. Various sensors and data acquisition devices are arranged on the production line. Multimodal data is obtained from the sensors and data acquisition devices. The data types of multimodal data include image data, sound data and infrared data.
[0047] S15. Preprocess the collected multimodal data and determine the data type of the multimodal data, and determine a feature extraction strategy based on the data type.
[0048] The feature extraction strategy is that when the data type is image data, the image features in the image data are extracted according to the preset image processing algorithm; when the data type is sound data, the sound features in the sound data are extracted according to the signal processing algorithm; when the data type is infrared data, the infrared features in the infrared data are extracted according to the infrared image processing algorithm.
[0049] S16: Extract features from the corresponding type of multimodal data according to the feature extraction strategy to obtain a depth feature vector of the product to be detected. The depth feature vector in this embodiment includes at least one of image features, sound features, and infrared features.
[0050] S2. Perform an initial match between the depth feature vector of the product to be detected and the initial matching template and calculate a first matching degree, and obtain a first matching result based on the first matching degree. The first matching result includes qualified and unqualified results, and specifically includes the following steps.
[0051] S21. Determine the type corresponding to the deep feature vector, analyze the matching requirements of the current product to be detected, and determine the matching algorithm based on the data type and matching requirements.
[0052] In this embodiment, a feature-based template matching algorithm is used for image features, such as feature point matching and histogram comparison, wherein the feature point matching algorithms include SIFT, SURF and ORB.
[0053] For sound features, the dynamic time warping (DTW) algorithm is used to handle the matching problem between sound signals of different lengths.
[0054] For infrared features, it is necessary to combine image matching and signal processing methods for comprehensive evaluation.
[0055] The matching requirements in this implementation include matching accuracy and matching speed. When the matching requirement requires higher matching accuracy, a feature-based template matching algorithm is selected; when the matching requirement requires faster matching speed, the fast nearest neighbor search library (FLANN) is selected for feature point matching.
[0056] In order to better explain the above process, a simple embodiment that only requires image matching is described below: In a specific feasible implementation method, when the data type only contains image data, there are only image features in the depth feature vector. It is assumed that the matching requirement is to complete the matching task in a shorter time, that is, the matching speed is required to be faster. Therefore, based on the requirements of image features and matching speed, the matching algorithm is determined to be the SIFT algorithm combined with FLANN for feature point matching.
[0057] S22. Calculate a first matching degree and a first matching result between the depth feature vector and the initial matching template according to a matching algorithm.
[0058] As described in the above embodiment, when performing image matching, the similarity between each feature point in the image of the product to be detected and the feature point in the initial matching template is calculated by a matching algorithm, which is represented by the Euclidean distance in this embodiment.
[0059] According to the set similarity threshold, matching pairs with higher similarity are screened out, the number of the screened matching pairs is counted, and the first matching degree is calculated according to the number of matching pairs.
[0060] The first matching degree is the ratio of the number of matching pairs to the total number of feature points in the image to be detected.
[0061] S23. Compare the first matching degree with a preset matching threshold. When the first matching degree is higher than the matching threshold, the first matching result is qualified. If the first matching result is qualified, directly generate a qualified product yield test result of the product to be tested, and the product yield test result includes the value of the first matching degree and the first matching result.
[0062] When the first matching degree is not higher than the matching threshold, the first matching result is unqualified.
[0063] In a specific implementation method, assuming that the similarity value ranges from 0 to 1, 1 represents a complete match, if the total number of feature points in the image of the product to be detected is 1000, the number of matching pairs after screening is 500; the average similarity is 0.8, so the calculated matching degree is 50%.
[0064] The matching degree threshold set in this embodiment is 60%, so the matching fails and the first matching result is unqualified.
[0065] S3. When the first matching result is unqualified, a difference analysis is performed on the product to be tested and the initial matching template to obtain a difference analysis result, and the parameters of the initial matching template are dynamically adjusted according to the difference analysis result to obtain a final matching template, which specifically includes the following steps.
[0066] S31. Record the difference data between the product to be tested and the initial matching template during the matching process, quantify the difference data, and obtain a difference analysis result.
[0067] The quantitative processing method of the difference data includes using a similarity index or a distance index. In this embodiment, the similarity index is used, and the differences between the image, sound and infrared data are quantified by calculating the cosine similarity.
[0068] S32. Classify the difference analysis results according to data types.
[0069] The classification result includes at least one of image feature difference, sound feature difference and infrared feature difference.
[0070] The image feature differences include shape differences, texture differences and size differences, the sound feature differences include frequency differences and amplitude differences, and the infrared feature differences include absorption peak position differences and absorption peak intensity differences.
[0071] S33. When there are image feature differences in the difference analysis results, determine a first difference category that is the largest among shape difference, texture difference, and size difference, and determine an image parameter adjustment mechanism in a preset parameter adjustment strategy according to the first difference category.
[0072] In this embodiment, the preset parameter adjustment strategy includes adjusting the weight of each feature in the initial matching template and adjusting the size ratio parameter of the initial matching template to introduce new features.
[0073] For features that are significantly different from the product to be tested, appropriately increasing their weights can improve matching accuracy.
[0074] When there is a large difference in size or proportion between the product to be inspected and the template, the size or proportion parameters of the template can be adjusted, but this method is only applicable to image feature differences.
[0075] When the matching result is continuously unsatisfactory, the adjustment method of this embodiment is to introduce new features.
[0076] When the shape difference is the most significant among the above three, that is, when the first difference category is the shape difference, the image parameter adjustment mechanism determined from the parameter adjustment strategy is the weight of the shape feature; When the texture difference is the most significant among the above three, that is, when the first difference category is texture difference, the image parameter adjustment mechanism determined from the parameter adjustment strategy is to introduce new texture features; When the size difference is the most significant among the above three, that is, when the first difference category is the size difference, the image parameter adjustment mechanism determined from the parameter adjustment strategy is to adjust the size ratio parameter of the template.
[0077] S34. When there are sound feature differences in the difference analysis results, determine a second difference category that is larger between the frequency difference and the amplitude difference, and determine a sound parameter adjustment mechanism in the parameter adjustment strategy based on the second difference category.
[0078] When the frequency difference is the most significant of the above two, that is, when the second difference category is the frequency difference, the sound parameter adjustment mechanism determined from the parameter adjustment strategy is to adjust the weight of the frequency feature.
[0079] When the amplitude difference is the most significant of the above two, that is, when the second difference category is the amplitude difference, the sound parameter adjustment mechanism determined from the parameter adjustment strategy is to adjust the weight of the amplitude feature.
[0080] S35. When there are infrared characteristic differences in the difference analysis results, determine the third difference category which is larger between the absorption peak position difference and the absorption peak intensity difference, and determine the infrared parameter adjustment mechanism in the parameter adjustment strategy according to the third difference category.
[0081] When the difference in the absorption peak position is the most significant of the above two, that is, when the second difference category is the difference in the absorption peak position, the infrared parameter adjustment mechanism determined from the parameter adjustment strategy is to adjust the parameters of the wavelength range in the infrared characteristics.
[0082] When the difference in absorption peak intensity is the most significant among the above two, that is, when the second difference category is the difference in absorption peak intensity, the infrared parameter adjustment mechanism determined from the parameter adjustment strategy is to adjust the weights related to the infrared spectrum intensity in the infrared characteristics.
[0083] S36. According to the determined parameter adjustment strategy, that is, according to at least one of the image parameter adjustment mechanism, the sound parameter adjustment mechanism, and the infrared parameter adjustment mechanism, the initial matching template is adjusted to obtain a final matching template.
[0084] S4, performing secondary matching on the product to be tested and the final matching template to obtain a second matching degree, and obtaining a product yield test result of the product to be tested according to the second matching degree, specifically including the following steps.
[0085] S41, determining a matching algorithm according to the data type and the matching requirement, calculating a second matching degree and a second matching result between the depth feature vector and the final matching template according to the matching algorithm, and comparing the second matching degree with a preset matching threshold; S42: When the second matching degree is higher than the matching threshold, the second matching result is qualified; when the second matching degree is not higher than the matching threshold, the second matching result is unqualified.
[0086] The above process is the same as step S2 in principle and will not be repeated here.
[0087] S43: Generate a product yield test result of the product to be tested according to the second matching result and the second matching degree.
[0088] The content of the product yield test result includes the first matching degree, the second matching result and the second matching degree.
[0089] Based on the same inventive concept as above, the present application also discloses a product yield detection device, the structure of which is as follows: Figure 3 As shown, the device includes the following modules: The feature extraction module is used to obtain product samples of the product to be detected, generate an initial matching template based on the product samples, collect multimodal data of the product to be detected, extract features from the multimodal data, and obtain a deep feature vector of the product to be detected; A primary matching module, used for performing a primary matching between the depth feature vector of the product to be detected and the initial matching template and calculating a first matching degree, and obtaining a first matching result according to the first matching degree, wherein the first matching result includes a qualified result and a unqualified result; A parameter adjustment module, for performing a difference analysis between the product to be tested and the initial matching template and obtaining a difference analysis result when the first matching result is unqualified, and dynamically adjusting the parameters of the initial matching template according to the difference analysis result and obtaining a final matching template; The detection result generating module is used to perform secondary matching on the product to be detected and the final matching template and obtain a second matching degree, and obtain a product yield detection result of the product to be detected based on the second matching degree.
[0090] In a specific implementation scheme, the feature extraction module includes the following units: A first feature extraction unit is used to obtain a preset product sample of the product to be detected, where the product sample is a sample of the product to be detected that is collected in advance, and the types of the product sample include image samples, sound samples, and infrared samples; The second feature extraction unit is used to determine the feature extraction algorithm according to the type of product sample, extract features from the corresponding product sample according to the determined feature extraction algorithm to obtain sample features, encode each sample feature and summarize the encoded sample features to obtain a sample feature set, and train a preset initial model according to the sample feature set to obtain an initial matching template.
[0091] A third feature extraction unit is used to collect multimodal data of the product to be inspected in real time. The multimodal data is obtained based on a variety of sensors and data acquisition devices deployed on the production line. The data types of the multimodal data include image data, sound data, and infrared data; The fourth feature extraction unit is used to preprocess the collected multimodal data and determine the data type of the multimodal data, determine the feature extraction strategy according to the data type, and extract features from the corresponding type of multimodal data according to the feature extraction strategy to obtain a deep feature vector of the product to be inspected.
[0092] In a specific implementation scheme, the initial matching module includes the following units: A first initial matching unit is used to determine the type corresponding to the deep feature vector, analyze the matching requirements of the current product to be detected, determine the matching algorithm according to the data type and the matching requirements, and calculate the first matching degree and the first matching result between the deep feature vector and the initial matching template according to the matching algorithm; The second initial matching unit is used to compare the first matching degree with a preset matching threshold. When the first matching degree is higher than the matching threshold, the first matching result is qualified; when the first matching degree is not higher than the matching threshold, the first matching result is unqualified.
[0093] In a specific implementation scheme, the parameter adjustment module includes the following units: The first parameter adjustment unit is used to record the difference data between the product to be detected and the initial matching template during the matching process, quantify the difference data, and obtain the difference analysis result, and classify the difference analysis result according to the data type. The classification result includes at least one of the image feature difference, the sound feature difference and the infrared feature difference, wherein the image feature difference includes shape difference, texture difference and size difference, the sound feature difference includes frequency difference and amplitude difference, and the infrared feature difference includes absorption peak position difference and absorption peak intensity difference.
[0094] A second parameter adjustment unit is used to determine a first difference category that is the largest among shape difference, texture difference and size difference when there is an image feature difference in the difference analysis result, and determine an image parameter adjustment mechanism in a preset parameter adjustment strategy according to the first difference category; A third parameter adjustment unit is used to determine a second difference category that is larger between the frequency difference and the amplitude difference when there is a sound feature difference in the difference analysis result, and determine a sound parameter adjustment mechanism in the parameter adjustment strategy according to the second difference category; A fourth parameter adjustment unit is used to determine a third difference category, which is larger between the absorption peak position difference and the absorption peak intensity difference, when there is an infrared characteristic difference in the difference analysis result, and determine an infrared parameter adjustment mechanism in the parameter adjustment strategy according to the third difference category; The fifth parameter adjustment unit is used to adjust the initial matching template according to the determined parameter adjustment strategy to obtain a final matching template.
[0095] In a specific implementation scheme, the detection result generation module includes the following units: A first detection result generating unit, used to determine a matching algorithm according to a data type and a matching requirement, and calculate a second matching degree and a second matching result between a deep feature vector and a final matching template according to the matching algorithm; A second detection result generating unit, used for comparing the second matching degree with a preset matching threshold value, when the second matching degree is higher than the matching threshold value, the second matching result is qualified, when the second matching degree is not higher than the matching threshold value, the second matching result is unqualified; The third detection result generating unit is used to generate a product yield detection result of the product to be detected according to the second matching result and the second matching degree.
[0096] From the above functional introduction, it can be seen that a product yield detection device in this application has built a product detection system based on multimodal data and capable of secondary matching, which provides the necessary software technical support for efficient and high-precision matching and detection of products, provides guarantees for the quality of products on the production line, and promotes the sustained and rapid development of the economy and society.
[0097] Based on the same inventive concept mentioned above, an embodiment of the present application also discloses a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set, and at least one instruction, at least one program, code set or instruction set can be loaded and executed by a processor to implement the product yield detection method provided in the above method embodiment.
[0098] Also based on the same inventive concept mentioned above, an embodiment of the present application also discloses a computer-readable storage medium, in which at least one instruction, at least one program, code set or instruction set is stored, and at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the product yield detection method as mentioned above.
[0099] A person skilled in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by a program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium, and the computer-readable storage medium includes, for example: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0100] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A product yield detection method, characterized in that: The steps include: Acquire a product sample of the product to be detected, generate an initial matching template based on the product sample, collect multimodal data of the product to be detected, perform feature extraction on the multimodal data, and obtain a deep feature vector of the product to be detected; Performing an initial matching of the depth feature vector of the product to be inspected with the initial matching template and calculating a first matching degree, and obtaining a first matching result according to the first matching degree, wherein the first matching result includes a qualified result and a unqualified result; When the first matching result is unqualified, performing a difference analysis on the product to be tested and the initial matching template to obtain a difference analysis result, dynamically adjusting the parameters of the initial matching template according to the difference analysis result to obtain a final matching template; A second matching is performed on the product to be tested and the final matching template to obtain a second matching degree, and a product yield test result of the product to be tested is obtained according to the second matching degree.
2. The product yield detection method according to claim 1, characterized in that: The step of obtaining a product sample of the product to be tested and generating an initial matching template according to the product sample specifically includes the following steps: Obtaining a preset product sample of the product to be detected, wherein the product sample is a sample of the product to be detected collected in advance, and the types of the product sample include image samples, sound samples and infrared samples; Determine a feature extraction algorithm based on the type of the product sample, extract features from the corresponding product sample based on the determined feature extraction algorithm to obtain sample features, encode each sample feature and aggregate the encoded sample features to obtain a sample feature set, train a preset initial model based on the sample feature set to obtain an initial matching template.
3. The product yield detection method according to claim 2, characterized in that: The collecting of multimodal data of the product to be detected, performing feature extraction on the multimodal data, and obtaining a deep feature vector of the product to be detected specifically includes the following steps: Collect multimodal data of the product to be inspected in real time, the multimodal data is obtained based on a variety of sensors and data acquisition equipment deployed on the production line, and the data types of the multimodal data include image data, sound data and infrared data; The collected multimodal data is preprocessed and the data type of the multimodal data is determined, a feature extraction strategy is determined according to the data type, and features of the corresponding type of the multimodal data are extracted according to the feature extraction strategy to obtain a depth feature vector of the product to be inspected.
4. The product yield detection method according to claim 1, characterized in that: The first matching of the depth feature vector of the product to be detected with the initial matching template and calculating a first matching degree, and obtaining a first matching result according to the first matching degree, specifically includes the following steps: Determine the type corresponding to the deep feature vector, analyze the matching requirements of the current product to be detected, determine a matching algorithm according to the data type and the matching requirements, and calculate a first matching degree and a first matching result between the deep feature vector and the initial matching template according to the matching algorithm; The first matching degree is compared with a preset matching threshold. When the first matching degree is higher than the matching threshold, the first matching result is qualified. When the first matching degree is not higher than the matching threshold, the first matching result is unqualified.
5. The product yield detection method according to claim 1, characterized in that: The step of performing a difference analysis on the product to be detected and the initial matching template and obtaining a difference analysis result comprises the following steps: The difference data between the product to be detected and the initial matching template during the matching process are recorded, the difference data are quantified, and a difference analysis result is obtained. The difference analysis result is classified according to the data type, and the classification result includes at least one of image feature difference, sound feature difference and infrared feature difference. The image feature difference includes shape difference, texture difference and size difference, the sound feature difference includes frequency difference and amplitude difference, and the infrared feature difference includes absorption peak position difference and absorption peak intensity difference.
6. The product yield detection method according to claim 5, characterized in that: The dynamically adjusting the parameters of the initial matching template according to the difference analysis result to obtain the final matching template specifically includes the following steps: When the image feature difference exists in the difference analysis result, determining a first difference category which is the largest among the shape difference, the texture difference and the size difference, and determining an image parameter adjustment mechanism in a preset parameter adjustment strategy according to the first difference category; When the sound feature difference exists in the difference analysis result, determining a second difference category which is larger between the frequency difference and the amplitude difference, and determining a sound parameter adjustment mechanism in the parameter adjustment strategy according to the second difference category; When the infrared characteristic difference exists in the difference analysis result, determining a third difference category which is larger between the absorption peak position difference and the absorption peak intensity difference, and determining an infrared parameter adjustment mechanism in the parameter adjustment strategy according to the third difference category; The initial matching template is adjusted according to the determined parameter adjustment strategy to obtain a final matching template.
7. The product yield detection method according to claim 4, characterized in that: Performing secondary matching on the product to be detected and the final matching template to obtain a second matching degree, and obtaining a product yield test result of the product to be detected according to the second matching degree, specifically includes the following steps: Determining a matching algorithm according to the data type and the matching requirement, and calculating a second matching degree and a second matching result between the depth feature vector and the final matching template according to the matching algorithm; Comparing the second matching degree with a preset matching threshold, when the second matching degree is higher than the matching threshold, the second matching result is qualified, and when the second matching degree is not higher than the matching threshold, the second matching result is unqualified; A product yield test result of the product to be tested is generated according to the second matching result and the second matching degree.
8. A product yield detection device, characterized in that: Includes the following modules: A feature extraction module is used to obtain a product sample of the product to be detected, generate an initial matching template based on the product sample, collect multimodal data of the product to be detected, perform feature extraction on the multimodal data, and obtain a deep feature vector of the product to be detected; A primary matching module, used for performing a primary matching between the depth feature vector of the product to be detected and the initial matching template and calculating a first matching degree, and obtaining a first matching result according to the first matching degree, wherein the first matching result includes a qualified result and a unqualified result; A parameter adjustment module, configured to perform a difference analysis on the product to be detected and the initial matching template and obtain a difference analysis result when the first matching result is unqualified, and dynamically adjust the parameters of the initial matching template according to the difference analysis result to obtain a final matching template; The detection result generating module is used to perform secondary matching on the product to be detected and the final matching template to obtain a second matching degree, and obtain a product yield detection result of the product to be detected according to the second matching degree.
9. An intelligent terminal, characterized in that: It includes a memory and a processor, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the product yield detection method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The readable storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the product yield detection method as described in any one of claims 1 to 7.