Production material appearance adaptive recognition method and system based on dynamic template library

Abstract

The application provides a production material appearance adaptive recognition method and system based on a dynamic template library, relates to the technical field of machine vision appearance detection, and solves the problems of learning difficulty and poor recognition accuracy of the prior art when the production material appearance nonlinearly drifts and batch differences exist. In the method, a static reference library and a dynamic incremental library are constructed and maintained, global feature descriptors of a to-be-tested image are extracted, the features are compared with those of the models in the library, and then the highest matching score and the corresponding position in the candidate model set are obtained after matching; a hierarchical strategy is executed based on the highest matching score, and a strategy result of successful recognition, failed recognition and triggering of an incremental learning process is obtained; after triggering of the incremental learning, the features of the to-be-tested image are generated as new auxiliary feature models through multidimensional constraint verification, and the new auxiliary feature models are stored in the dynamic incremental library and applied in the subsequent feature descriptor comparison process. The application successfully realizes the high robustness and continuous evolution effect of the visual positioning system.

Description

Technical Field

[0001] This invention relates to the field of machine vision appearance inspection technology, specifically to a method and system for adaptive recognition of the appearance of production materials based on a dynamic template library. Background Technology

[0002] In the visual positioning systems of precision electronic manufacturing equipment, "offline teaching-online reproduction" is currently the common working mode. Existing high-precision recognition and positioning technologies are developing along multiple technical paths to meet increasingly stringent production demands.

[0003] Some technologies employ acceleration algorithms based on static template matching, typically using standardized cross-correlation scores or geometric contour similarity as recognition criteria, combined with strategies such as image pyramids to improve matching speed. The inherent flaw of these methods lies in their over-reliance on fixed static templates. In semiconductor manufacturing, when materials experience non-functional variations in appearance due to coating, oxidation, or reflectivity fluctuations, static models cannot adapt to feature drift, easily leading to frequent false alarms or rejections, forcing production line shutdowns for parameter adjustments, and severely impairing overall equipment efficiency.

[0004] Other technologies employ supervised learning models based on offline training, utilizing large amounts of pre-collected data for training, thus possessing a certain degree of anti-interference capability. However, the weights of such deep learning models are usually fixed after deployment, and due to their black-box nature, they cannot be flexibly fine-tuned while preserving the original static process baseline. Once a new appearance model not previously covered appears in production, the system struggles to adapt adaptively, still requiring a cumbersome offline retraining process, making it difficult to meet the rapid changeover requirements of flexible production.

[0005] In addition, some technologies involve online learning and resource optimization in specific scenarios, focusing on geometric feedback of mobile robot posture errors or coordinated allocation of computing resources. However, in the specific industrial scenario of semiconductor precision packaging, these solutions lack a complete management logic that balances process standard locking and dynamic feature evolution. They fail to address how to ensure the real-time requirements of high-speed production through weight maintenance mechanisms as the number of templates continues to increase, leading to a significant increase in system load over time.

[0006] In summary, current technologies lack an adaptive recognition scheme that can simultaneously maintain baseline stability, enable online adaptive learning of new features, and keep the processing time constant under multi-template matching. The current industry situation is that technologies focused on speed and accuracy are relatively mature, but technical means to address the issues of continuous evolution and adaptive appearance drift in visual systems are still lacking. Summary of the Invention

[0007] The purpose of this invention is to address the problems of existing visual recognition systems in handling nonlinear drift and batch variations in the appearance of production materials, which cannot simultaneously prevent drift of the original process baseline, online adaptive feature learning, constant computation time, and prevention of mislearning from defective products. This invention proposes an adaptive recognition method and system for the appearance of production materials based on a dynamic template library. By constructing a two-level template library containing static process baselines and dynamically evolving features, and introducing a dual constraint mechanism of geometry and photometry, the system automatically captures and safely learns qualified variation features without downtime. Simultaneously, by combining a cascaded screening and capacity-limited weight maintenance strategy, the system effectively prevents mislearning from defective products and baseline drift, while ensuring that the method's computation time does not deteriorate with the increase of samples, achieving high robustness and continuous evolution of the visual positioning system.

[0008] The present invention employs the following technical solutions to achieve its objective: An adaptive recognition method for the appearance of production materials based on a dynamic template library, the method comprising the following steps: S1. Collect images of standard samples, generate benchmark feature models and store them in a static benchmark library. The models in the static benchmark library have deletion exemption attributes. At the same time, initialize the dynamic incremental library. Set a first preset threshold for judging the recognition result and a second preset threshold for triggering incremental learning. The first preset threshold is greater than the second preset threshold. S2. Collect the image to be tested at the current workstation and extract the global feature descriptor. Compare the global feature descriptor with the feature descriptors of each model in the static benchmark library and the dynamic incremental library to select a set of candidate models whose similarity meets the preset conditions. Match each model in the candidate model set to obtain the highest matching score and the corresponding best matching position. S3. Based on the relationship between the highest matching score and the first and second preset thresholds, a hierarchical strategy is executed. If the highest matching score is greater than or equal to the first preset threshold, the recognition is successful and the location coordinates are output, and the historical call parameters of the matching model are updated. If the highest matching score is less than the second preset threshold, the recognition is failed. If the highest matching score is less than the first preset threshold but greater than or equal to the second preset threshold, an incremental learning process for the image to be tested is triggered. S4. After triggering the incremental learning process, perform multi-dimensional constraint verification on the image to be tested, including affine transformation consistency verification and / or local defect residual detection; when the multi-dimensional constraint verification is passed, extract the features of the image to be tested, generate a new auxiliary feature model, and store it in the dynamic incremental library. S5. When the number of models in the dynamic incremental library has not reached the preset capacity limit, new auxiliary feature models are directly stored. When the number of models in the dynamic incremental library has reached the preset capacity limit, the corresponding activity score is calculated based on the historical call parameters of each existing auxiliary feature model, and the auxiliary feature model with the lowest activity score is eliminated. The new auxiliary feature model is stored in the position of the eliminated auxiliary feature model, completing the update of the dynamic incremental library, and is applied in the subsequent feature descriptor comparison process.

[0009] Specifically, in step S2, a candidate model set that meets the preset similarity conditions is selected, including: calculating the global feature descriptor of the image to be tested, comparing the global feature descriptor with the feature descriptors of all models in the static benchmark library and the dynamic incremental library using Hamming distance or Euclidean distance, and selecting the preset number of models with the highest similarity as the candidate model set. Matching is performed on each model in the candidate model set, including: geometric contour matching and / or full gray-level normalized cross-correlation matching, to obtain the highest matching score and the corresponding best matching position in the candidate model set.

[0010] Preferably, the highest matching score is normalized and its value range is [value range missing]. .

[0011] Specifically, in step S3, if the highest matching score is less than the second preset threshold, a rejection signal is output or a material throwing action is performed while determining that the recognition has failed. When the highest matching score is greater than or equal to the first preset threshold, the historical call parameters of the matching model are updated, and the historical hit count of the model corresponding to the best matching position in the static benchmark library or dynamic incremental library and the timestamp of the most recent call are updated.

[0012] Specifically, in step S4, an affine transformation consistency check is performed on the image to be tested, including: calculating the affine transformation matrix of the image to be tested relative to the reference feature model, decomposing the affine transformation matrix to obtain the corresponding scaling factor and shearing angle; determining whether the scaling factor and shearing angle are both within the preset tolerance range. If so, the affine transformation consistency check is deemed to have passed, and a new auxiliary feature model is generated and stored in the dynamic incremental library; otherwise, the check is deemed to have failed.

[0013] Specifically, in step S4, local defect residual detection is performed on the image to be tested, including: spatially aligning the image to be tested with the baseline feature model and then performing an image subtraction operation to obtain the corresponding residual image; detecting whether there are local difference regions with contrast higher than a preset contrast threshold in the residual image; if no local difference regions exist, the local defect residual detection is deemed to have passed verification; if local difference regions exist, the area of ​​the local difference regions is calculated; if the area of ​​the local difference regions is less than a preset area threshold, the local defect residual detection is deemed to have passed verification; when the verification is deemed to have passed, a new auxiliary feature model is generated and stored in the dynamic incremental library; if the area of ​​the local difference regions is greater than or equal to the preset area threshold, the local defect residual detection is deemed to have failed verification.

[0014] Preferably, in step S4, the multidimensional constraint verification also includes a human-computer interaction confirmation operation; after the affine transformation consistency verification and / or local defect residual detection pass, the process of storing the new auxiliary feature model into the dynamic incremental library is paused, and a comparison image of the image to be tested and the benchmark feature model is presented externally while the process is paused; if an external confirmation instruction is received, the process is resumed, a new auxiliary feature model is generated and stored in the dynamic incremental library; if an external rejection instruction is received, the process is stopped, and the image features of the image to be tested are stored in the negative sample cache library; the features stored in the negative sample cache library are used to compare with the image to be tested in the preprocessing stage of subsequent recognition to intercept abnormal targets in advance.

[0015] Specifically, in step S5, the activity score is calculated. The calculation formula used is as follows:

[0016] In the formula, Historical hit count The weighting coefficient corresponding to the number of historical hits; This is the current timestamp. The timestamp of the most recent call. Recent activity status The corresponding weighting coefficients; When storing a new auxiliary feature model in the position of an obsolete auxiliary feature model, the historical hit count of the new auxiliary feature model is set to 1, and the timestamp of the most recent call to the new auxiliary feature model is set to the current entry time.

[0017] Preferably, when the method is applied to a networked production line containing multiple devices of the same model, the method further includes a network collective intelligence sharing process. In the network collective intelligence sharing process, when one of the devices generates and obtains a new auxiliary feature model through multi-dimensional constraint verification, it broadcasts the data packet of the new auxiliary feature model to the dynamic incremental library of other devices in the networked production line via the industrial local area network. When other devices receive the corresponding data packet, they directly store it into their respective dynamic incremental libraries, thereby completing the synchronization of model data in the dynamic incremental libraries among multiple devices of the same model.

[0018] This invention also provides an adaptive recognition system for the appearance of production materials to implement the aforementioned method. The system includes the following modules: The image acquisition module is used to acquire images of standard samples to generate a baseline feature model and to acquire the image to be tested at the current workstation; it is also used to transmit the acquired image data to the cascaded matching decision module and the constraint verification learning module. The feature library management module is used to maintain the static benchmark library and the dynamic incremental library. It stores the benchmark feature models into the static benchmark library with deletion exemption attributes and initializes the dynamic incremental library. It is also used to calculate the activity score based on the historical call parameters of each auxiliary feature model when the dynamic incremental library reaches the preset capacity limit, eliminate the auxiliary feature model with the lowest activity score, and store the new auxiliary feature model in the corresponding position. The cascaded matching decision module is used to extract the global feature descriptor of the image to be tested, compare it with the feature descriptors of each model in the static benchmark library and the dynamic incremental library, filter out the candidate model set, and match the candidate model set to obtain the highest matching score and the best matching position; it is also used to execute a hierarchical strategy based on the relationship between the highest matching score and the set first preset threshold and second preset threshold, and output the decision results including recognition success, recognition failure, and triggering the incremental learning process; The constraint verification learning module is used to perform multi-dimensional constraint verification on the image under test after triggering the incremental learning process, including affine transformation consistency verification and / or local defect residual detection. It is also used to extract the features of the image under test and generate a new auxiliary feature model when the multi-dimensional constraint verification is passed, and send it to the feature library management module to be stored in the dynamic incremental library.

[0019] In summary, due to the adoption of this technical solution, the beneficial effects of this invention are as follows: This invention possesses extremely high process robustness and adaptability, solving the problem of traditional vision systems being sensitive to material color and texture gradations. It can effectively cope with nonlinear appearance drift and significantly improve the recognition pass rate. When material batch switching causes slight changes in appearance, the system can automatically complete the learning of new features within a very short period of time, achieving seamless line switching without the need for manual shutdown for teaching, and greatly improving the overall utilization rate of the equipment.

[0020] This invention also achieves excellent real-time computation and controllable computing power. Through optimized matching strategies and library capacity control mechanisms, it effectively curbs the linear increase in algorithm time complexity with the increase of sample size, keeping it at a stable constant level. This approach ensures that the system maintains a constant computation time when performing multi-template matching, and its performance does not degrade due to the expansion of the feature library, meeting the stringent real-time requirements of high-speed precision manufacturing.

[0021] This invention also fundamentally guarantees the reliability of process standards. The system ensures that the original process baseline is always locked through a mechanism to remove exemption attributes, effectively preventing baseline drift or catastrophic amnesia of the model over time. Simultaneously, combined with multi-dimensional verification methods, it intercepts abnormal targets, completely eliminating the risk of misjudging defective products as acceptable variations for learning, thus ensuring continuous and stable product quality. Attached Figure Description

[0022] The present invention is described in detail with reference to the following figures, which include five figures as follows: Figure 1 This is a schematic diagram illustrating the overall process of the adaptive identification method for the appearance of production materials according to the present invention. Figure 2 This is a detailed schematic diagram illustrating the process of the method of the present invention; Figure 3 This is a schematic diagram illustrating the mechanism of the static benchmark library and the dynamic incremental library in the method of this invention; Figure 4 This is a schematic diagram illustrating the execution of a hierarchical strategy based on a preset threshold in the method of the present invention; Figure 5 This is a schematic diagram illustrating the dynamic incremental library synchronization when the method of the present invention is applied to a networked production line. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The parts of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0025] Example 1 An adaptive recognition method for the appearance of production materials based on a dynamic template library. Figure 1 A brief overview of the overall process of this method is shown below and can be viewed concurrently; the details of the key steps of this method can be described as follows: S1. Collect images of standard samples, generate benchmark feature models and store them in a static benchmark library. The models in the static benchmark library have deletion exemption attributes. At the same time, initialize the dynamic incremental library. Set a first preset threshold for judging the recognition result and a second preset threshold for triggering incremental learning. The first preset threshold is greater than the second preset threshold. S2. Collect the image to be tested at the current workstation and extract the global feature descriptor. Compare the global feature descriptor with the feature descriptors of each model in the static benchmark library and the dynamic incremental library to select a set of candidate models whose similarity meets the preset conditions. Match each model in the candidate model set to obtain the highest matching score and the corresponding best matching position. S3. Based on the relationship between the highest matching score and the first and second preset thresholds, a hierarchical strategy is executed. If the highest matching score is greater than or equal to the first preset threshold, the recognition is successful and the location coordinates are output, and the historical call parameters of the matching model are updated. If the highest matching score is less than the second preset threshold, the recognition is failed. If the highest matching score is less than the first preset threshold but greater than or equal to the second preset threshold, an incremental learning process for the image to be tested is triggered. S4. After triggering the incremental learning process, perform multi-dimensional constraint verification on the image to be tested, including affine transformation consistency verification and / or local defect residual detection; when the multi-dimensional constraint verification is passed, extract the features of the image to be tested, generate a new auxiliary feature model, and store it in the dynamic incremental library. S5. When the number of models in the dynamic incremental library has not reached the preset capacity limit, new auxiliary feature models are directly stored. When the number of models in the dynamic incremental library has reached the preset capacity limit, the corresponding activity score is calculated based on the historical call parameters of each existing auxiliary feature model, and the auxiliary feature model with the lowest activity score is eliminated. The new auxiliary feature model is stored in the position of the eliminated auxiliary feature model, completing the update of the dynamic incremental library, and is applied in the subsequent feature descriptor comparison process.

[0026] The following section of this embodiment will provide a detailed and optimized description of each step in the method. For the complete process, please refer to [link / reference needed]. Figure 2 The illustration.

[0027] Before execution, the method in this embodiment pre-builds and maintains two logically independent feature libraries in the hardware memory, such as... Figure 3 As shown, it includes a static benchmark library and a dynamic incremental library.

[0028] The static benchmark library stores benchmark feature models generated from the initial teaching by engineers, i.e., models generated based on the images of standard samples in step S1. The library can contain one or more benchmark feature models. These benchmark feature models are always assigned the highest priority and deletion exemption attributes during method execution; the method has no right to delete or modify these benchmark feature models during automated execution.

[0029] The dynamic incremental library is a first-in, first-out (FIFO) or priority queue with a fixed capacity limit. This embodiment uses the elimination mechanism implemented in step S5, which will be described in subsequent steps as a priority queue. The activity score calculated in step S5 represents the priority of the auxiliary feature models in the dynamic incremental library, and the elimination operation is performed accordingly when the capacity is full. The auxiliary feature models are appearance features automatically learned during the online production process of the materials, serving as an aid to identification.

[0030] In addition, in this embodiment, a negative sample cache library can be built and maintained in the hardware memory. The negative sample cache library stores typical defect features obtained after manual confirmation or strong algorithm judgment. These features are manifested in the appearance of production materials as chipped corners, cracks, etc., and abnormal targets can be directly and quickly intercepted in the preprocessing stage of material identification.

[0031] After the method begins execution, it enters the initialization and parameter configuration stage of step S1. First, images of the standard sample are acquired, a baseline feature model is generated, and stored in the static baseline library. Then, a first preset threshold and a second preset threshold are set. These two thresholds are normalized, and their values ​​range from [value range missing]. The first preset threshold is the pass threshold, and the corresponding value is preferably 0.85; the second preset threshold is the learning threshold, and the corresponding value is preferably 0.65; when making a decision later, if the score is lower than the second preset threshold, it means that the recognition is rejected, indicating that the production material corresponding to the image to be tested is not the target material or has serious defects.

[0032] In step S2, as is preferred in this embodiment, a set of candidate models whose similarity meets the preset conditions is selected. This process is the "coarse screening stage". The subsequent matching of each model in the candidate model set is the "fine matching stage".

[0033] In the coarse screening stage, the global feature descriptor of the image to be tested is calculated. This can be done through perceptual hashing (PHash), edge orientation histogram, or downsampling pyramid features, all of which are mature technologies in the field. This embodiment does not modify or limit this part. The calculated global feature descriptor is compared with the feature descriptors of all models in the static benchmark library and the dynamic incremental library using Hamming distance or Euclidean distance. After sorting the similarity in descending order, a preset number of models with the highest similarity can be quickly selected as the candidate model set.

[0034] In the fine-matching stage, geometric contour matching and / or full gray-level normalized cross-correlation matching are performed on each model in the candidate model set to obtain the highest matching score and the corresponding optimal matching position in the candidate model set. The two matching methods used in this embodiment are also mature technologies, and no additional improvements are made to them.

[0035] Geometric contour matching extracts the edge and contour features of an image, including gradient direction and edge points, and locates the image by calculating the similarity between the image to be tested and the template in the geometric edge distribution. Its advantage lies in its strong robustness to changes in illumination, contrast fluctuations, and partial occlusion. It is particularly suitable for industrial materials with clear edge features and fixed shapes, but its recognition effect is limited in scenarios with complex surface textures or blurred edges.

[0036] Full grayscale normalized cross-correlation matching is based on the pixel grayscale values ​​of an image. It evaluates similarity by calculating the normalized cross-correlation coefficient between the template and the local region to be tested. This method introduces normalization processing of mean and standard deviation, which can effectively overcome the influence of linear changes in overall illumination. Therefore, its advantages are high matching accuracy and easy positioning. It is suitable for materials with rich surface texture and complete grayscale information, but its adaptability to nonlinear illumination changes, large-angle rotation or severe occlusion is relatively weak.

[0037] Combining the two matching methods allows for the most accurate acquisition of the highest matching score and corresponding optimal matching position within the candidate model set. Similar to the normalization of the threshold, the highest matching score is also normalized, with a value range of [value missing]. .

[0038] In step S3, based on the highest matching score For details on implementing a tiered strategy, please refer to [link / reference]. Figure 2 or Figure 4 The diagram illustrates that the score range is divided into three confidence intervals by the first preset threshold and the second preset threshold. These are interval A, which is greater than or equal to the first preset threshold, and is considered the qualified zone; interval B, which is less than the second preset threshold, and is considered the rejection zone; and interval C, which is between intervals A and B, and is considered the learning zone.

[0039] In this embodiment, when the highest matching score is greater than or equal to the first preset threshold, it means that the recognition is successful, the positioning coordinates are output and the historical call parameters of the matching model are updated; during the update, the historical hit count of the model corresponding to the best matching position in the static benchmark library or dynamic incremental library and the timestamp of the most recent call are updated.

[0040] If the highest matching score is less than the second preset threshold, a rejection signal will be output or a material throwing action will be performed at the same time as determining that the recognition has failed.

[0041] If the highest matching score is in interval C, it indicates that the current image to be tested differs from the existing model but the main features match, thus triggering the incremental learning process of "verification-input".

[0042] After triggering the incremental learning process in step S4, before generating a new auxiliary feature model from the current image to be tested, it must undergo multidimensional constraint verification. The multidimensional constraint verification must perform at least one of the following: affine transformation consistency verification and local defect residual detection.

[0043] For affine transformation consistency verification, the affine transformation matrix of the image under test relative to the reference feature model is calculated. After decomposing the affine transformation matrix, the corresponding scaling factor and shearing angle are obtained. It is determined whether the scaling factor and shearing angle are both within the preset tolerance range. If so, the affine transformation consistency verification is deemed successful, a new auxiliary feature model is generated and stored in the dynamic incremental library; otherwise, the verification is deemed unsuccessful. In this embodiment, the preset tolerance range represents the precision tolerance. Although it can be determined according to the precision requirements of the actual production materials, taking the scaling factor as an example, it is preferred that the change in the scaling factor is less than 1% before the change in the image under test is determined to be "acceptable variation".

[0044] For local defect residual detection, the image to be tested is spatially aligned with the baseline feature model, and then an image subtraction operation is performed to obtain the corresponding residual image. The residual image is then checked for local difference regions with contrast exceeding a preset contrast threshold. These local difference regions are typically caused by scratches, foreign objects, etc. If no local difference region exists, the local defect residual detection is considered valid. If a local difference region exists, its area is calculated. If the area of ​​the local difference region is less than a preset area threshold, the local defect residual detection is considered valid. When the detection is considered valid, a new auxiliary feature model is generated and stored in the dynamic incremental library. If the area of ​​the local difference region is greater than or equal to the preset area threshold, the local defect residual detection is considered invalid.

[0045] As a preferred embodiment, the multidimensional constraint verification in step S4 also includes a human-computer interaction confirmation operation. After the affine transformation consistency verification and / or local defect residual detection pass, the process of storing new auxiliary feature models into the dynamic incremental library is paused, and a comparison chart of the image to be tested and the benchmark feature model is presented externally while the process is paused.

[0046] Subsequently, if an external confirmation instruction is received from the operator, the process resumes, a new auxiliary feature model is generated, and stored in the dynamic incremental library; if an external rejection instruction is received, the process stops, and the image features of the image to be tested are stored in the aforementioned preferred negative sample cache library; the features stored in the negative sample cache library are used to compare with the image to be tested in the preprocessing stage of subsequent recognition, and to intercept abnormal targets in advance; since the image to be tested at this time is determined by the method to be a learning area, but needs to be rejected after manual confirmation by the operator, the additional storage in the negative sample cache library can provide early judgment of such changes in the image to be tested, and intercept them in advance to reduce the subsequent processing of the method.

[0047] Finally, in step S5, when a new auxiliary feature model needs to be stored, but the dynamic incremental library has reached its capacity limit, a replacement strategy is executed. First, each auxiliary feature model in the dynamic incremental library is traversed, and a comprehensive activity score is calculated based on its historical hit count and the most recent call timestamp. These two criteria reflect the model's versatility and timeliness, respectively.

[0048] Calculate activity score The calculation formula used is as follows:

[0049] In the formula, Historical hit count The weighting coefficient corresponding to the number of historical hits; This is the current timestamp. The timestamp of the most recent call. Recent activity status The corresponding weighting coefficients.

[0050] After the calculation is complete, the auxiliary feature model with the lowest activity score can be identified. This model is one that has not been matched for a long time or has been matched very rarely. It is deleted from memory to make room for a new auxiliary feature model, thus completing the elimination process. Then, when storing the new auxiliary feature model in the position of the eliminated auxiliary feature model, the historical hit count of the new auxiliary feature model is set to 1, and the timestamp of the most recent call of the new auxiliary feature model is set to the current entry time.

[0051] Furthermore, the benchmark feature models in the static benchmark library do not participate in any modification, replacement, or elimination processes, nor do they need to calculate their activity level. The benchmark feature models are permanently retained and used in each screening and matching step S2, which can effectively address the situation of different batches of production materials and serve as a unified benchmark.

[0052] As a preferred embodiment, the method further includes step S6, which is a network-wide intelligence sharing process, and can be viewed synchronously. Figure 5The diagram illustrates this process. In this process, the method is applied to a networked production line containing multiple devices of the same model. When one device generates and passes multi-dimensional constraint verification to obtain a new auxiliary feature model, it broadcasts the data packet of the new auxiliary feature model to the dynamic incremental library of other devices in the networked production line via the industrial local area network. When other devices receive the corresponding data packet, they directly store it in their respective dynamic incremental libraries, thus completing the synchronization of model data in the dynamic incremental libraries among multiple devices of the same model.

[0053] Example 2 Building upon Example 1, this example provides an adaptive recognition system for the appearance of production materials based on a dynamic template library. This system can implement the method of Example 1, meaning it serves as the hardware foundation for the method of Example 1. The system includes the following modules: The image acquisition module is used to acquire images of standard samples to generate a baseline feature model and to acquire the image to be tested at the current workstation; it is also used to transmit the acquired image data to the cascaded matching decision module and the constraint verification learning module. The feature library management module is used to maintain the static benchmark library and the dynamic incremental library. It stores the benchmark feature models into the static benchmark library with deletion exemption attributes and initializes the dynamic incremental library. It is also used to calculate the activity score based on the historical call parameters of each auxiliary feature model when the dynamic incremental library reaches the preset capacity limit, eliminate the auxiliary feature model with the lowest activity score, and store the new auxiliary feature model in the corresponding position. The cascaded matching decision module is used to extract the global feature descriptor of the image to be tested, compare it with the feature descriptors of each model in the static benchmark library and the dynamic incremental library, filter out the candidate model set, and match the candidate model set to obtain the highest matching score and the best matching position; it is also used to execute a hierarchical strategy based on the relationship between the highest matching score and the set first preset threshold and second preset threshold, and output the decision results including recognition success, recognition failure, and triggering the incremental learning process; The constraint verification learning module is used to perform multi-dimensional constraint verification on the image under test after triggering the incremental learning process, including affine transformation consistency verification and / or local defect residual detection. It is also used to extract the features of the image under test and generate a new auxiliary feature model when the multi-dimensional constraint verification is passed, and send it to the feature library management module to be stored in the dynamic incremental library.

[0054] This embodiment illustrates the system's handling methods under different conditions during method execution through a specific application scenario example. The application scenario is the chip picking and positioning process of a high-precision multi-chip placement machine.

[0055] The equipment used in this scenario is a high-precision pick-and-place machine equipped with a 5-megapixel coaxial light source camera. The identified production material is an RF chip with a silicon nitride passivation layer on its surface, and the standard sample appears dark blue. Therefore, parameter configuration is performed first. In this embodiment, a reference feature model of a dark blue standard chip is stored in the static reference library through the feature library management module; and the capacity of the dynamic incremental library is set to 10 empty slots. The first preset threshold is set to 0.85, and the second preset threshold is set to 0.65.

[0056] During the normal production identification phase, the appearance of the first 2,000 chips was highly consistent. After the system matched them with the benchmark feature model, the highest matching score was above 0.9. Therefore, the system ran at full speed without any identification failures or triggering of the incremental learning process.

[0057] The following are examples of scenarios involving mutation triggering and incremental learning during the normal production identification phase: When the production line feeder was replaced with a new batch of chips, due to slight fluctuations in the coating thickness, this batch of chips appeared slightly dark purple under the coaxial light source camera. At this time, after the equipment's vision system acquired the image to be tested, it matched it with the dark blue baseline feature model. Due to the difference in grayscale distribution, the highest matching score calculated was 0.74.

[0058] The highest matching score of 0.74 falls between the first and second preset thresholds. The system will not report an error on the production line, but will instead trigger an incremental learning process for background processing. During processing, based on affine transformation consistency verification and local defect residual detection, the aspect ratio of the extracted chip edge is 1.01, within the preset tolerance. No abnormalities such as chipped corner residuals are detected. Therefore, the system classifies it as a qualified color variation. Assuming this scenario does not require human-computer interaction confirmation, the multi-dimensional constraint verification passes. Finally, the system generates the dark purple test image as the first auxiliary feature model and stores it in the dynamic incremental library.

[0059] Subsequently, during the production of the same batch of chips, when the next dark purple chip arrived, the system simultaneously searched for and matched the baseline feature model and the first auxiliary feature model, selecting the highest matching score. The first auxiliary feature model had a score of 0.96, which was the highest matching score. The system then accepted this matching result, determined that the chip was successfully identified, output the positioning coordinates, updated the historical call parameters, and guided the robotic arm to accurately grasp the chip and enter the subsequent production line. Throughout this process, the system operated without interruption, the operators were unaware of it, and the production line maintained efficient automatic operation.

[0060] In the maintenance and interference removal scenario of the dynamic incremental library, as production progresses, the dynamic incremental library has filled with 10 models of different subtle color differences. At this point, a new texture feature needs to be added to the library. The system begins scanning the 10 auxiliary feature models in the dynamic incremental library and finds that the third auxiliary feature model is a one-time model that was accidentally generated 4 hours ago due to a momentary flicker of the light source. Its historical hit count is only 1, and its most recent call time is also relatively long. After calculating the activity score, the third auxiliary feature model has the lowest activity score and is therefore eliminated. The system deletes it from the dynamic incremental library and writes the auxiliary feature model corresponding to the new texture feature.

[0061] Finally, regarding the anomaly interception mechanism, during the production process, if a defective chip with severe scratches is encountered, and its highest matching score is below 0.65, it can be directly deemed an identification failure, and a rejection signal or a discard action can be executed simultaneously. If the chip's highest matching score is between 0.65 and 0.85, after triggering the incremental learning process, local defect residual detection reveals a local difference region on its surface, and the area is greater than a preset area threshold, thus verification fails. Since it cannot pass the multidimensional constraint verification, the system refuses to learn it as a new auxiliary feature model and, depending on the situation, adds it to the negative sample cache. At the same time, it directly sends a discard command to the equipment to prevent defective products from flowing into the good product area.

Claims

1. A method for adaptive recognition of the appearance of production materials based on a dynamic template library, characterized in that, The method includes the following steps: S1. Collect images of standard samples, generate benchmark feature models and store them in a static benchmark library. The models in the static benchmark library have deletion exemption attributes; at the same time, initialize the dynamic incremental library. Set a first preset threshold for determining the recognition result and a second preset threshold for triggering incremental learning, wherein the first preset threshold is greater than the second preset threshold; S2. Collect the image to be tested at the current workstation and extract the global feature descriptor. Compare the global feature descriptor with the feature descriptors of each model in the static benchmark library and the dynamic incremental library to select a set of candidate models whose similarity meets the preset conditions. Match each model in the candidate model set to obtain the highest matching score and the corresponding best matching position. S3. Based on the relationship between the highest matching score and the first and second preset thresholds, execute the tiered strategy; If the highest matching score is greater than or equal to the first preset threshold, the recognition is considered successful and the location coordinates are output, and the historical call parameters of the matching model are updated; if the highest matching score is less than the second preset threshold, the recognition is considered unsuccessful; if the highest matching score is less than the first preset threshold but greater than or equal to the second preset threshold, the incremental learning process for the image to be tested is triggered. S4. After triggering the incremental learning process, perform multi-dimensional constraint verification on the image to be tested, including affine transformation consistency verification and / or local defect residual detection. When the multidimensional constraint verification is passed, the features of the image to be tested are extracted, a new auxiliary feature model is generated, and it is stored in the dynamic incremental library. S5. When the number of models in the dynamic incremental library has not reached the preset capacity limit, new auxiliary feature models are directly stored. When the number of models in the dynamic incremental library has reached the preset capacity limit, the corresponding activity score is calculated based on the historical call parameters of each existing auxiliary feature model, and the auxiliary feature model with the lowest activity score is eliminated. The new auxiliary feature model is stored in the position of the eliminated auxiliary feature model, completing the update of the dynamic incremental library, and is applied in the subsequent feature descriptor comparison process.

2. The adaptive recognition method for the appearance of production materials according to claim 1, characterized in that, In step S2, a candidate model set that meets the preset similarity conditions is selected, including: calculating the global feature descriptor of the image to be tested, comparing the global feature descriptor with the feature descriptors of all models in the static benchmark library and the dynamic incremental library using Hamming distance or Euclidean distance, and selecting the preset number of models with the highest similarity as the candidate model set. Matching is performed on each model in the candidate model set, including: geometric contour matching and / or full gray-level normalized cross-correlation matching, to obtain the highest matching score and the corresponding best matching position in the candidate model set.

3. The adaptive recognition method for the appearance of production materials according to claim 2, characterized in that: The highest matching score is normalized and its value range is [value range missing]. .

4. The adaptive recognition method for the appearance of production materials according to claim 1, characterized in that: In step S3, if the highest matching score is less than the second preset threshold, a rejection signal is output or a material throwing action is performed at the same time as determining that the recognition has failed. When the highest matching score is greater than or equal to the first preset threshold, the historical call parameters of the matching model are updated, and the historical hit count of the model corresponding to the best matching position in the static benchmark library or dynamic incremental library and the timestamp of the most recent call are updated.

5. The adaptive recognition method for the appearance of production materials according to claim 1, characterized in that, In step S4, an affine transformation consistency check is performed on the image to be tested, including: calculating the affine transformation matrix of the image to be tested relative to the reference feature model, decomposing the affine transformation matrix to obtain the corresponding scaling factor and shearing angle; determining whether the scaling factor and shearing angle are both within the preset tolerance range. If so, the affine transformation consistency check is deemed to have passed, and a new auxiliary feature model is generated and stored in the dynamic incremental library; otherwise, the check is deemed to have failed.

6. The adaptive recognition method for the appearance of production materials according to claim 1, characterized in that, In step S4, local defect residual detection is performed on the image to be tested, including: spatially aligning the image to be tested with the reference feature model and then performing an image subtraction operation to obtain the corresponding residual image; detecting whether there are local difference regions with contrast higher than a preset contrast threshold in the residual image; if no local difference regions exist, the local defect residual detection is deemed to have passed the verification; if local difference regions exist, the area of ​​the local difference regions is calculated; if the area of ​​the local difference regions is less than a preset area threshold, the local defect residual detection is deemed to have passed the verification; when the verification is deemed to have passed, a new auxiliary feature model is generated and stored in the dynamic incremental library; if the area of ​​the local difference regions is greater than or equal to the preset area threshold, the local defect residual detection is deemed to have failed the verification.

7. The adaptive recognition method for the appearance of production materials according to claim 5 or 6, characterized in that: In step S4, the multidimensional constraint verification also includes a human-computer interaction confirmation operation; after the affine transformation consistency verification and / or local defect residual detection pass, the process of storing the new auxiliary feature model into the dynamic incremental library is paused, and a comparison image of the image to be tested and the benchmark feature model is presented externally while the process is paused; if an external confirmation instruction is received, the process is resumed, a new auxiliary feature model is generated and stored in the dynamic incremental library; if an external rejection instruction is received, the process is stopped, and the image features of the image to be tested are stored in the negative sample cache library. The features stored in the negative sample cache are used to compare with the image to be tested in the preprocessing stage of subsequent identification, so as to intercept abnormal targets in advance.

8. The adaptive recognition method for the appearance of production materials according to claim 1, characterized in that, In step S5, the activity score is calculated. The calculation formula used is as follows: In the formula, Historical hit count The weighting coefficient corresponding to the number of historical hits; This is the current timestamp. The timestamp of the most recent call. Recent activity status The corresponding weighting coefficients; When storing a new auxiliary feature model in the position of an obsolete auxiliary feature model, the historical hit count of the new auxiliary feature model is set to 1, and the timestamp of the most recent call to the new auxiliary feature model is set to the current entry time.

9. The adaptive recognition method for the appearance of production materials according to claim 1, characterized in that: When this method is applied to a networked production line containing multiple devices of the same model, the method also includes a network collective intelligence sharing process. In the network collective intelligence sharing process, when one of the devices generates and obtains a new auxiliary feature model through multi-dimensional constraint verification, it broadcasts the data packet of the new auxiliary feature model to the dynamic incremental library of other devices in the networked production line via the industrial local area network. When other devices receive the corresponding data packet, they directly store it into their respective dynamic incremental libraries, thus completing the synchronization of model data in the dynamic incremental libraries among multiple devices of the same model.

10. A production material appearance adaptive recognition system for implementing the method of claim 1, characterized in that, The system includes the following modules: The image acquisition module is used to acquire images of standard samples to generate a baseline feature model and to acquire the image to be tested at the current workstation; it is also used to transmit the acquired image data to the cascaded matching decision module and the constraint verification learning module. The feature library management module is used to maintain the static benchmark library and the dynamic incremental library. It stores the benchmark feature models into the static benchmark library with deletion exemption attributes and initializes the dynamic incremental library. It is also used to calculate the activity score based on the historical call parameters of each auxiliary feature model when the dynamic incremental library reaches the preset capacity limit, eliminate the auxiliary feature model with the lowest activity score, and store the new auxiliary feature model in the corresponding position. The cascaded matching decision module is used to extract the global feature descriptor of the image to be tested, compare it with the feature descriptors of each model in the static benchmark library and the dynamic incremental library, filter out the candidate model set, and match the candidate model set to obtain the highest matching score and the best matching position; it is also used to execute a hierarchical strategy based on the relationship between the highest matching score and the set first preset threshold and second preset threshold, and output the decision results including recognition success, recognition failure, and triggering the incremental learning process; The constraint verification learning module is used to perform multi-dimensional constraint verification on the image under test after triggering the incremental learning process, including affine transformation consistency verification and / or local defect residual detection. It is also used to extract the features of the image under test and generate a new auxiliary feature model when the multi-dimensional constraint verification is passed, and send it to the feature library management module to be stored in the dynamic incremental library.

Images