Defect detection method and storage medium

Abstract

The present application relates to a defect detection method and storage medium. The present application first obtains multiple time-series images of a target object, performs defect detection on the multiple time-series images, and obtains a first candidate defect type of the target object. Then, a target detection mode that matches the first candidate defect type is selected from a plurality of pre-set detection modes to verify the first candidate defect type. Finally, the first candidate defect type that passes the verification is determined as the first defect type of the target object. In this way, customized training can be performed for different targets and different production scenarios to meet the needs of complex scenarios and improve the accuracy and adaptability of defect detection on targets.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and in particular to a defect detection method and storage medium. Background Art

[0002] Industrial visual inspection is a technical system that leverages computer vision technology, image processing algorithms, and hardware equipment to perform automated quality inspection and defect identification on products, components, or production processes in industrial production. Amid the rapid development of intelligent manufacturing, industrial visual inspection, as a core quality control technology, is widely used across a wide range of industries. Traditional industrial visual inspection primarily relies on deep learning algorithms and conventional image processing algorithms. Deep learning algorithms, with their powerful feature extraction capabilities, demonstrate certain advantages in defect identification. However, they suffer from model complexity, high training costs, and poor generalization. Conventional algorithms, such as threshold segmentation and edge detection, while computationally efficient and highly interpretable, struggle when handling complex scenarios, such as those with dynamic interference, such as in glass bottle production lines. Dynamic interference, such as water ripples and light and shadow changes, can easily be misidentified as surface defects such as cracks and stains. Traditional defect detection methods are usually calculated based on fixed thresholds. Due to the single threshold screening and lack of dynamic adaptability, they cannot effectively deal with the time-varying and diversity of dynamic interference, making it difficult to accurately distinguish between real defects and interference factors, resulting in a high false detection rate. This not only increases the cost of manual re-inspection, but also reduces production efficiency, becoming a bottleneck restricting the further development and application of industrial visual inspection technology. Summary of the Invention

[0003] The purpose of this application is to provide a defect detection method and storage medium to solve the problem of high false positive rate in traditional defect detection.

[0004] In order to achieve the above objectives, the present application provides a first aspect of a defect detection method, comprising:

[0005] Acquire multiple time-series images of the target object;

[0006] Performing defect detection on the multiple time-series images to obtain a first candidate defect type of the target object;

[0007] Selecting a target detection mode that matches the first candidate defect type from a plurality of pre-set detection modes to verify the first candidate defect type, wherein the plurality of detection modes include a dynamic detection mode and a static detection mode;

[0008] The first candidate defect type that passes the verification is determined as the first defect type of the target object.

[0009] A second aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program, and the program can be loaded by a processor to execute the above-mentioned defect detection method.

[0010] The beneficial effects of this application are:

[0011] This application first captures the dynamic change process of the target object by acquiring multiple time-series images of the target object, and then performs defect detection on the multiple time-series images to obtain the first candidate defect type of the target object. Then, a target detection mode that matches the first candidate defect type is selected from multiple pre-set detection modes to verify the first candidate type. By pre-setting multiple detection modes including dynamic detection modes and static detection modes, customized training can be performed for different targets and different production scenarios, and it can adapt to complex working conditions such as lighting changes and product model switching to meet the needs of complex scenarios. Compared with the single-dimensional feature judgment and detection under a fixed threshold of traditional defect detection, this application performs defect detection on multiple time-series images through multiple detection modes, which improves the accuracy and adaptability of defect detection on targets.

[0012] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic flow chart of a defect detection method provided in one embodiment of the present application;

[0014] Figure 2 A schematic flow chart of a defect detection method in a global static detection mode provided in an embodiment of the present application;

[0015] Figure 3 A schematic flow chart of a defect detection method in a global dynamic detection mode provided in an embodiment of the present application;

[0016] Figure 4 A schematic flow chart of a defect detection method in a moving target locking mode provided in an embodiment of the present application;

[0017] Figure 5 A schematic flow chart of a defect detection method in a static target enhancement mode provided in an embodiment of the present application;

[0018] Figure 6 A schematic flow chart of a defect detection method provided in another embodiment of the present application;

[0019] Figure 7 This is a structural diagram of a defect detection system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0021] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly specifying the number of the technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the described features. In the description of this application, "plurality" means two or more, unless otherwise specifically qualified. In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is provided to enable anyone skilled in the art to implement and use the present application. In the following description, details are listed for illustrative purposes. It should be understood that one of ordinary skill in the art will recognize that the present application can be implemented without these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0022] The defect detection method in the embodiment of the present application may be defect detection for a target object. As an example, the target object in the embodiment of the present application may be an industrial product. Industrial products are products manufactured through standardized production processes with the help of industrial technology. In the embodiment of the present application, the target object is a glass medicine bottle as an example. The bottle diameter, bottle mouth thread specifications, etc. of the glass medicine bottle must comply with the pharmaceutical packaging standards. Once a defect occurs in a glass medicine bottle, it may cause batch quality problems. Therefore, defect detection of glass medicine bottles to ensure that the glass medicine bottles meet standardization requirements is a key link in quality control in industrial production.

[0023] Figure 1 FIG. 1 is a flow chart of a defect detection method provided in one embodiment of the present application. Figure 1 As shown, the defect detection method may include steps 101-103, which are described in detail below.

[0024] Step 101: Acquire multiple time-series images of a target object.

[0025] In the embodiments of the present application, a multi-time sequence image refers to a series of images captured of a target object at different time points. The sequence information of the time dimension can reflect the state change of the target object, that is, there is a time sequence between the multiple captured images of the target object. In one example, the image capture of the target object can be performed at fixed time intervals. In another example, the image capture of the target object can also be triggered by a specific event. In another example, continuous shooting can also be performed without interruption to form a long time-series video stream.

[0026] Taking the glass medicine bottle as an example, multi-time series imaging not only captures surface defects in static glass bottles, but also records dynamic information such as light and shadow changes caused by the impact of liquid flow, bubble formation, and the movement trajectory of defects, providing more comprehensive and rich data for subsequent defect detection. Therefore, by introducing dynamic information in the time dimension, multi-time series imaging can provide a more comprehensive analytical perspective for defect detection of targets and is a key means of connecting static vision with dynamic scenes.

[0027] Step 102: perform defect detection on the multiple time-series images to obtain a first candidate defect type of the target object.

[0028] In an embodiment of the present application, the first candidate defect type refers to a possible defect type obtained by preliminary detection of defects of the target object based on multiple time-series images. The first candidate type belongs to a small category of defect types. In an example, the first candidate defect type can be detected by traditional visual algorithms (such as edge detection, threshold segmentation, and template matching) or by deep learning algorithms (such as single-stage target detection algorithms (You Only Look Once, YOLO) and region generation networks (Faster Region-based Convolutional Neural Networks, Faster R-CNN), etc.). Since the first candidate defect type may contain pseudo defects caused by misdetection, the first candidate defect type needs to be further verified to determine whether the first candidate defect type is a real defect.

[0029] Step 103: Select a target detection mode that matches the first candidate defect type from a plurality of pre-set detection modes to verify the first candidate defect type. The plurality of detection modes include a dynamic detection mode and a static detection mode.

[0030] In this step, the detection mode is used to verify the first candidate defect type. Since the target object may have multiple defects, such as dynamic defects and static defects, in the embodiment of the present application, multiple detection modes including dynamic detection mode and static detection mode can be pre-set. The dynamic detection mode is suitable for scenarios where the defect characteristics change over time (such as foreign objects moving inside liquids, etc.), and the static detection mode is suitable for scenarios where the defect characteristics are stable (such as fixed scratches, etc.). For example, if the position and shape of the defect are fixed, the static detection mode can be selected to match the defect. If the position of the defect changes significantly, the dynamic detection mode can be selected.

[0031] Each defect type can correspond to one or more detection modes. If a defect type corresponds to multiple detection modes, a comprehensive evaluation based on the results of these multiple detection modes can be performed to verify the defect type. Therefore, for each first candidate defect type, a matching target detection mode can be selected to verify the first candidate defect type. The target detection mode refers to the detection mode among the multiple detection modes that matches the first candidate defect type.

[0032] By selecting a target detection mode from multiple detection modes, including static and dynamic, to match the first candidate defect type, the system can adapt to different defect types and various operating conditions. Furthermore, when switching between operating conditions, targeted defect detection is possible without manual parameter adjustment. Furthermore, static detection mode consumes fewer resources than dynamic detection mode, effectively utilizing system resources and reducing resource utilization.

[0033] Step 104: Determine the first candidate defect type that passes the verification as the first defect type of the target object. The first defect type is the first candidate defect type that passes the verification and can be used for subsequent classification, alarming, or sorting.

[0034] The embodiment of the present application first captures the dynamic change process of the target object by acquiring multiple time-series images of the target object, and then performs defect detection on the multiple time-series images to obtain the first candidate defect type of the target object. Next, a target detection mode that matches the first candidate defect type is selected from a plurality of pre-set detection modes to verify the first candidate type. By pre-setting multiple detection modes including dynamic detection modes and static detection modes, customized training can be performed for different targets and different production scenarios, and it can adapt to complex working conditions such as lighting changes and product model switching to meet the needs of complex scenarios. Compared with the single-dimensional feature judgment and detection under a fixed threshold of traditional defect detection, the embodiment of the present application performs defect detection on multiple time-series images through multiple detection modes, which improves the accuracy and adaptability of defect detection on targets.

[0035] In an embodiment of the present application, multiple time-series images of the target object can be collected by at least two non-tracking cameras. In one example, a dual-color image acquisition device system can be set to work, and the dual-color image acquisition device can include a first image acquisition device and a second image acquisition device. The image range acquired by the first image acquisition device and the second image acquisition device covers the target acquisition area of ​​the target object. The target acquisition area refers to the area required for defect detection and analysis. The first image acquisition device and the second image acquisition device cover the target acquisition area of ​​the target object, which can reduce the occurrence of detection blind spots. Taking the target object as a glass medicine bottle as an example, the target acquisition area can be the entire surface of the bottle body. The first image acquisition device can be set at a position away from the bottleneck to observe the internal situation of the liquid in the glass medicine bottle, and the second image acquisition device can be set at a position away from the bottom of the bottle to focus on the detection of the lower part of the bottle body, thereby minimizing the interference of the bottle bottom structure on the imaging.

[0036] Therefore, in step 101, images of the target object at multiple time points captured by at least two image capture devices can be acquired to obtain multiple time-series images of the target object, wherein the image range captured by the at least two image capture devices covers the target capture area of ​​the target object.

[0037] Taking the example of an image acquisition device comprising a first acquisition device and a second acquisition device, images of the target object at multiple time points can be acquired based on the first image acquisition device and the second image acquisition device to obtain multiple time-series images of the target object. In one example, multiple frames of images of the target object in the second direction after rotation can be synchronously acquired at set time intervals to form an image set containing a time series, i.e., a multiple time-series image. For example, during the inspection process, the moving target object is continuously photographed (e.g., 3-10 frames), while the first image acquisition device and the second image acquisition device remain stationary. This dynamic shooting method not only ensures the stability of image acquisition, but also comprehensively captures the characteristics of the target object at all angles. This makes it easier to analyze the evolution rate of dynamic defects.

[0038] In traditional inspections, the random angle of the target object can cause the same defect to have different morphologies in different images, increasing the difficulty of algorithmic identification. To address this, in step 101, in response to detecting a target object in a first orientation, the target object can be rotated to a second orientation. The second orientation is angled relative to the first orientation. The image acquisition device captures a wider field of view for the target object in the second orientation than for the first orientation. Therefore, by rotating the target object to a uniform orientation, the spatial distribution of similar defects can be made consistent across all images, facilitating the extraction of uniform image features.

[0039] For example, assuming the first direction is vertical, the second direction is horizontal. The embodiment of the present application adopts a gentle design that effectively expands the visible area inside the glass medicine bottle by rotating a vertically fed target object, such as a glass medicine bottle, 90 degrees to a horizontal direction. While avoiding complex structural areas such as the bottle bottom and bottleneck, the glass medicine bottle to be inspected for defects is guided to areas with less interference, such as the bottle shoulder and bottle body, thereby improving inspection accuracy.

[0040] The embodiments of the present application combine the spatial posture control of the target object with the depth of multi-time series images through the design of posture standardization and multi-perspective collaboration, breaking through the problems of feature variation caused by random posture and insufficient single-perspective coverage in traditional detection, and can be applied to scenarios with high requirements for detection accuracy and integrity.

[0041] In step 102, a target detection mode matching each first candidate defect type may be determined based on a pre-built defect feature knowledge base, wherein the defect feature knowledge base is built based on a mapping relationship between defect types and detection modes.

[0042] In the embodiments of the present application, each defect type can correspond to at least one target detection mode, allowing for more targeted detection of different defect types. Therefore, the target detection mode is selected based on the corresponding detection mode for the first candidate defect type. By pre-establishing a mapping between defect types and detection modes, the optimal detection modes corresponding to different defect types can be recorded.

[0043] For example, let's consider the detection of heavy foreign objects. Due to their physical characteristics, they exhibit minimal displacement across multiple time-series images, so static target detection can be employed. However, the specific characteristics of each defect (such as color change, shape, motion, and fluid properties) ultimately determine the weighting used in similarity assessment and the maximum displacement used in target trajectory determination. These parameters are derived from extensive data validation, analysis, and statistical analysis.

[0044] This embodiment of the present application uses an example where the detection mode includes four types. These four types may include dynamic defects, static defects, material defects, and composite defects. In one example, defect types can be categorized using a four-dimensional (i.e., four-type) defect classification system. Table 1 shows an example of a four-dimensional defect classification system.

[0045] Table 1

[0046] Dimensions Physical properties Static defects Shape parameters, grayscale distribution Dynamic defects Motion trajectory, acceleration Material defects Reflectance, texture spectrum Composite defects Spatiotemporal correlation matrix

[0047] In an embodiment of the present application, the target detection mode may include a full-area static detection mode, a full-area dynamic detection mode, a moving target locking mode, and a static target enhancement mode. The full-area static detection mode is applicable to scenarios where defect characteristics are mainly manifested as static attributes, such as scratches at fixed positions, dimensional deviations, color anomalies, etc. The full-area dynamic detection mode is applicable to scenarios where defect characteristics are mainly manifested as dynamic changes, such as crack expansion, foreign matter in liquids, and liquid stirring. The moving target locking mode is applicable to scenarios where the target is in motion but there is static interference in the background (such as shadows from fixed light sources, surface textures of the target). The static target enhancement mode is applicable to scenarios where the target is static but there is dynamic interference (such as jitter of the image acquisition device, stirring of the liquid surface, etc.).

[0048] Based on this, the embodiments of the present application can construct an intelligent decision-making system of "feature retrieval-pattern matching". Based on experiments and experience, etc., for the characteristics of each defect type, a corresponding relationship between each defect type and the detection mode can be established. By quantitatively analyzing the defect type, the optimal detection mode is automatically selected. For example, the retrieval results obtained by first performing feature retrieval on the multi-time series images of the target object are automatically selected based on the retrieval results to obtain the optimal algorithm combination. Intelligent decisions can be made without human intervention, which significantly improves the detection accuracy in complex scenarios. There is no need to use a unified complex algorithm to process all scenarios, reducing the consumption of computing resources.

[0049] After matching the target detection mode of the first candidate defect type, the first candidate defect type can be verified through each target detection mode respectively. In an embodiment of the present application, defects can be classified in the form of major categories and minor categories. The major defect type refers to a collection of features for macro-classification of target defects, which is used to define the basic type or category of defects. The minor defect type is a subdivision feature of the major defect type, which is used to describe the specific manifestation or technical parameters of the defect. Therefore, the defect type of the target object can include a first defect type and a second defect type that includes the first defect type. Among them, the first defect type is a minor defect, and the second defect type is a major defect. Based on this, in an embodiment of the present application, for each target detection mode, a preliminary defect verification can be performed on the second defect type, and then a detailed defect verification can be performed on the first defect type included in the second defect type that has passed the verification. In this way, the first defect types belonging to the same second defect type can be uniformly verified to reduce the computational complexity of the verification. Compared with the traditional method that requires verification of each sub-category defect type, layered detection of major and minor categories can reduce time consumption, lower the probability of confusion and false detection rate, and share the framework of major category defect detection and parameterized configuration of minor categories to reduce the cost of defect detection.

[0050] Specifically, the first candidate defect type corresponding to each target detection mode can be clustered separately to obtain the second candidate defect type corresponding to the target detection mode. The second candidate defect type is a large category defect that may be the second defect type. Each target detection mode is matched with the corresponding first candidate defect type. Therefore, for each target detection mode, the corresponding first candidate defect type can be clustered first to obtain the large category defect type that may exist under the target detection mode, that is, the second candidate defect type. Then, the second candidate defect type is verified. If the second candidate defect type passes the verification, it means that the second candidate defect type is not a pseudo-large category defect. Therefore, the second candidate defect type that passes the verification can be determined as the second defect type. Furthermore, the first candidate defect type included in the second defect type can be verified.

[0051] In one example, a post-processing switch can be set. This switch is a control node in the defect detection process for an object, used to enable or disable the refined processing of defect types to determine whether to perform verification operations on the first candidate defect type initially detected. After obtaining the first candidate defect type, it can be determined whether the post-processing switch is enabled. If the post-processing switch is enabled, verification operations can be performed on the first candidate defect type. If the post-processing switch is disabled, this step can be skipped, and the preliminary detection result (i.e., the first candidate defect type) can be directly output or other processes can be entered. In industrial defect detection, each sub-category of defect type can be algorithmically calculated with a confidence score that represents the likelihood or severity of the sub-category of defect type. Therefore, if the post-processing switch is disabled, the first candidate defect type and the sub-category defect type with the highest score can be directly returned, allowing the object's defect to be determined based on quantified priority. In this way, in scenarios with a single known defect and few interfering factors, disabling the post-processing switch can simplify the detection process and reduce computing resource consumption.

[0052] If the post-processing switch is on, the first candidate defect type can be verified. First, the second defect type of the target object in each target detection mode is determined to narrow the subsequent screening range. Then, a deeper verification of the second defect type is performed.

[0053] Specifically, the first candidate defect type can be screened for multi-dimensional features to obtain a multi-dimensional confidence evaluation value. Multi-dimensionality refers to the use of multiple feature extraction methods to evaluate the confidence of the first candidate defect type. The confidence level can reflect whether the first candidate defect type truly belongs to the subcategory of defect types. For example, the confidence level can be evaluated based on multiple dimensions, such as the location and shape of the region of interest (ROI), grayscale distribution, mean and variance, area and size, and texture, to obtain a multi-dimensional confidence evaluation value. For example, this can be a confidence score, which can range from 0 to 1. A higher confidence score indicates a greater probability that the first candidate defect type is true. Then, based on the multi-dimensional confidence evaluation values, a confidence evaluation value for the first candidate defect type is obtained. For example, the confidence evaluation values ​​from multiple dimensions can be comprehensively calculated using a weighted summation method to obtain an overall confidence evaluation value for the first candidate defect type. Finally, the first candidate defect type is verified based on the confidence evaluation value. If the first candidate defect type passes the verification, the first candidate defect type with the smallest confidence evaluation value can be returned, so that the retained first defect type meets the minimum confidence requirement, thereby reducing the existence of false defects.

[0054] As an example, embodiments of the present application can also perform regional screening on the first region where the second defect type is located to obtain a second region where the first candidate defect type is located. The first region refers to the ROI where the second defect type is located, and the second region refers to the ROI where the first candidate defect type is included in the second defect type. Therefore, the second region is contained within the first region.

[0055] For example, ROI screening, grayscale screening, and area screening can be performed on the first region containing the second defect type to obtain the second region containing the first candidate defect type. ROI screening eliminates irrelevant background interference based on the areas where the second defect type may be present. Grayscale screening filters out pseudo-defects with grayscales close to those of normal areas by setting a grayscale threshold. Area screening eliminates minor noise interference by setting an area threshold based on the target's specifications.

[0056] In the embodiments of this application, the global static detection mode is applicable to a single-frame image, while the global dynamic detection mode, the moving target lock mode, and the static target enhancement mode are applicable to multiple frames. Therefore, for the global dynamic detection mode, the moving target lock mode, and the static target enhancement mode, it is necessary to first align the multiple time-series images before the detection begins. For example, the first time-series image in the time sequence is used as a template to calculate the feature parameters, and then the subsequent time-series images are shifted.

[0057] Similarly, before performing the ROI screening, grayscale screening, and area screening operations on the first region where the second defect type is located, it is also necessary to perform an alignment operation on the multiple time-series images.

[0058] Specifically, if the second defect type is the target defect type, which is a defect type that requires multiple sequential images for preprocessing, the multiple sequential images are mapped from the original coordinate system to the standard coordinate system. The multiple sequential images mapped to the standard coordinate system are cropped, rotated, and aligned to obtain the preprocessed second defect type.

[0059] The following describes the verification of the first candidate defect type using the four target detection modes described above as examples. For example, the first defect type is a minor defect and the second defect type is a major defect. In the flowchart, screening the current major defect type involves clustering the initially detected minor defects into major defects. The system then determines whether a major defect exists under each target detection mode. If so, preprocessing is performed. Otherwise, the system exits the current detection phase and proceeds to the next major defect type.

[0060] When the target detection mode is the global static detection mode, if the confidence evaluation value of the first candidate defect type meets the first confidence threshold, the first candidate defect type is determined to have passed the verification. The first confidence threshold refers to the threshold at which the first candidate defect type can be determined to be the first defect type in the global static detection mode.

[0061] Figure 2 This is a flow chart illustrating a defect detection method in a global static inspection mode, provided in an embodiment of the present application. Based on defect types (such as scratches and bubbles) preset in a defect feature knowledge base, static defect features (such as linear grayscale mutations for cracks and area thresholds for stains) are prioritized, while dynamic interference (such as ripples and reflective glint) is filtered out. For complex structures like irregularly shaped bottles, defect regions (such as the bottle shoulder) can be mapped to standard coordinates through cropping and rotation, simplifying feature calculation. Feature screening can then be performed. For example, ROI extraction focuses on key areas such as the bottle body and PCB solder joints, while excluding invalid background such as the bottle bottom and neck. Grayscale / area thresholding: Potential defects are screened based on the grayscale variance (such as grayscale mutations for scratches) and area range (such as the minimum area of ​​impurities) within a single frame image. Next, the first candidate defect type is determined, and thresholds are applied to the selected features (such as a crack aspect ratio ≥ 5 and a grayscale variance ≥ 20), directly outputting the first defect type. If the detection fails (e.g., feature mismatch), the current test is aborted to avoid misidentifying dynamic interference as static defects.

[0062] When the target detection mode is the global dynamic detection mode, if the confidence evaluation value of the first candidate defect type meets the second confidence threshold, and the area similarity and grayscale similarity of the first candidate defect type in the multi-time series images meet the similarity threshold, the first candidate defect type is determined to have passed the verification. The second confidence threshold refers to the threshold at which the first candidate defect type can be determined to meet the first defect type determination criteria in the global dynamic detection mode.

[0063] Figure 3 The following is a flow chart of a defect detection method in a global dynamic detection mode provided in an embodiment of the present application. First, multiple time-series images are aligned. For example, the first frame of the time-series image is used as a template to calculate the characteristic parameters of the subsequent frame time-series images, and then the subsequent frame time-series images are translated (such as the coordinates of the bottle edge). This can reduce the spatial offset of the target object's movement. The global dynamic defect detection logic is then entered. When the post-processing switch is "Yes," the second defect feature (major defect category) is screened, dynamic defect features (such as foreign object motion trajectories and periodic fluctuations of water ripples) are matched, and static interference (such as bottle texture) is filtered. Conversely, when the post-processing switch is "No," the maximum-scoring subcategory of the detection result is returned. Next, the pre-processing judgment is entered. If pre-processing is required, the multiple time-series images are mapped to a standard coordinate system through cropping, rotation, and alignment operations. If preprocessing is not required, the ROI, grayscale, and area screening operations are performed directly. If the confidence evaluation value of the first candidate defect type meets the second confidence threshold, and the area similarity and grayscale similarity of the first candidate defect type in the multi-time series images meet the similarity threshold, the first candidate defect type is determined to be the first defect type. This allows analysis of the dynamic changes of the target object (such as foreign object displacement and water ripple fluctuations). After the inspection is completed, the minimum score is returned.

[0064] When the target detection mode is the moving target lock mode, if the confidence evaluation value of the first candidate defect type meets the third confidence threshold, and the motion displacement of the first candidate defect type in the multi-time series images is greater than or equal to the first displacement threshold, the first candidate defect type is determined to have passed verification. The third confidence threshold is the threshold at which, in the moving target lock mode, it is determined that the first candidate defect type meets the first defect type determination criteria.

[0065] Figure 4 This is a flow chart of a defect detection method in a moving target locking mode provided in an embodiment of the present application. Figure 3Similar, the difference is that if the confidence evaluation value of the first candidate defect type meets the third confidence threshold, the motion trajectory (such as the movement path of the foreign body) of the first candidate defect type is analyzed by calculating the motion displacement of the first candidate defect type. When the motion displacement (such as pixel distance) of the first candidate defect type in the multi-time series image is greater than or equal to the first displacement threshold, the first candidate defect type is determined to have passed the verification.

[0066] When the target detection mode is static target enhancement mode, if the confidence evaluation value of the first candidate defect type meets the fourth confidence threshold, and the motion displacement of the first candidate defect type in the multiple time-series images is less than or equal to the second displacement threshold, the first candidate defect type is determined to have passed verification. The fourth confidence threshold is the threshold at which, in static target enhancement mode, it can be determined that the first candidate defect type meets the first defect type determination criteria.

[0067] Figure 5 This is a flow chart of a defect detection method in a static target enhancement mode provided in an embodiment of the present application. Figure 3 and Figure 4 Similar, the difference is that if the confidence evaluation value of the first candidate defect type meets the fourth confidence threshold, the static features of the first candidate defect type, such as the area, grayscale and other parameters of a single frame (such as crack area ≥ 50 pixels, grayscale variance ≥ 20) are calculated, and the multi-frame displacement is calculated (such as adjacent frame displacement ≤ 1 pixel, verifying that the target is stationary), when the motion displacement of the first candidate defect type in multiple time-series images (such as pixel distance) is less than or equal to the second displacement threshold, it is determined that the first candidate defect type meets the threshold of the determination condition of the first defect type.

[0068] Since in defect detection, there may be situations where interference cannot be distinguished for conventional areas, scores, colors, etc., the embodiment of the present application can further determine the first defect type through a dual-path verification algorithm to improve the algorithm's anti-interference and accuracy.

[0069] Figure 6 FIG. 1 is a flow chart of a defect detection method provided in another embodiment of the present application. Figure 6 As shown, the defect detection method may further include:

[0070] Step 105 : Based on the continuity test of the motion trajectory and the interference elimination of the morphological similarity, the first defect type is filtered to obtain the third defect type of the target object.

[0071] Motion trajectory continuity verification targets dynamic inspection scenarios, analyzing whether the motion trajectory of a defect type in consecutive image frames conforms to physical laws. If not, it is identified as interference. Morphological similarity compares the features of the detected first defect type with pre-set interference patterns. By calculating similarity thresholds (such as contour matching and texture entropy), false defects that are highly similar to the interference patterns are filtered out. Therefore, the third defect type is the final defect type obtained after filtering the first defect type through interference determination.

[0072] The embodiment of the present application constructs an intelligent anti-interference system in dynamic industrial scenarios through a combined strategy of "motion trajectory continuity + morphological similarity". It not only utilizes the trajectory rules in the time dimension to distinguish between real defects and dynamic interference, but also eliminates static pseudo-defects through the morphological characteristics of the spatial dimension, thereby improving the accuracy of defect type determination and reducing the cost of manual re-inspection and production misjudgment losses.

[0073] In step 105, the motion trajectory-based defect continuity test includes two parameters: one is the number of occurrences of the first defect type in the multi-time sequence images. This counts the frequency of the first defect type in the multi-time sequence images to reduce the possibility of occasional interference (such as single-frame noise) being mistakenly identified as a real defect. The other is the maximum displacement between adjacent frames in the multi-time sequence images. This calculation calculates the maximum offset between the positions of the first defect type in adjacent frames and verifies whether it matches the target object's motion speed (for example, when the assembly line is in uniform motion, the displacement should be positively correlated with the speed). Only when this threshold is exceeded can the first defect type be considered continuous and meet the defect characteristics.

[0074] Therefore, we can first obtain the number of occurrences of the first defect type in multiple time-series images and the maximum displacement of adjacent frames in the multiple time-series images. For example, based on the number of multiple time-series images, we can determine the search range on the adjacent frame time-series images, which is generally the current image and the frame before and after it [n-1, n+1]. In some special cases, it can be expanded to the range of two frames before and after [n-2, n+2], where n represents the sequence number of the current image in the time sequence. Then, define the maximum displacement of the same target on adjacent images, where the target refers to the detection target on the target object. This parameter limits the upper limit of the displacement of the same target. Then, define the number of times the same target appears in multiple time-series images. This parameter defines the minimum number of times the target is detected on the time sequence diagram.

[0075] Then, the morphological defect similarity and grayscale similarity of the first defect type in adjacent frames are obtained, and the morphological similarity of the first defect type in adjacent frames is determined based on the morphological defect similarity and grayscale similarity.

[0076] Morphological defect similarity refers to comparing the outline and shape parameters (such as area, perimeter, and aspect ratio) of the first defect type in adjacent frames to eliminate pseudo-defects caused by deformation (such as the change in shape of liquid stains as the workpiece moves). For example, morphological defect similarity can include the consistency of parameters such as area, length, and width. Grayscale similarity refers to analyzing the consistency of the grayscale distribution of the defect area (such as histogram matching and mean-variance), filtering out grayscale anomalies caused by sudden changes in illumination (such as reflective spots). For example, grayscale similarity can include the consistency of average grayscale, maximum grayscale, and minimum grayscale. The two similarities are calculated in a similar manner. Different feature parameters are assigned different weights. All features are weighted averaged to obtain the final morphological similarity parameter. Morphological similarity is expressed as a percentage, with 100% indicating the highest similarity.

[0077] If the occurrence count of the first defect type is greater than or equal to the occurrence count threshold, and the maximum displacement is less than or equal to the maximum displacement threshold, the first defect type is determined to meet the motion trajectory continuity requirement. The occurrence count threshold and the maximum displacement threshold are thresholds for determining whether the first defect type meets the motion trajectory continuity requirement.

[0078] For a time sequence image n in a multi-frame time series, targets with similar features (including morphological and grayscale similarity) are searched within the search range of its adjacent time sequence images. Next, the displacement between the similar target in the adjacent time sequence image and the target in the current image is determined to be within a maximum displacement threshold. After completing this screening for all time sequence images, the number of similar targets for each target is counted. If the number of similar targets exceeds the number of occurrences, the current target and its similar targets are considered to form a complete motion trajectory.

[0079] If the morphological similarity is greater than or equal to the similarity threshold, the first defect type is determined to meet the morphological similarity criteria. The similarity threshold determines whether the first defect type meets the morphological similarity criteria. Interference exclusion based on morphological similarity can be performed within a time series of adjacent frames. Defect similarity is determined using the algorithm's built-in area similarity threshold and color distribution similarity methods. Defects that meet the threshold are considered the third defect type. This effectively reduces interference such as water ripples.

[0080] Finally, the first defect type that meets the motion trajectory continuity and morphological similarity conditions is determined as the third defect type. In this way, through the dual-path verification algorithm of "motion trajectory continuity verification" and "morphological similarity interference elimination" two technical paths, the detection results are cross-validated. Based on the dual constraints of spatiotemporal and visual features, the recognition accuracy of defect detection can be improved. At the same time, the motion trajectory curve and morphological evolution map can be output, providing a quantitative basis for process optimization and equipment maintenance.

[0081] Figure 7FIG. 7 is a schematic diagram of the structure of a defect detection system 700 provided in an embodiment of the present application. Figure 7 As shown, the defect detection system 700 may include a memory 701 and a processor 702. The memory 701 is configured to store instructions. The processor 702 is configured to call instructions from the memory 701 and implement the above-mentioned defect detection method when executing the instructions.

[0082] An embodiment of the present application further provides a computer-readable storage medium, in which a program is stored. The program can be loaded by a processor and execute any one of the defect detection methods in the embodiments of the present application.

[0083] Since the instructions stored in the defect detection system and the computer-readable storage medium can execute the steps in any defect detection method provided in the embodiments of the present application, the beneficial effects that can be achieved by any defect detection method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0084] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer program. When all or part of the functions in the above embodiments are implemented by computer program, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above functions can be implemented. In addition, when all or part of the functions in the above embodiments are implemented by computer program, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and saved in the memory of the local device by downloading or copying, or the system of the local device is updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be implemented.

[0085] The above specific examples are used to illustrate the present application, which is only used to help understand the present application and is not intended to limit the present application. For those skilled in the art of the present application, based on the concept of the present application, they can also make some simple deductions, modifications or substitutions.

Claims

1. A defect detection method, characterized in that: include: Acquire multiple time-series images of the target object; Performing defect detection on the multiple time-series images to obtain a first candidate defect type of the target object; Selecting a target detection mode that matches the first candidate defect type from a plurality of pre-set detection modes to verify the first candidate defect type, wherein the plurality of detection modes include a dynamic detection mode and a static detection mode. The dynamic detection mode is suitable for a motion scene where defect characteristics change over time, and the static detection mode is suitable for a scene where defect characteristics are stable. The first candidate defect type that passes the verification is determined as the first defect type of the target object.

2. The defect detection method according to claim 1, characterized in that: The acquiring of multiple time-series images of the target object includes: Acquire images of the target object at multiple time points captured by at least two image acquisition devices to obtain multiple time-series images of the target object; The image range captured by the at least two image capture devices covers the target capture area of ​​the target object.

3. The defect detection method according to claim 1, wherein: The selecting a target detection mode matching the first candidate defect type from a plurality of pre-set detection modes to verify the first candidate defect type includes: Determining a target detection mode that matches each first candidate defect type based on a pre-built defect feature knowledge base, wherein the defect feature knowledge base is built based on a mapping relationship between defect types and detection modes; The first candidate defect type is verified using each target detection mode respectively.

4. The defect detection method according to claim 3, characterized in that: Verifying the first candidate defect type using each target detection mode includes: Clustering the first candidate defect types corresponding to each target detection mode respectively to obtain second candidate defect types corresponding to the target detection mode; Verifying the second candidate defect type, and determining the second candidate defect type that passes the verification as the second defect type; The first candidate defect type included in the second defect type is verified.

5. The defect detection method according to claim 4, characterized in that: The verifying the first candidate defect type included in the second defect type includes: Performing multi-dimensional feature screening on the first candidate defect type to obtain multi-dimensional confidence evaluation values; Obtaining a confidence evaluation value of the first candidate defect type based on the multi-dimensional confidence evaluation values; Based on the confidence evaluation value, the first candidate defect type is verified.

6. The defect detection method according to claim 5, characterized in that: The target detection mode includes a global static detection mode, a global dynamic detection mode, a moving target locking mode, and a static target enhancement mode. The verifying of the first candidate defect type based on the confidence evaluation value includes: When the target detection mode is the global static detection mode, if the confidence evaluation value of the first candidate defect type meets a first confidence threshold, determining that the first candidate defect type passes verification; When the target detection mode is the global dynamic detection mode, if the confidence evaluation value of the first candidate defect type satisfies a second confidence threshold, and the area similarity and grayscale similarity of the first candidate defect type in the multi-time series images meet a similarity threshold, then it is determined that the first candidate defect type passes verification; When the target detection mode is the moving target locking mode, if the confidence evaluation value of the first candidate defect type meets a third confidence threshold, and the motion displacement of the first candidate defect type in the multi-time series images is greater than or equal to a first displacement threshold, then it is determined that the first candidate defect type passes verification; When the target detection mode is the static target enhancement mode, if the confidence evaluation value of the first candidate defect type meets the fourth confidence threshold, and the motion displacement of the first candidate defect type in the multi-time series image is less than or equal to the second displacement threshold, then the first candidate defect type is determined to have passed the verification.

7. The defect detection method according to claim 4, characterized in that: The verifying the first candidate defect type included in the second defect type further includes: Performing region screening on the first region where the second defect type is located to obtain a second region where the first candidate defect type is located, where the second region is included in the first region.

8. The defect detection method according to any one of claims 1 to 7, characterized in that: Also includes: Based on the continuity test of the motion trajectory and the interference elimination of the morphological similarity, the first defect type is filtered to obtain the third defect type of the target object.

9. The defect detection method according to claim 8, characterized in that: The first defect type is filtered based on the continuity check of the motion trajectory and the interference elimination of the morphological similarity to obtain the third defect type of the target object, including: Obtaining the number of occurrences of the first defect type in the multiple time-series images and the maximum displacement of adjacent frames in the multiple time-series images; If the number of occurrences of the first defect type is greater than or equal to the occurrence number threshold, and the maximum displacement is less than or equal to the maximum displacement threshold, then it is determined that the first defect type satisfies the motion trajectory continuity requirement; Acquire morphological defect similarity and grayscale similarity of the first defect type in the adjacent frames, and determine morphological similarity of the first defect type in the adjacent frames based on the morphological defect similarity and the grayscale similarity; If the morphological similarity is greater than or equal to a similarity threshold, determining that the first defect type meets the morphological similarity condition; The first defect type that meets the motion trajectory continuity and the morphological similarity conditions is determined as the third defect type.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, which can be loaded by a processor to execute the defect detection method according to any one of claims 1 to 9.

Images