Defect Detection Method, Device, Non-Volatile Storage Medium and Electronic Device

Abstract

The present application discloses a defect detection method, apparatus, non-volatile storage medium, and electronic device. Among them, the method includes: obtaining the detection performance of a target class of objects, where the detection performance is obtained by detecting the target class of objects using an initial detection model; determining target features that need to be data-augmented based on the detection performance and the environmental parameters of the target class of objects; adjusting the initial detection model using the data augmentation strategy corresponding to the target features to obtain a target detection model; and performing defect detection on the target class of objects using the target detection model to obtain a target detection result. The present application solves the technical problem of unsatisfactory product quality detection effect in the related art.

Description

Technical Field

[0001] The present application relates to the fields of industrial automation and computer vision, and in particular, to a defect detection method, device, non-volatile storage medium, and electronic device. Background Art

[0002] In the field of industrial quality inspection, the quality inspection of products mainly relies on image processing and object detection algorithms. Facing challenges such as complex and diverse product appearances, different shapes, and subtle defects, deep learning models have become the core of automated inspection systems. However, industrial quality inspection image data often has uneven distribution, and combined with complex situations such as lighting changes and product surface reflections, the difficulty of quality inspection has been greatly improved. Current technical means are difficult to dynamically adapt to the changes of different products and quality inspection environments, resulting in poor performance of the quality inspection system in actual applications, especially when detecting tiny defects or flaws. Most existing image enhancement methods adopt preset static strategies and lack the ability to dynamically adjust according to the specific requirements of industrial quality inspection, thus affecting the effect of product quality inspection. There are technical problems in the related art that the effect of product quality inspection is not ideal.

[0003] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0004] Embodiments of the present application provide a defect detection method, device, non-volatile storage medium, and electronic device, so as to at least solve the technical problem that the effect of product quality inspection in the related art is not ideal.

[0005] According to one aspect of the embodiments of the present application, a defect detection method is provided, including: obtaining the detection performance of a target class of objects, where the detection performance is obtained by detecting the target class of objects using an initial detection model; determining a target feature that needs to be data-enhanced based on the detection performance and the environmental parameters of the target class of objects; adjusting the initial detection model using the data enhancement strategy corresponding to the target feature to obtain a target detection model; and performing defect detection on the target class of objects using the target detection model to obtain a target detection result.

[0006] Optionally, the determining a target feature that needs to be data-enhanced based on the detection performance and the environmental parameters of the target class of objects includes: determining a plurality of predetermined surface features for which the target class of objects needs to be defect-detected based on the environmental parameters and the object material of the target class of objects; and determining the target feature from the plurality of predetermined surface features based on the detection performance.

[0007] Optionally, the multiple predetermined surface features respectively correspond to predetermined defect patterns. Determining the target feature from the multiple predetermined surface features based on the detection performance includes: obtaining reference defect data that matches the object type of the target class of objects; determining the target defect pattern existing in the target class of objects based on the reference defect data and the detection performance; and determining the target feature that matches the target defect pattern from the multiple predetermined surface features.

[0008] Optionally, adjusting the initial detection model by using the data augmentation strategy corresponding to the target feature to obtain a target detection model includes: determining the types of parameters to be adjusted in the initial detection model based on the recognition requirements of the target feature; and adjusting the parameter settings of the types of parameters in the initial detection model by using the data augmentation strategy to obtain the target detection model.

[0009] Optionally, adjusting the parameter settings of the types of parameters in the initial detection model by using the data augmentation strategy to obtain the target detection model includes: determining the feature channels associated with the target feature in the initial detection model; and adjusting the initial weights of the feature channels according to the weight adjustment method indicated by the data augmentation strategy to obtain the target detection model.

[0010] Optionally, the method further includes: obtaining change information of the environmental parameters; allowing the environmental parameters to be updated to changed parameters when the change information is greater than a predetermined environmental change threshold; and determining to perform adjustment processing on the initial detection model in response to generating the changed parameters.

[0011] Optionally, the method further includes: determining the types of objects allowed to be produced by the target production line; and determining the initial detection model from a plurality of predetermined candidate models based on the types of objects, where the plurality of candidate models have different defect detection capabilities for different types of objects.

[0012] According to another aspect of the embodiments of the present application, a defect detection device is provided, including: a detection performance acquisition module, configured to acquire the detection performance of a target class of objects, where the detection performance is obtained by detecting the target class of objects by using an initial detection model; a target feature determination module, configured to determine a target feature that needs to be data-augmented based on the detection performance and the environmental parameters of the target class of objects; a detection model adjustment module, configured to adjust the initial detection model by using the data augmentation strategy corresponding to the target feature to obtain a target detection model; and a detection result determination module, configured to perform defect detection on the target class of objects by using the target detection model to obtain a target detection result.

[0013] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided. The non-volatile storage medium stores multiple instructions, and the instructions are adapted to be loaded and executed by a processor to perform any one of the defect detection methods.

[0014] According to another aspect of the embodiments of the present application, an electronic device is provided, including: one or more processors and a memory. The memory is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement any one of the defect detection methods.

[0015] In the embodiments of the present application, by obtaining the detection performance of a target class of objects, where the detection performance is obtained by detecting the target class of objects using an initial detection model; based on the detection performance and the environmental parameters of the target class of objects, determining target features that need to be data-augmented; using the data augmentation strategy corresponding to the target features to adjust the initial detection model to obtain a target detection model; and using the target detection model to perform defect detection on the target class of objects to obtain a target detection result. The purpose of improving the adaptability of the detection model is achieved, the technical effect of improving the product detection accuracy is realized, and thus the technical problem of unsatisfactory product quality detection effect in the related art is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0017] Figure 1 is a flowchart of an optional defect detection method provided according to the embodiments of the present application;

[0018] Figure 2 is a schematic diagram of an optional defect detection method provided according to the embodiments of the present application;

[0019] Figure 3 is a schematic diagram of an optional defect detection device provided according to the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] In order to enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0021] It should be noted that the terms "first", "second", etc. in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products or devices.

[0022] For the convenience of description, some nouns or terms related to the embodiments of this application are described below:

[0023] RGB (Red Green Blue) is a color model based on the three primary colors (red, green, and blue). Red represents the intensity of the red component in the color model, and its range is usually from 0 to 255. Green represents the intensity of the green component in the color model, and its range is usually from 0 to 255. Blue represents the intensity of the blue component in the color model, and its range is usually from 0 to 255. In the RGB color model, by adjusting the intensities of these three components, more than 16 million different colors can be generated.

[0024] HSV (Hue Saturation Value) is a color model based on human visual perception. Hue represents the basic attribute of a color, that is, the type of color. The value of hue is usually between 0 and 360 degrees, representing a color circle that starts from red and rotates clockwise. Saturation represents the purity of a color, that is, the gray component contained in the color. The value range of saturation is usually from 0% to 100%, where 0% represents gray and 100% represents the purest color. Value represents the brightness of a color, that is, the light and dark degree of the color. The value range of value is usually from 0% to 100%, where 0% represents black and 100% represents white.

[0025] Convolutional Neural Networks (CNN) is a deep learning model mainly used to process data with grid structures, such as images. By simulating the processing method of the human visual system, CNN can automatically learn the features and patterns in images.

[0026] The convolutional layer is the core part of CNN and is responsible for extracting the features of the input data. The convolutional layer contains multiple convolutional kernels or filters, and each convolutional kernel is responsible for detecting specific features. The convolutional kernel slides on the input data, calculates the weighted sum of the local area, and generates a feature map.

[0027] The Spatial Pyramid Pooling (SPP) layer is a technique used in convolutional neural networks to handle changes in image size. The main purpose of the SPP layer is to enable the network to process input images of different sizes while keeping the size of the output feature map unchanged.

[0028] According to the embodiments of the present application, an embodiment of a method for defect detection is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0029] Figure 1 is a flowchart of an optional defect detection method provided according to the embodiments of the present application, as Figure 1 shown, the method includes the following steps:

[0030] Step S102, obtain the detection performance of the target class of objects, where the detection performance is obtained by using an initial detection model to detect the target class of objects;

[0031] It can be understood that obtaining the detection performance of a specific target class of objects, such as metal parts, electronic components, etc., the detection performance is obtained by applying an initial detection model to detect the defects of the target class of objects. Obtaining the detection performance helps to identify the weaknesses of the model, especially the deficiencies in the recognition of certain specific defects, and deeply understand the performance of the model when processing specific types of objects. Furthermore, the model can be optimized targeted to ensure that the model can maintain a high detection accuracy in various quality inspection environments, effectively improving the efficiency and reliability of industrial quality inspection.

[0032] In an optional embodiment, the method further includes: determining the types of objects allowed to be produced on the target production line; based on the types of objects, determining an initial detection model among a plurality of predetermined candidate models, where the plurality of candidate models have different defect detection capabilities for different types of objects.

[0033] It can be understood that the types of objects allowed to be produced on the target production line are determined, and the most suitable initial model for defect detection of the target class of objects is selected from a plurality of preset candidate models. Each candidate model is optimized and upgraded for a specific type of object. Therefore, there are differences in their detection capabilities when dealing with products of different materials, shapes or surface features. The model selection mechanism based on the object type ensures that the most suitable detection model for the products on the target production line has been selected before starting the quality inspection process, thus maximizing the accuracy and efficiency of detection.

[0034] Optionally, there can be multiple types of the above-mentioned object types. For example, a production line may mainly produce plastic parts. The shapes, sizes and designs of plastic parts will vary, and they share certain attributes and potential defect types of plastic materials, such as cracks, scratches, air bubbles, etc. Therefore, a group of candidate models specifically trained for plastic materials are pre-trained. Each model is optimized for the common defects of different types of plastic parts. For example, one model may be particularly good at detecting surface cracks, while another model may perform better in identifying internal air bubbles. Selecting the most suitable initial detection model from this group of candidate models according to the type of plastic parts currently or planned to be produced on the production line can avoid separate model training for each specific product shape and reduce costs. At the same time, since the model has been optimized for the defects of plastic materials, its online upgrade and adjustment are more convenient, reducing the online upgrade cost caused by the model's inadaptability to the material or defect type. The model screening mechanism based on the production line material and defect type not only improves the economy of model online upgrade, but also ensures the high precision and robustness of the model when dealing with specific products, providing a more efficient solution for industrial automation quality inspection.

[0035] Step S104: Determine the target features that need to be data-augmented based on the detection performance and the environmental parameters of the target class of objects.

[0036] It can be understood that in-depth analysis is carried out based on the detection performance data and the specific environmental parameters of the target class of objects to determine the target features that need further data augmentation. The target features at least include the texture, brightness distribution, reflectivity, etc. of the object surface. The environmental parameters cover lighting conditions, shooting angles, background complexity, etc. By adding environmental parameters, the key factors causing the decline in detection performance can be identified, providing accurate guidance for subsequent data augmentation strategies.

[0037] Optionally, if the detection performance analysis shows that the model has a high false negative rate in a highly reflective environment, determine the reflective feature of the object surface as the target feature and enhance the performance of this feature in the data accordingly. For example, increase the simulation of local area reflection or use special image processing techniques to reduce the impact of reflection. The detection ability of the model in a highly reflective environment will be significantly improved, reducing false positives and false negatives, thereby improving the overall detection efficiency and accuracy. This feature enhancement method based on performance analysis and environmental parameters makes the data enhancement process more scientific and targeted.

[0038] In an alternative embodiment, based on the detection performance and the environmental parameters of the target object, determine the target features that need to be enhanced, including: based on the environmental parameters and the material of the target object, determine multiple predetermined surface features for which the target object needs to be defect-detected; based on the detection performance, determine the target features from the multiple predetermined surface features.

[0039] It can be understood that according to the specific environmental parameters of the target object and the material properties of the object, a series of predetermined surface features closely related to defect detection are determined. The strategy of determining the data enhancement target features based on environmental parameters and detection performance can accurately identify and solve the main problems encountered by the model in specific detection scenarios, thereby significantly improving the detection accuracy and robustness of the model.

[0040] Optionally, based on the performance feedback when the model detects the target object, screen out the key features that cause the detection performance to decline from the preselected surface features as the target features for targeted data enhancement. For example, if the detection accuracy of the model significantly decreases when processing plastic parts with complex surface textures, "surface texture enhancement" will be used as the target feature, and by increasing texture details or simulating texture performance under different lighting conditions, the recognition ability of the model can be improved.

[0041] Optionally, the above multiple predetermined surface features at least include the clarity of the surface texture, the consistency of the color, the sharpness of the edge, the reflectivity, the transparency, the surface smoothness, etc.

[0042] In an alternative embodiment, the multiple predetermined surface features respectively correspond to predetermined defect patterns. Based on the detection performance, determine the target features from the multiple predetermined surface features, including: obtain reference defect data matching the object type of the target object; based on the reference defect data and the detection performance, determine the target defect pattern existing in the target object; among the multiple predetermined surface features, determine the target features matching the target defect pattern.

[0043] It can be understood that each predetermined surface feature is associated with one or more specific defect patterns. Reference defect data matching the object type of the target class object is obtained, and the data provides detailed information about the types and distribution patterns of defects that the class of objects may encounter. Then, combined with the performance of the model in the current detection task, that is, the detection performance, the target defect pattern existing in the target class object is identified. Finally, among the set of predetermined surface features, the features that match the target defect pattern are selected as the target features. Based on the data augmentation strategy of the defect pattern, the precise positioning and efficient implementation of data augmentation are achieved, improving the detection accuracy of the model for specific types of defects.

[0044] Optionally, each predetermined surface feature is associated with one or more specific defect patterns. For example, the clarity of the surface texture may be related to the detection of cracks or scratches, while the lighting conditions may affect the recognition of micro depressions or protrusions. If the model performs poorly in identifying micro cracks on plastic parts, the surface texture features related to crack detection will be enhanced, that is, the contrast and clarity of the texture details will be increased.

[0045] Optionally, there can be multiple defect features and their corresponding data augmentation strategies and model adjustment methods. For example, micro defect features such as scratches, small holes, and slight deformations. The corresponding data augmentation strategies should be local magnification, high-contrast enhancement, and detail enhancement. The model adjustment method is to increase the local detail perception ability of the model, such as enhancing the local feature extraction ability of the model by increasing the number of convolutional layers or changing the size of the convolutional kernel. At the same time, the loss function of the model can be adjusted to increase the weight for micro defect detection, prompting the model to pay more attention to these detail features.

[0046] Optionally, defect features sensitive to lighting changes, such as reflections, shadows, and highlights. The corresponding data augmentation strategies are lighting randomization, shadow generation, and highlight simulation. The model adjustment method can consider adding a lighting normalization layer or an adaptive lighting correction module to enable the model to better process inputs under different lighting conditions. At the same time, adjust the model's strategy, such as using online data augmentation, to ensure that the model can see samples with lighting changes every time it is upgraded, improving the model's robustness to lighting changes.

[0047] Optionally, defect features with complex textures, such as surface defects and uneven textures. The corresponding data augmentation strategies are texture random generation, color jitter, and contrast adjustment. The model adjustment method is to add a convolutional network module for texture recognition, such as using the Inception module or Dense module with strong texture recognition ability, to enhance the model's perception ability of complex textures. At the same time, adjust the weight learning rate of the texture feature extraction layer of the model to ensure that texture features can be extracted more accurately.

[0048] Optionally, defect features sensitive to position changes, such as edge cracks and labeling position deviations. The corresponding data augmentation strategies are random cropping, position offset, and rotation. The model adjustment method is to enhance the position invariance of the model, such as using a position-sensitive loss function or introducing a spatial pyramid pooling layer, so that the model can stably identify the same defect at different positions. In addition, adjust the input size of the model and the resolution of the output feature map to adapt to defect features at different positions.

[0049] Optionally, defect features sensitive to color changes, such as color difference, fading, and color change. The corresponding data augmentation strategies are color space transformation, brightness adjustment, and hue randomization. The model adjustment method is to add a color space conversion layer so that the model can process inputs in different color spaces, such as converting from RGB (Red Green Blue) to HSV (Hue Saturation Value), thereby enhancing color sensitivity. Adjust the weights of the color feature extraction layer in the model to ensure that color changes can be more accurately identified.

[0050] Optionally, defect features sensitive to shape changes, such as deformation and unevenness. The corresponding data augmentation strategies are geometric transformations (such as stretching and distortion) and morphological operations (such as erosion and dilation). The model adjustment method is to introduce a shape invariance mechanism, such as using a shape-invariant convolution kernel or adding a network branch for shape feature extraction. Adjust the geometric transformation parameters of the model, such as controlling the degree and direction of deformation, to optimize the model's ability to identify shape changes.

[0051] Optionally, defect features affected by background complexity, such as cluttered background and similar background. The data augmentation strategies are background randomization, adding random noise, and adjusting the contrast between the foreground and the background. The model adjustment method is to add a background suppression or attention mechanism so that the model can focus on the defect area and reduce background interference. In addition, a regional attention network module can be adopted so that the model can dynamically adjust the attention to different regions and enhance the adaptability to changes in background complexity.

[0052] In an optional embodiment, the method further includes: obtaining change information of environmental parameters; in the case where the change information is greater than a predetermined environmental change threshold, allowing the environmental parameters to be updated to changed parameters; in response to generating the changed parameters, determining to perform an adjustment process on the initial detection model.

[0053] It can be understood that the changes in environmental parameters are continuously detected, and the change information of the parameters is recorded. If the detected environmental change exceeds the predetermined environmental change threshold, that is, the environmental conditions have changed significantly, the system will automatically update the current environmental parameters and adjust them to the changed parameters reflecting the new environmental conditions. In response to the generation of the changed parameters, an adjustment process for the initial detection model is executed. By continuously detecting the changes in environmental parameters and automatically adjusting the detection model, the environmental adaptability and performance stability of the model are improved. It is ensured that even when conditions such as light and background change, the detection model can still maintain high precision and robustness, reducing the detection errors caused by environmental changes and improving the quality control level of the product.

[0054] Optionally, the environmental parameters change in real time, but small changes are not expected to trigger model updates until there are significant environmental changes. Therefore, an intelligent response mechanism for environmental parameter changes is designed to balance the frequency of model updates and resource consumption, thereby reducing the overhead of model upgrades. Continuously detect the changes in environmental parameters in real time, such as light intensity, angle, background complexity, etc. Set a predetermined environmental change threshold, and only when the change in environmental parameters exceeds this threshold, that is, the environmental conditions have changed substantially, will the model adjustment process be initiated. The setting of the threshold is based on an in-depth understanding of the industrial quality inspection environment and considerations of the model update cost. The size of the threshold can be flexibly adjusted to adapt to the change speed of different quality inspection environments and the requirements of model update frequencies. For example, in a quality inspection environment with frequent light changes, the threshold may be set lower to adjust the model more frequently; while in a relatively stable environment, the threshold can be set higher to reduce unnecessary model updates.

[0055] Step S106, adopt a data augmentation strategy corresponding to the target features to adjust the initial detection model to obtain a target detection model;

[0056] It can be understood that by adopting a data augmentation strategy matching the target features, the initial detection model is specifically adjusted and optimized to obtain a target detection model that is more suitable for specific quality inspection requirements, enabling the model to have stronger adaptability and generalization ability when facing the actual industrial quality inspection environment.

[0057] In an optional embodiment, adopting a data augmentation strategy corresponding to the target features to adjust the initial detection model to obtain a target detection model includes: determining the types of parameters to be adjusted in the initial detection model based on the recognition requirements of the target features; adopting a data augmentation strategy to adjust the parameter settings of the parameter types in the initial detection model to obtain a target detection model.

[0058] It can be understood that based on the recognition requirements of the target feature, it is identified which parameter types in the initial detection model are most crucial for improving the detection effect of this feature. A data augmentation strategy matching the target feature is adopted to make targeted adjustments to the identified parameter types, resulting in a target detection model. By directly adjusting the parameter types directly related to the target feature recognition in the model, the efficient optimization of the model is achieved, avoiding the waste of computing resources caused by making overall adjustments to the model.

[0059] In an optional embodiment, a data augmentation strategy is adopted to adjust the parameter settings of the parameter types in the initial detection model to obtain a target detection model, including: determining the feature channels associated with the target feature in the initial detection model; adjusting the initial weights of the feature channels according to the weight adjustment method indicated by the data augmentation strategy to obtain the target detection model.

[0060] It can be understood that by determining the feature channels associated with the target feature in the initial detection model and adjusting the initial weights of the feature channels according to the weights indicated by the data augmentation strategy to obtain a target detection model, the generalization ability and accuracy of the model are improved through the enhancement strategy.

[0061] Step S108, using the target detection model, perform defect detection on the target class of objects to obtain a target detection result.

[0062] It can be understood that the target detection model performs in-depth analysis on the input quality inspection image data, and uses the data augmentation strategy to accurately locate and classify various defects in the image, such as cracks, scratches, missing parts, etc., and finally outputs a target detection result, including the location, type, and severity of the defects. The target model can effectively identify tiny defects, reduce the possible false detections and missed detections in quality inspection, improve the qualified rate of products, and at the same time speed up the quality inspection process and reduce the need for manual quality inspection.

[0063] Through the above step S102, the detection performance of the target class of objects is obtained, where the detection performance is obtained by using the initial detection model to detect the target class of objects; step S104, based on the detection performance and the environmental parameters of the target class of objects, determine the target feature that needs to be data-augmented; step S106, adopt the data augmentation strategy corresponding to the target feature to adjust the initial detection model to obtain a target detection model; step S108, use the target detection model to perform defect detection on the target class of objects to obtain a target detection result, which can achieve the purpose of improving the adaptability of the detection model, achieve the technical effect of improving the product detection accuracy, and further solve the technical problem of unsatisfactory product quality detection effect in the related art.

[0064] Based on the above embodiments and optional embodiments, the present application proposes an optional implementation manner Figure 2is a schematic diagram of an optional defect detection method provided according to an embodiment of the present application, such as Figure 2 As shown in the figure, the data input module, data feature analysis module, enhancement strategy generation module, detection adaptive enhancement strategy module, training status detection module, and enhancement effect evaluation module, etc., are demonstrated, showing the division of labor and cooperation between different modules.

[0065] The data feature analysis module is specifically used to analyze the image data features in industrial quality inspection, including product surface reflection, texture complexity, color distribution, etc. By analyzing these features, factors that may cause detection errors can be identified and enhancement strategies can be dynamically adjusted.

[0066] The detection performance of specific target objects, such as metal parts and electronic components, is obtained by applying an initial detection model to detect defects in the target objects. The detection performance acquisition helps to identify the weaknesses of the model, especially the deficiencies in the recognition of certain specific defects, and gain an in-depth understanding of the performance of the model when processing specific types of objects, so as to optimize the model in a targeted manner, ensuring that the model can maintain high detection accuracy in various quality inspection environments, effectively improving the efficiency and reliability of industrial quality inspection.

[0067] Based on the detection performance data and the specific environmental parameters of the target object, in-depth analysis is performed to determine the target features that require further data enhancement. The target features include at least the texture, brightness distribution, and reflectivity of the object surface. Environmental parameters cover lighting conditions, shooting angles, background complexity, etc. By adding environmental parameters, the key factors that lead to reduced detection performance can be identified, providing precise guidance for subsequent data enhancement strategies.

[0068] According to the specific environmental parameters of the target object and the material properties of the object, a series of predetermined surface features closely related to defect detection are determined. The strategy of determining data enhancement target features based on environmental parameters and detection performance can accurately identify and solve the main problems encountered by the model in specific detection scenarios, thereby significantly improving the detection accuracy and robustness of the model.

[0069] The model status detection module is used to detect the upgrade process of the quality inspection model, including training loss (retraining loss during the upgrade process), detection accuracy, missed detection rate and other indicators. This module can automatically optimize the data enhancement strategy based on the real-time performance of the model. For example, if the missed detection rate of minor defects in a certain type of product is high, the system will enhance the local magnification and high contrast processing of this type of product.

[0070] If the detection performance analysis shows that the missed detection rate of the model is relatively high in a highly reflective environment, determine the reflective characteristics of the object surface as the target characteristics, and specifically enhance the performance of this characteristic in the data. For example, increase the simulation of local area reflection or use special image processing techniques to reduce the impact of reflection. The detection ability of the model in a highly reflective environment will be significantly improved, reducing false alarms and missed detections, thereby improving the overall detection efficiency and accuracy. This feature enhancement method based on performance analysis and environmental parameters makes the data enhancement process more scientific and targeted.

[0071] Continuously detect changes in environmental parameters and record the change information of the parameters. If the detected environmental change exceeds the predetermined environmental change threshold, that is, the environmental conditions have changed significantly, the system will automatically update the current environmental parameters and adjust them to the changed parameters reflecting the new environmental conditions. In response to generating the changed parameters, perform an adjustment process on the initial detection model. By detecting environmental parameter changes in real time and automatically adjusting the detection model, the environmental adaptability and performance stability of the model are improved. Ensure that even when conditions such as lighting and background change, the detection model can still maintain high precision and robustness, reducing detection errors caused by environmental changes and improving the product quality control level.

[0072] The adaptive enhancement strategy module covers a variety of enhancement methods, such as adding noise to local areas, adjusting lighting, and processing surface reflection. The system will flexibly adjust the enhancement methods and their intensities according to the surface characteristics, shape, and lighting conditions of the product to ensure that the enhancement process is more targeted and efficient.

[0073] Tiny defect features, such as scratches, small holes, and slight deformations. The corresponding data enhancement strategy should be local magnification, high-contrast enhancement, and detail enhancement. The model adjustment method is to increase the local detail perception ability of the model. For example, enhance the local feature extraction ability of the model by increasing the number of convolutional layers or changing the size of the convolutional kernel. At the same time, the loss function of the model can be adjusted to increase the weight of tiny defect detection, prompting the model to pay more attention to these detail features.

[0074] Defect features sensitive to lighting changes, such as reflection, shadow, and highlight. The corresponding data enhancement strategy is lighting randomization, shadow generation, and highlight simulation. The model adjustment method can consider adding a lighting normalization layer or an adaptive lighting correction module to enable the model to better process inputs under different lighting conditions. At the same time, adjust the model upgrade strategy, such as using online data enhancement, to ensure that the model can see samples with lighting changes every time it is upgraded, improving the robustness of the model to lighting changes.

[0075] Defect features with complex textures, such as surface defects and uneven textures. The corresponding data augmentation strategies are random texture generation, color jittering, and contrast adjustment. The model adjustment method is to add a convolutional network module for texture recognition, such as using convolutional kernels of different sizes for feature extraction in the same layer and then concatenating the extracted features. This can enable the model to learn richer features at different scales to enhance its perception ability of complex textures. At the same time, adjust the weight learning rate of the texture feature extraction layer of the model to ensure that texture features can be extracted more accurately.

[0076] Defect features sensitive to position changes, such as edge cracks and label position deviations. The corresponding data augmentation strategies are random cropping, position offset, and rotation. The model adjustment method is to enhance the position invariance of the model, such as using a position-sensitive loss function or introducing a spatial pyramid pooling layer, so that the model can stably identify the same defect at different positions. In addition, adjust the input size of the model and the resolution of the output feature map to adapt to defect features at different positions.

[0077] Defect features sensitive to color, such as color difference, fading, and color change. The corresponding data augmentation strategies are color space transformation, brightness adjustment, and hue randomization. The model adjustment method is to add a color space conversion layer to enable the model to process inputs in different color spaces, such as converting from RGB (Red Green Blue) to HSV (Hue Saturation Value), thereby enhancing color sensitivity. Adjust the weights of the color feature extraction layer in the model to ensure that color changes can be recognized more precisely.

[0078] Defect features sensitive to shape changes, such as deformation and unevenness. The corresponding data augmentation strategies are geometric transformations (such as stretching and twisting) and morphological operations (such as erosion and dilation). The model adjustment method is to introduce a shape invariance mechanism, such as using a shape-invariant convolutional kernel or adding a network branch for shape feature extraction. Adjust the geometric transformation parameters of the model, such as controlling the degree and direction of deformation, to optimize the model's recognition ability for shape changes.

[0079] Defect features affected by background complexity, such as cluttered backgrounds and similar backgrounds. The data augmentation strategies are background randomization, adding random noise, and adjusting the contrast between the foreground and the background. The model adjustment method is to add a background suppression or attention mechanism to enable the model to focus on the defect area and reduce background interference. In addition, a regional attention network module can be used to enable the model to dynamically adjust its attention to different regions and enhance its adaptability to changes in background complexity.

[0080] The enhancement effect evaluation module compares the effects of different enhancement strategies through experiments and actual quality inspection data, and optimizes the enhancement parameter settings. This module can also dynamically evaluate the enhancement effect during the quality inspection process, make timely strategy adjustments, and then the model training / application module will execute after the adjustment.

[0081] The above optional implementation manners can at least achieve the following effects: for different products and quality inspection environments, it is possible to dynamically adjust the data enhancement strategy according to product characteristics, improving the accuracy of defect detection; introducing a variety of enhancement techniques, including local magnification, surface reflection processing, lighting adjustment, etc., to ensure that the enhancement method can better handle different defects in complex quality inspection scenarios; the adaptive data enhancement method effectively improves the detection accuracy of the model for tiny defects on the product surface, reduces the situation of missed detection and misdetection, has strong versatility, and can be applied to various types of industrial quality inspection tasks; in the case of relatively scarce data, the adaptive enhancement method can effectively expand the dataset, make the most of the existing quality inspection image data, dynamically adjust the data enhancement strategy, improve the data utilization rate and the generalization ability of the model.

[0082] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0083] In this embodiment, a defect detection device is also provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the terms "module" and "device" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0084] According to an embodiment of the present application, an apparatus embodiment for implementing the defect detection method is also provided. Figure 3 A schematic diagram of a defect detection device according to an embodiment of the present application, as Figure 3 shown, the above defect detection device includes a detection performance acquisition module 302, a target feature determination module 304, a detection model adjustment module 306, and a detection result determination module 308. The device will be described below.

[0085] The detection performance acquisition module 302 is used to acquire the detection performance of the target class object, where the detection performance is obtained by detecting the target class object using the initial detection model.

[0086] The target feature determination module 304, which is connected to the detection performance acquisition module 302, is configured to determine the target features that need to be data-augmented based on the detection performance and the environmental parameters of the target class of objects.

[0087] The detection model adjustment module 306, which is connected to the target feature determination module 304, is configured to adjust the initial detection model by using the data augmentation strategy corresponding to the target features to obtain the target detection model.

[0088] The detection result determination module 308, which is connected to the detection model adjustment module 306, is configured to perform defect detection on the target class of objects by using the target detection model to obtain the target detection result.

[0089] In a defect detection device provided by an embodiment of the present application, by setting the detection performance acquisition module 302 for acquiring the detection performance of the target class of objects, where the detection performance is obtained by detecting the target class of objects by using the initial detection model; the target feature determination module 304, which is connected to the detection performance acquisition module 302, is configured to determine the target features that need to be data-augmented based on the detection performance and the environmental parameters of the target class of objects; the detection model adjustment module 306, which is connected to the target feature determination module 304, is configured to adjust the initial detection model by using the data augmentation strategy corresponding to the target features to obtain the target detection model; the detection result determination module 308, which is connected to the detection model adjustment module 306, is configured to perform defect detection on the target class of objects by using the target detection model to obtain the target detection result. The purpose of improving the adaptability of the detection model is achieved, the technical effect of improving the product detection accuracy is realized, and further the technical problem of unsatisfactory product quality detection effect in the related art is solved.

[0090] It should be noted that the above-mentioned various modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following manner: the above-mentioned various modules can be located in the same processor; or, the above-mentioned various modules are located in different processors in any combination.

[0091] It should be noted here that the above-mentioned detection performance acquisition module 302, target feature determination module 304, detection model adjustment module 306, and detection result determination module 308 correspond to steps S102 to S108 in the embodiment. The examples and application scenarios implemented by the above-mentioned modules and the corresponding steps are the same, but are not limited to the content disclosed in the above-mentioned embodiment. It should be noted that the above-mentioned modules, as part of the device, can run in a computer terminal.

[0092] It should be noted that the optional or preferred implementation manners of this embodiment can refer to the relevant descriptions in the embodiment, and will not be repeated here.

[0093] The above defect detection device may further include a processor and a memory. The acquisition of detection performance module 302, the determination of target feature module 304, the adjustment of detection model module 306, the determination of detection result module 308, etc. are all stored in the memory as program units, and the corresponding functions are implemented by the processor executing the above program units stored in the memory.

[0094] The processor contains a kernel, and the kernel retrieves the corresponding program units from the memory. One or more kernels can be set. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one storage chip.

[0095] An embodiment of the present application provides a non-volatile storage medium, on which a program is stored, and the program implements a defect detection method when executed by a processor.

[0096] An embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program stored on the memory and executable on the processor. When the processor executes the program, the following steps are implemented: acquiring the detection performance of a target class of objects, where the detection performance is obtained by detecting the target class of objects using an initial detection model; based on the detection performance and the environmental parameters of the target class of objects, determining the target features that need to be data-augmented; using the data augmentation strategy corresponding to the target features to adjust the initial detection model to obtain a target detection model; using the target detection model to perform defect detection on the target class of objects to obtain a target detection result. The device in this article can be a server, a PC, etc.

[0097] The present application also provides a computer program product, which is adapted to execute a program initialized with the following method steps when executed on a data processing device: acquiring the detection performance of a target class of objects, where the detection performance is obtained by detecting the target class of objects using an initial detection model; based on the detection performance and the environmental parameters of the target class of objects, determining the target features that need to be data-augmented; using the data augmentation strategy corresponding to the target features to adjust the initial detection model to obtain a target detection model; using the target detection model to perform defect detection on the target class of objects to obtain a target detection result.

[0098] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0099] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0100] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0102] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0103] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0104] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0105] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0106] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, system, or computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0107] The above are only embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A defect detection method, characterized in that, Including: Obtain the detection performance of the target class of objects, where the detection performance is obtained by detecting the target class of objects using an initial detection model; Based on the detection performance and the environmental parameters of the target class of objects, determine the target features that need to be data-augmented; Use the data augmentation strategy corresponding to the target features to adjust the initial detection model to obtain a target detection model; Use the target detection model to perform defect detection on the target class of objects to obtain a target detection result; The method further includes: determining the types of objects allowed to be produced on the target production line; based on the types of objects, determining the initial detection model among a plurality of predetermined candidate models, where the plurality of candidate models have different defect detection capabilities for different types of objects.

2. The method according to claim 1, wherein The determining the target features that need to be data-augmented based on the detection performance and the environmental parameters of the target class of objects includes: Based on the environmental parameters and the object material of the target class of objects, determine a plurality of predetermined surface features for which the target class of objects needs to be defect-detected; Based on the detection performance, determine the target features among the plurality of predetermined surface features.

3. The method according to claim 2, wherein The plurality of predetermined surface features respectively correspond to predetermined defect patterns. The determining the target features among the plurality of predetermined surface features based on the detection performance includes: Obtain reference defect data that matches the type of the target class of objects; Based on the reference defect data and the detection performance, determine the target defect pattern existing in the target class of objects; Among the plurality of predetermined surface features, determine the target features that match the target defect pattern.

4. The method according to claim 1, characterized in that, The using the data augmentation strategy corresponding to the target features to adjust the initial detection model to obtain a target detection model includes: Based on the recognition requirements of the target features, determine the types of parameters in the initial detection model that need to be adjusted; Use the data augmentation strategy to adjust the parameter settings of the types of parameters in the initial detection model to obtain the target detection model.

5. The method according to claim 4, characterized in that, The using the data augmentation strategy to adjust the parameter settings of the types of parameters in the initial detection model to obtain the target detection model includes: Determine the feature channels in the initial detection model that are associated with the target features; According to the weight adjustment method indicated by the data augmentation strategy, adjust the initial weights of the feature channels to obtain the target detection model.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain the change information of the environmental parameters; When the change information is greater than a predetermined environmental change threshold, allow the environmental parameters to be updated to changed parameters; In response to generating the changed parameters, determine to perform the adjustment process on the initial detection model.

7. A defect detection device, characterized in that, Including: An acquisition detection performance module, configured to acquire the detection performance of the target class of objects, where the detection performance is obtained by detecting the target class of objects using an initial detection model; A target feature determination module, configured to determine a target feature that needs data augmentation based on the detection performance and the environmental parameters of the target class of objects; An adjusted detection model module, configured to adjust the initial detection model by using the data augmentation strategy corresponding to the target feature to obtain a target detection model; A detection result determination module, configured to perform defect detection on the target class of objects by using the target detection model to obtain a target detection result; The apparatus is further configured to determine the types of objects allowed to be produced on the target production line; and determine the initial detection model from a plurality of predetermined candidate models based on the types of objects, wherein the plurality of candidate models have different defect detection capabilities for different types of objects.

8. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a plurality of instructions, and the instructions are adapted to be loaded and executed by a processor to perform the defect detection method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, Comprising: One or more processors and a memory, where the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the defect detection method according to any one of claims 1 to 6.

Images