Defect detection method and device, nonvolatile storage medium and electronic equipment

Abstract

The invention discloses a defect detection method and device, a nonvolatile storage medium and electronic equipment. The method comprises the steps that the detection performance of a target type object is acquired, and the detection performance is obtained by detecting the target type object through an initial detection model; based on the detection performance and the parameters of the environment where the target object is located, target features needing data enhancement are determined; adjusting the initial detection model by adopting a data enhancement strategy corresponding to the target feature to obtain a target detection model; and performing defect detection on the target object by adopting the target detection model to obtain a target detection result. The technical problem that the product quality detection effect is not ideal in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the fields of industrial automation and computer vision, and more specifically, to a defect detection method, device, non-volatile storage medium, and electronic device. Background Art

[0002] In the field of industrial quality inspection, product quality inspection mainly relies on image processing and target detection algorithms. Faced with challenges such as complex and changeable product appearance, various shapes, and subtle defects, deep learning models have become the core of automated inspection systems. However, industrial quality inspection image data is often unevenly distributed, and complex situations such as changes in lighting and reflections on product surfaces make quality inspection much more difficult. Current technical means are difficult to dynamically adapt to changes in different products and quality inspection environments, resulting in poor performance of quality inspection systems in practical applications, especially when detecting tiny defects or flaws. Most existing image enhancement methods use preset static strategies and lack the ability to make dynamic adjustments based on the specific needs of industrial quality inspection, which affects the effect of product quality inspection. There are technical problems in related technologies that result in unsatisfactory product quality inspection results.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a defect detection method, device, non-volatile storage medium and electronic device to at least solve the technical problem of unsatisfactory product quality detection effect existing in the related art.

[0005] According to one aspect of an embodiment of the present application, a defect detection method is provided, including: obtaining detection performance of a target class object, wherein the detection performance is obtained by detecting the target class object using an initial detection model; determining target features that require data enhancement based on the detection performance and environmental parameters of the target class object; adjusting the initial detection model using a data enhancement strategy corresponding to the target feature to obtain a target detection model; and performing defect detection on the target class object using the target detection model to obtain a target detection result.

[0006] Optionally, the target features that require data enhancement are determined based on the detection performance and the environmental parameters of the target object, including: determining a plurality of predetermined surface features of the target object that require defect detection based on the environmental parameters and the object material of the target object; and determining the target feature from the plurality of predetermined surface features based on the detection performance.

[0007] Optionally, the multiple predetermined surface features correspond to predetermined defect patterns respectively, and determining the target feature from the multiple predetermined surface features based on the detection performance includes: acquiring reference defect data that matches the object type of the target class object; determining the target defect pattern existing in the target class object based on the reference defect data and the detection performance; and determining the target feature that matches the target defect pattern from the multiple predetermined surface features.

[0008] Optionally, the data enhancement strategy corresponding to the target feature is adopted to adjust the initial detection model to obtain the target detection model, including: determining the parameter type that needs to be adjusted in the initial detection model based on the recognition requirements of the target feature; and adopting the data enhancement strategy to adjust the parameter settings of the parameter type in the initial detection model to obtain the target detection model.

[0009] Optionally, the data enhancement strategy is adopted to adjust the parameter settings of the parameter type in the initial detection model to obtain the target detection model, including: determining the feature channel associated with the target feature in the initial detection model; adjusting the initial weight of the feature channel according to the weight adjustment method indicated by the data enhancement strategy to obtain the target detection model.

[0010] Optionally, the method further includes: acquiring change information of the environmental parameter; allowing the environmental parameter to be updated to a change parameter when the change information is greater than a predetermined environmental change threshold; and determining to perform adjustment processing on the initial detection model in response to generating the change parameter.

[0011] Optionally, the method further includes: determining the object types that the target production line is allowed to produce; and determining the initial detection model from a plurality of predetermined candidate models based on the object types, wherein the plurality of candidate models have different defect detection capabilities for different types of objects.

[0012] According to another aspect of an embodiment of the present application, a defect detection device is provided, including: a detection performance acquisition module, used to acquire the detection performance of a target class object, wherein the detection performance is obtained by detecting the target class object using an initial detection model; a target feature determination module, used to determine the target feature that needs data enhancement based on the detection performance and the environmental parameters of the target class object; a detection model adjustment module, used to adjust the initial detection model using the data enhancement strategy corresponding to the target feature to obtain a target detection model; and a detection result determination module, used to perform defect detection on the target class object using the target detection model to obtain a target detection result.

[0013] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, wherein the non-volatile storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing any one of the defect detection methods.

[0014] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the defect detection methods described.

[0015] In the embodiment of the present application, by obtaining the detection performance of the target class object, wherein the detection performance is obtained by detecting the target class object using the initial detection model; based on the detection performance and the environmental parameters of the target class object, the target features that need data enhancement are determined; the data enhancement strategy corresponding to the target features is used to adjust the initial detection model to obtain the target detection model; the target detection model is used to perform defect detection on the target class object to obtain the target detection result. The purpose of improving the adaptability of the detection model is achieved, and the technical effect of improving the product detection accuracy is achieved, thereby solving the technical problem of unsatisfactory product quality detection effect existing in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a flow chart of an optional defect detection method provided according to an embodiment of the present application;

[0018] Figure 2 is a schematic diagram of an optional defect detection method provided according to an embodiment of the present application;

[0019] Figure 3 It is a schematic diagram of an optional defect detection device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0023] RGB (Red Green Blue) is a color model based on the three primary colors (red, green, and blue). Red represents the intensity of the red component in the color model, usually ranging from 0 to 255. Green represents the intensity of the green component in the color model, usually ranging from 0 to 255. Blue represents the intensity of the blue component in the color model, usually ranging from 0 to 255. In the RGB color model, more than 16 million different colors can be generated by adjusting the intensity of these three components.

[0024] HSV (Hue Saturation Value) is a color model based on human visual perception. Hue represents the basic properties of color, that is, the type of color. The value of hue is usually between 0 and 360 degrees, representing a color ring that starts from red and rotates clockwise. Saturation represents the purity of the color, that is, the gray component contained in the color. The value range of saturation is usually from 0% to 100%, 0% represents gray, and 100% represents the purest color. Value represents the brightness of the color, that is, the lightness and darkness of the color. The value range of value is usually from 0% to 100%, 0% represents black, and 100% represents white.

[0025] Convolutional Neural Networks (CNN) is a deep learning model that is mainly used to process data with a grid structure, such as images. CNN can automatically learn features and patterns in images by simulating the processing method of the human visual system.

[0026] The convolutional layer is the core part of CNN, responsible for extracting features of input data. The convolutional layer contains multiple convolution kernels or filters, each of which is responsible for detecting specific features. The convolution kernel slides over the input data, calculates the weighted sum of the local area, and generates a feature map.

[0027] The Spatial Pyramid Pooling (SPP) layer is a technique used in convolutional neural networks to handle image size changes. The main purpose of the SPP layer is to enable the network to process input images of different sizes while keeping the size of the output feature map unchanged.

[0028] According to an embodiment of the present application, a method embodiment of defect detection is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] Figure 1 is a flow chart of an optional defect detection method provided according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0030] Step S102, obtaining the detection performance of the target object, wherein the detection performance is obtained by detecting the target object using the initial detection model;

[0031] It can be understood that the detection performance of specific target objects, such as metal parts and electronic components, is obtained by applying an initial detection model to detect defects in the target objects. The acquisition of detection performance helps to identify the weaknesses of the model, especially the deficiencies in the recognition of certain specific defects, and gain an in-depth understanding of the performance of the model when processing specific types of objects, so as to optimize the model in a targeted manner, ensuring that the model can maintain a high detection accuracy in various quality inspection environments, effectively improving the efficiency and reliability of industrial quality inspection.

[0032] In an optional embodiment, the method further includes: determining the types of objects that the target production line is allowed to produce; and based on the object type, determining an initial detection model from a plurality of predetermined candidate models, wherein the plurality of candidate models have different defect detection capabilities for different types of objects.

[0033] It can be understood that the types of objects that the target production line is allowed to produce are determined, and based on these object types, the most suitable initial model for defect detection of the target object is selected from multiple preset candidate models. Each candidate model is optimized and upgraded for a specific type of object. Therefore, they have different detection capabilities when processing products with different materials, shapes or surface features. The model selection mechanism based on object type ensures that the most suitable detection model for the target production line products has been selected before starting the quality inspection process, thereby maximizing the accuracy and efficiency of detection.

[0034] Optionally, the above-mentioned object types can be multiple. For example, a production line may mainly produce plastic parts. The shapes, sizes and designs of plastic parts may be different. They share certain properties of plastic materials and potential defect types, such as cracks, scratches, bubbles, etc. Therefore, a set of candidate models specifically for plastic materials are pre-trained, and each model is optimized for common defects of different types of plastic parts. For example, one model may be particularly good at detecting surface cracks, while another model may perform better in identifying internal bubbles. According to the type of plastic parts currently or planned to be produced by the production line, the most suitable initial detection model is selected from this set of candidate models, which can avoid separate model training for each specific product shape and reduce costs. At the same time, since the model has been optimized for the defects of plastic materials, its online upgrade and adjustment are also more convenient, reducing the online upgrade cost caused by the model not being suitable for the material or defect type. The model screening mechanism based on the production line material and defect type not only improves the economy of the model online upgrade, but also ensures the high accuracy and robustness of the model when processing specific products, providing a more efficient solution for industrial automation quality inspection.

[0035] Step S104, determining target features that require data enhancement based on the detection performance and the environmental parameters of the target object;

[0036] It can be understood that based on the detection performance data and the specific environmental parameters of the target object, in-depth analysis is performed to determine the target features that require further data enhancement. The target features include at least the texture, brightness distribution, and reflectivity of the object surface. Environmental parameters cover lighting conditions, shooting angles, background complexity, etc. By adding environmental parameters, the key factors that lead to the decline in detection performance can be identified, providing precise guidance for subsequent data enhancement strategies.

[0037] Optionally, if the detection performance analysis shows that the model has a high missed detection rate in a highly reflective environment, the reflective features on the surface of the object are identified as target features, and the performance of the features in the data is enhanced in a targeted manner, such as increasing the reflection simulation in the local area or using special image processing techniques to reduce the impact of reflection. The detection capability of the model in a highly reflective environment will be significantly improved, reducing false positives and missed positives, thereby improving the overall detection efficiency and accuracy. This feature enhancement method based on performance analysis and environmental parameters makes the data enhancement process more scientific and targeted.

[0038] In an optional embodiment, based on the detection performance and the environmental parameters of the target object, the target features that need data enhancement are determined, including: based on the environmental parameters and the object material of the target object, determining multiple predetermined surface features of the target object that need defect detection; based on the detection performance, determining the target feature from multiple predetermined surface features.

[0039] It can be understood that a series of predetermined surface features closely related to defect detection are determined based on the specific environmental parameters of the target object and the material properties of the object. The strategy of determining data enhancement target features based on environmental parameters and detection performance can accurately identify and solve the main problems encountered by the model in specific detection scenarios, thereby significantly improving the detection accuracy and robustness of the model.

[0040] Optionally, based on the performance feedback of the model when detecting the target class of objects, the key features that cause the detection performance to decline are selected from the pre-selected surface features as target features for targeted data enhancement. For example, if the model's detection accuracy is significantly reduced when processing plastic parts with complex surface textures, "surface texture enhancement" will be used as a target feature to improve the model's recognition ability by increasing texture details or simulating texture performance under different lighting conditions.

[0041] Optionally, the above-mentioned multiple predetermined surface features include at least clarity of surface texture, consistency of color, sharpness of edges, reflectivity, transparency, surface smoothness, etc.

[0042] In an optional embodiment, multiple predetermined surface features correspond to predetermined defect patterns respectively, and based on the detection performance, a target feature is determined among the multiple predetermined surface features, including: obtaining reference defect data that matches the object type of the target class object; based on the reference defect data and the detection performance, determining the target defect pattern existing in the target class object; and determining, among the multiple predetermined surface features, a target feature that matches the target defect pattern.

[0043] It can be understood that each predetermined surface feature is associated with one or more specific defect patterns. Reference defect data matching the object type of the target class object is obtained, and the data provides detailed information about the defect types and distribution patterns that the object may encounter. Then, combined with the performance of the model in the current detection task, that is, the detection performance, the target defect pattern existing in the target class object is identified. Finally, in the predetermined surface feature set, the features that match the target defect pattern are selected as the target features. The data enhancement strategy based on the defect pattern realizes the precise positioning and efficient implementation of data enhancement, and improves the detection accuracy of the model for specific types of defects.

[0044] Optionally, the predetermined surface features are associated with one or more specific defect modes, for example, the clarity of the surface texture may be related to the detection of cracks or scratches, while the lighting conditions may affect the recognition of small depressions or protrusions. If the model performs poorly in identifying small cracks on plastic parts, it will focus on enhancing the surface texture features related to crack detection, that is, increasing the contrast and clarity of the texture details.

[0045] Optionally, there can be multiple defect features and their corresponding data enhancement strategies and model adjustment methods, for example, small defect features such as scratches, small holes, and slight deformations. The corresponding data enhancement strategies should be local magnification, high contrast enhancement, and detail enhancement. The model adjustment method is to increase the local detail perception ability of the model, for example, by increasing the number of convolutional layers or changing the size of the convolution kernel to enhance the local feature extraction ability of the model. At the same time, the loss function of the model can be adjusted to increase the weight of small defect detection, prompting the model to pay more attention to these detail features.

[0046] Optionally, defect features that are sensitive to lighting changes, such as reflections, shadows, and highlights. The corresponding data enhancement strategies are lighting randomization, shadow generation, and highlight simulation. The model adjustment method is to consider adding a lighting normalization layer or an adaptive lighting correction module so that the model can better handle inputs under different lighting conditions. At the same time, adjust the model's strategy, such as using online data enhancement, to ensure that the model can see samples of lighting changes every time it is upgraded, thereby improving the model's robustness to lighting changes.

[0047] Optionally, complex texture defect features, such as surface flaws and uneven texture. The corresponding data enhancement strategies are random texture generation, color jitter, and contrast adjustment. The model adjustment method is to add a convolutional network module for texture recognition, such as using an Inception module or a Dense module with strong texture recognition capabilities to enhance the model's perception of complex textures. At the same time, adjust the weight learning rate of the model's texture feature extraction layer to ensure that texture features can be extracted more accurately.

[0048] Optionally, defect features that are sensitive to position changes, such as edge cracks and labeling position deviations. The corresponding data enhancement strategies are random cropping, position offset, and rotation. The model adjustment method is to enhance the position invariance of the model, such as using a position-sensitive loss function or introducing a spatial pyramid pooling layer, so that the model can stably identify the same defect at different positions. In addition, the input size of the model and the resolution of the output feature map are adjusted to adapt to defect features at different positions.

[0049] Optionally, color-sensitive defect features, such as color difference, fading, and discoloration. The corresponding data enhancement strategies are color space transformation, brightness adjustment, and hue randomization. The model is adjusted by adding a color space conversion layer so that the model can handle inputs in different color spaces, such as converting from RGB (Red Green Blue) to HSV (Hue Saturation Value), thereby enhancing color sensitivity. Adjust the weights of the color feature extraction layer in the model to ensure that color changes can be identified more accurately.

[0050] Optionally, defect features that are sensitive to shape changes, such as deformation and unevenness. The corresponding data enhancement strategies are geometric transformation (such as scaling and distortion) and morphological operations (such as corrosion and expansion). The model adjustment method is to introduce a shape invariance mechanism, such as using a shape-invariant convolution kernel, or adding a network branch for shape feature extraction. Adjust the model's geometric transformation parameters, such as controlling the degree and direction of deformation, to optimize the model's ability to recognize shape changes.

[0051] Optionally, defect features affected by background complexity, such as cluttered background, similar background. Data enhancement strategies include background randomization, adding random noise, and contrast adjustment between foreground and background. Model adjustment methods include adding background suppression or attention mechanisms to enable the model to focus on the defect area and reduce background interference. In addition, a regional attention network module can be used to enable the model to dynamically adjust its attention to different areas and enhance its adaptability to changes in background complexity.

[0052] In an optional embodiment, the method further includes: obtaining change information of environmental parameters; when the change information is greater than a predetermined environmental change threshold, allowing the environmental parameters to be updated to change parameters; and in response to generating change parameters, determining to perform adjustment processing on the initial detection model.

[0053] It can be understood that the changes in environmental parameters are continuously detected and the parameter change information is recorded. If the detected environmental changes exceed the predetermined environmental change threshold, that is, the environmental conditions have changed significantly, the system will automatically update the current environmental parameters and adjust them to change parameters that reflect the new environmental conditions. In response to the generation of change parameters, the adjustment process of the initial detection model is performed. By detecting changes in environmental parameters in real time and automatically adjusting the detection model, the environmental adaptability and performance stability of the model are improved. It ensures that even when conditions such as lighting and background change, the detection model can still maintain high accuracy and robustness, reducing detection errors caused by environmental changes and improving the quality control level of the product.

[0054] Optionally, environmental parameters change in real time, but small changes do not want to trigger model updates until the environment changes significantly. Therefore, an intelligent response mechanism for changes in environmental parameters is designed to balance the frequency of model updates and resource consumption, thereby reducing the cost of model upgrades. Continuously detect changes in environmental parameters in real time, such as light intensity, angle, background complexity, etc. Set a predetermined environmental change threshold. Only when the change in environmental parameters exceeds this threshold, that is, when the environmental conditions have changed substantially, will the model adjustment process be initiated. The setting of the threshold is based on a deep understanding of the industrial quality inspection environment and consideration of the cost of model updates. The size of the threshold can be flexibly adjusted to adapt to the speed of change of different quality inspection environments and the requirements for model update frequency. For example, in a quality inspection environment where the light changes frequently, the threshold may be set lower to adjust the model more frequently; while in a scene with a relatively stable environment, the threshold can be set higher to reduce unnecessary model updates.

[0055] Step S106, using a data enhancement strategy corresponding to the target feature to adjust the initial detection model to obtain a target detection model;

[0056] It can be understood that by adopting a data enhancement strategy that matches the target features, the initial detection model is adjusted and optimized in a targeted manner to obtain a target detection model that is more suitable for specific quality inspection needs, so that the model has stronger adaptability and generalization capabilities when facing the actual industrial quality inspection environment.

[0057] In an optional embodiment, a data enhancement strategy corresponding to the target feature is adopted to adjust the initial detection model to obtain the target detection model, including: determining the parameter type that needs to be adjusted in the initial detection model based on the recognition requirements of the target feature; and using the data enhancement strategy to adjust the parameter settings of the parameter type in the initial detection model to obtain the target detection model.

[0058] It can be understood that based on the recognition requirements of the target features, it is important to identify which parameter types in the initial detection model are most critical to improving the detection effect of the feature. A data enhancement strategy that matches the target features is used to make targeted adjustments to the identified parameter types to obtain the target detection model. By directly adjusting the parameter types in the model that are directly related to the recognition of the target features, efficient optimization of the model is achieved, avoiding the waste of computing resources caused by comprehensive adjustments to the model.

[0059] In an optional embodiment, a data enhancement strategy is used to adjust the parameter settings of the parameter types in the initial detection model to obtain a target detection model, including: determining the feature channels associated with the target features in the initial detection model; adjusting the initial weights of the feature channels according to the weight adjustment method indicated by the data enhancement strategy to obtain the target detection model.

[0060] It can be understood that the feature channels associated with the target features in the initial detection model are determined, and the initial weights of the feature channels are adjusted according to the weights indicated by the data enhancement strategy to obtain the target detection model, and the generalization ability and accuracy of the model are improved through the enhancement strategy.

[0061] Step S108, using the target detection model to perform defect detection on the target object to obtain a target detection result.

[0062] It can be understood that the target detection model conducts in-depth analysis of the input quality inspection image data, uses data enhancement strategies to accurately locate and classify various defects in the image, such as cracks, scratches, missing parts, etc., and finally outputs the target detection results, including the location, type and severity of the defects. The target model can effectively identify minor defects, reduce possible false detections and missed detections in quality inspections, improve the product qualification rate, and also speed up the quality inspection process and reduce the need for manual quality inspections.

[0063] Through the above-mentioned step S102, the detection performance of the target class object is obtained, wherein the detection performance is obtained by detecting the target class object using the initial detection model; step S104, based on the detection performance, and the environmental parameters of the target class object, the target features that need data enhancement are determined; step S106, the initial detection model is adjusted using the data enhancement strategy corresponding to the target features to obtain the target detection model; step S108, the target detection model is used to perform defect detection on the target class object to obtain the target detection result, which can achieve the purpose of improving the adaptability of the detection model and realize the technical effect of improving the product detection accuracy, thereby solving the technical problem of unsatisfactory product quality detection effect existing in the related technology.

[0064] Based on the above embodiments and optional embodiments, the present application proposes an optional implementation mode: Figure 2is a schematic diagram of an optional defect detection method provided according to an embodiment of the present application, such as Figure 2 As shown in the figure, the data input module, data feature analysis module, enhancement strategy generation module, detection adaptive enhancement strategy module, training status detection module, and enhancement effect evaluation module, etc., are demonstrated, showing the division of labor and cooperation between different modules.

[0065] The data feature analysis module is specifically used to analyze the image data features in industrial quality inspection, including product surface reflection, texture complexity, color distribution, etc. By analyzing these features, factors that may cause detection errors can be identified and enhancement strategies can be dynamically adjusted.

[0066] The detection performance of specific target objects, such as metal parts and electronic components, is obtained by applying an initial detection model to detect defects in the target objects. The detection performance acquisition helps to identify the weaknesses of the model, especially the deficiencies in the recognition of certain specific defects, and gain an in-depth understanding of the performance of the model when processing specific types of objects, so as to optimize the model in a targeted manner, ensuring that the model can maintain high detection accuracy in various quality inspection environments, effectively improving the efficiency and reliability of industrial quality inspection.

[0067] Based on the detection performance data and the specific environmental parameters of the target object, in-depth analysis is performed to determine the target features that require further data enhancement. The target features include at least the texture, brightness distribution, and reflectivity of the object surface. Environmental parameters cover lighting conditions, shooting angles, background complexity, etc. By adding environmental parameters, the key factors that lead to reduced detection performance can be identified, providing precise guidance for subsequent data enhancement strategies.

[0068] According to the specific environmental parameters of the target object and the material properties of the object, a series of predetermined surface features closely related to defect detection are determined. The strategy of determining data enhancement target features based on environmental parameters and detection performance can accurately identify and solve the main problems encountered by the model in specific detection scenarios, thereby significantly improving the detection accuracy and robustness of the model.

[0069] The model status detection module is used to detect the upgrade process of the quality inspection model, including training loss (retraining loss during the upgrade process), detection accuracy, missed detection rate and other indicators. This module can automatically optimize the data enhancement strategy based on the real-time performance of the model. For example, if the missed detection rate of minor defects in a certain type of product is high, the system will enhance the local magnification and high contrast processing of this type of product.

[0070] If the detection performance analysis shows that the model has a high missed detection rate in a highly reflective environment, the reflective features on the surface of the object are identified as target features, and the performance of the features in the data is enhanced in a targeted manner, such as increasing the reflection simulation in the local area or using special image processing technology to reduce the impact of reflection. The detection capability of the model in a highly reflective environment will be significantly improved, reducing false positives and missed positives, thereby improving the overall detection efficiency and accuracy. This feature enhancement method based on performance analysis and environmental parameters makes the data enhancement process more scientific and targeted.

[0071] Continuously detect changes in environmental parameters and record parameter change information. If the detected environmental changes exceed the predetermined environmental change threshold, that is, the environmental conditions have changed significantly, the system will automatically update the current environmental parameters and adjust them to change parameters that reflect the new environmental conditions. In response to the generation of change parameters, perform adjustment processing on the initial detection model. By detecting changes in environmental parameters in real time and automatically adjusting the detection model, the environmental adaptability and performance stability of the model are improved. Ensure that even when conditions such as lighting and background change, the detection model can still maintain high accuracy and robustness, reduce detection errors caused by environmental changes, and improve the quality control level of the product.

[0072] The adaptive enhancement strategy module covers a variety of enhancement methods, such as noise addition in local areas, lighting adjustment, surface reflection processing, etc. The system will flexibly adjust the enhancement method and its intensity according to the surface characteristics, shape and lighting conditions of the product to ensure that the enhancement process is more targeted and efficient.

[0073] Small defect features, such as scratches, small holes, and slight deformations. The corresponding data enhancement strategy should be local amplification, high contrast enhancement, and detail enhancement. The model adjustment method is to increase the local detail perception ability of the model, such as by increasing the number of convolutional layers or changing the size of the convolution kernel to enhance the local feature extraction ability of the model. At the same time, the loss function of the model can be adjusted to increase the weight of small defect detection, so that the model pays more attention to these detail features.

[0074] Defect features that are sensitive to lighting changes, such as reflections, shadows, and highlights. The corresponding data enhancement strategies are lighting randomization, shadow generation, and highlight simulation. The model adjustment method is to consider adding a lighting normalization layer or an adaptive lighting correction module so that the model can better handle inputs under different lighting conditions. At the same time, adjust the model upgrade strategy, such as using online data enhancement, to ensure that the model can see samples of lighting changes every time it is upgraded, thereby improving the model's robustness to lighting changes.

[0075] Defect features with complex textures, such as surface flaws and uneven textures. The corresponding data enhancement strategies are random texture generation, color jitter, and contrast adjustment. The model adjustment method is to add a convolutional network module for texture recognition, such as using convolution kernels of different sizes for feature extraction in the same layer, and then splicing the extracted features. This allows more features to be learned at different scales to enhance the model's perception of complex textures. At the same time, adjust the weight learning rate of the model's texture feature extraction layer to ensure that texture features can be extracted more accurately.

[0076] Defect features that are sensitive to position changes, such as edge cracks and label position deviations. The corresponding data enhancement strategies are random cropping, position offset, and rotation. The model adjustment method is to enhance the position invariance of the model, such as using a position-sensitive loss function or introducing a spatial pyramid pooling layer, so that the model can stably identify the same defect at different positions. In addition, the input size of the model and the resolution of the output feature map are adjusted to adapt to defect features at different positions.

[0077] Color-sensitive defect features, such as color difference, fading, and discoloration. The corresponding data enhancement strategies are color space transformation, brightness adjustment, and hue randomization. The model adjustment method is to add a color space conversion layer so that the model can handle inputs in different color spaces, such as converting from RGB (Red Green Blue) to HSV (Hue Saturation Value), thereby enhancing color sensitivity. Adjust the weights of the color feature extraction layer in the model to ensure that color changes can be more accurately identified.

[0078] Defect features that are sensitive to shape changes, such as deformation and unevenness. The corresponding data enhancement strategies are geometric transformation (such as scaling and distortion) and morphological operations (such as corrosion and expansion). The model adjustment method is to introduce a shape invariance mechanism, such as using a shape-invariant convolution kernel, or adding a network branch for shape feature extraction. Adjust the model's geometric transformation parameters, such as controlling the degree and direction of deformation, to optimize the model's ability to recognize shape changes.

[0079] Defect features affected by background complexity, such as cluttered background and similar background. Data enhancement strategies include background randomization, adding random noise, and adjusting the contrast between foreground and background. Model adjustment methods include adding background suppression or attention mechanisms to enable the model to focus on the defect area and reduce background interference. In addition, a regional attention network module can be used to enable the model to dynamically adjust its attention to different areas and enhance its adaptability to changes in background complexity.

[0080] The enhancement effect evaluation module compares the effects of different enhancement strategies through experiments and actual quality inspection data, and optimizes the enhancement parameter settings. This module can also dynamically evaluate the enhancement effect during the quality inspection process and make timely strategy adjustments, which are then executed by the model training / application module.

[0081] The above optional implementation methods achieve at least the following effects: for different products and quality inspection environments, the data enhancement strategy can be dynamically adjusted according to product characteristics to improve the accuracy of defect detection; a variety of enhancement technologies are introduced, including local magnification, surface reflection processing, lighting adjustment, etc., to ensure that the enhancement method can better cope with different defects in complex quality inspection scenarios; the adaptive data enhancement method effectively improves the model's detection accuracy of tiny defects on the product surface, reduces missed detections and false detections, has strong versatility, and can be applied to various types of industrial quality inspection tasks; when data is relatively scarce, the adaptive enhancement method can effectively expand the data set, maximize the use of existing quality inspection image data, dynamically adjust the data enhancement strategy, improve data utilization and improve the generalization ability of the model.

[0082] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0083] In this embodiment, a defect detection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module" and "device" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0084] According to an embodiment of the present application, a device embodiment for implementing a defect detection method is also provided. Figure 3 A schematic diagram of a defect detection device according to an embodiment of the present application is shown in Figure 3 As shown, the above-mentioned defect detection device includes a detection performance acquisition module 302, a target feature determination module 304, a detection model adjustment module 306, and a detection result determination module 308. The device is described below.

[0085] A detection performance acquisition module 302 is used to acquire detection performance of a target object, wherein the detection performance is obtained by detecting the target object using an initial detection model;

[0086] The target feature determination module 304 is connected to the detection performance acquisition module 302 and is used to determine the target features that need data enhancement based on the detection performance and the environmental parameters of the target object;

[0087] The detection model adjustment module 306 is connected to the target feature determination module 304 and is used to adjust the initial detection model by adopting a data enhancement strategy corresponding to the target feature to obtain a target detection model;

[0088] The detection result determination module 308 is connected to the detection model adjustment module 306 and is used to use the target detection model to perform defect detection on the target object to obtain the target detection result.

[0089] In a defect detection device provided by an embodiment of the present application, a detection performance acquisition module 302 is set to acquire the detection performance of a target object, wherein the detection performance is obtained by detecting the target object using an initial detection model; a target feature determination module 304 is connected to the detection performance acquisition module 302, and is used to determine the target features that need data enhancement based on the detection performance and the environmental parameters of the target object; a detection model adjustment module 306 is connected to the target feature determination module 304, and is used to use the data enhancement strategy corresponding to the target feature to adjust the initial detection model to obtain a target detection model; a detection result determination module 308 is connected to the detection model adjustment module 306, and is used to use the target detection model to perform defect detection on the target object to obtain a target detection result. The purpose of improving the adaptability of the detection model is achieved, and the technical effect of improving the product detection accuracy is achieved, thereby solving the technical problem of unsatisfactory product quality detection effect existing in the related art.

[0090] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0091] It should be noted that the above-mentioned acquisition detection performance module 302, determination target feature module 304, adjustment detection model module 306, and determination detection result module 308 correspond to steps S102 to S108 in the embodiment, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment. It should be noted that the above-mentioned modules can be run in a computer terminal as part of the device.

[0092] It should be noted that the optional or preferred implementation of this embodiment can refer to the relevant description in the embodiment, which will not be repeated here.

[0093] The above-mentioned defect detection device may also include a processor and a memory. The detection performance acquisition module 302, the target feature determination module 304, the detection model adjustment module 306, the detection result determination module 308, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0094] The processor includes a kernel, which retrieves the corresponding program unit from the memory. There can be one or more kernels. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0095] An embodiment of the present application provides a non-volatile storage medium on which a program is stored, and when the program is executed by a processor, a defect detection method is implemented.

[0096] The embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program stored in the memory and can be run on the processor. When the processor executes the program, the following steps are implemented: obtaining the detection performance of the target class object, wherein the detection performance is obtained by detecting the target class object using an initial detection model; based on the detection performance and the environmental parameters of the target class object, determining the target features that need data enhancement; using the data enhancement strategy corresponding to the target features, adjusting the initial detection model to obtain a target detection model; using the target detection model, performing defect detection on the target class object to obtain a target detection result. The device in this article can be a server, a PC, etc.

[0097] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: obtaining detection performance of a target class object, wherein the detection performance is obtained by detecting the target class object using an initial detection model; determining target features that require data enhancement based on the detection performance and environmental parameters of the target class object; adjusting the initial detection model using a data enhancement strategy corresponding to the target features to obtain a target detection model; and performing defect detection on the target class object using the target detection model to obtain a target detection result.

[0098] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0099] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0102] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0103] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0104] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0105] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0106] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0107] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A defect detection method, characterized in that: include: Acquire detection performance of the target class object, wherein the detection performance is obtained by detecting the target class object using an initial detection model; Based on the detection performance and the environmental parameters of the target object, determine the target features that need data enhancement; Adopting a data enhancement strategy corresponding to the target feature, adjusting the initial detection model to obtain a target detection model; The target detection model is used to perform defect detection on the target object to obtain a target detection result.

2. The method according to claim 1, characterized in that Determining target features that require data enhancement based on the detection performance and the environmental parameters of the target object includes: Based on the environmental parameters and the object material of the target object, determining a plurality of predetermined surface features of the target object that require defect detection; Based on the detection performance, the target feature is determined among the plurality of predetermined surface features.

3. The method according to claim 2, characterized in that The plurality of predetermined surface features respectively correspond to predetermined defect patterns, and determining the target feature from the plurality of predetermined surface features based on the detection performance comprises: Acquire reference defect data matching the object type of the target class object; Determining a target defect pattern existing in the target class object based on the reference defect data and the detection performance; Among the plurality of predetermined surface features, the target feature matching the target defect pattern is determined.

4. The method according to claim 1, characterized in that The adopting the data enhancement strategy corresponding to the target feature to adjust the initial detection model to obtain the target detection model includes: Based on the recognition requirements of the target features, determining the type of parameters that need to be adjusted in the initial detection model; The data enhancement strategy is adopted to adjust the parameter settings of the parameter type in the initial detection model to obtain the target detection model.

5. The method according to claim 4, characterized in that The adopting the data enhancement strategy to adjust the parameter settings of the parameter type in the initial detection model to obtain the target detection model includes: Determining a feature channel associated with the target feature in the initial detection model; The initial weights of the feature channels are adjusted according to the weight adjustment method indicated by the data enhancement strategy to obtain the target detection model.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Obtaining change information of the environmental parameters; When the change information is greater than a predetermined environment change threshold, allowing the environment parameter to be updated to a change parameter; In response to generating the change parameter, it is determined to perform an adjustment process on the initial detection model.

7. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Determine the types of objects that the target production line is allowed to produce; Based on the object type, the initial detection model is determined from a plurality of predetermined candidate models, wherein the plurality of candidate models have different defect detection capabilities for different types of objects.

8. A defect detection device, characterized in that: include: A detection performance acquisition module, used to acquire detection performance of a target object, wherein the detection performance is obtained by detecting the target object using an initial detection model; A target feature determination module is used to determine target features that require data enhancement based on the detection performance and the environmental parameters of the target object; A detection model adjustment module is used to adjust the initial detection model by adopting a data enhancement strategy corresponding to the target feature to obtain a target detection model; The detection result determination module is used to use the target detection model to perform defect detection on the target class object to obtain the target detection result.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the defect detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the defect detection method described in any one of claims 1 to 7.

Images