Defect detection system, method and device based on artificial intelligence image processing

Abstract

The present invention discloses a defect detection system, method and device based on artificial intelligence image processing, and specifically relates to the field of image defect detection. The data acquisition module is used to obtain image data from industrial equipment, sensors and cameras, and perform preliminary formatting processing on the image data. The present invention avoids redundant parameter debugging, shortens the model convergence time, improves the utilization efficiency of training resources, enhances the generalization ability of the model, and improves the performance of the model when processing rare scenarios; it can maintain high accuracy while significantly reducing computing resource consumption, ensure that the defect detection system can also run efficiently in resource-constrained devices, achieve adaptation to different hardware conditions, expand the application scenario requirements of the system, reduce operation problems caused by environmental conflicts, enhance the security of the system, and avoid the impact of external malware attacks or resource leakage on system operation.

Description

Technical Field

[0001] The present invention relates to the field of image defect detection, and in particular to a defect detection system, method and device based on artificial intelligence image processing. Background Art

[0002] In modern manufacturing, product quality assurance is an important factor in market competition, and defect detection, as a key link in quality control, has received much attention. With the rapid development of industrial manufacturing and the continuous improvement of product quality requirements, traditional defect detection methods rely on manual experience and simple rule matching, gradually revealing problems such as low efficiency, insufficient accuracy, and inability to meet industrial needs for complex scenarios and high precision requirements. In recent years, image processing technology based on artificial intelligence has gradually become the mainstream direction of industrial defect detection due to its powerful feature extraction capabilities and efficient automation characteristics. The rapid development of artificial intelligence (AI) technology, especially image processing and deep learning technology, has provided a new solution for defect detection. Image processing technology based on artificial intelligence can efficiently process complex image data and extract accurate features from it to identify tiny defects in products. This technology is applied to automated inspection equipment to achieve high-precision, high-efficiency and non-contact defect detection, significantly improving the automation and intelligence level of the manufacturing industry. To this end, we propose a defect detection system, method and device based on artificial intelligence image processing. Summary of the invention

[0003] The purpose of the present invention is to solve the problems of easy redundant parameter debugging, long model convergence time, low utilization efficiency of training resources, reduced generalization ability of the model, poor performance of the model when processing rare scenes; and failure to maintain high accuracy while reducing computing resource consumption, failure to run efficiently in resource-constrained devices, and many operating problems caused by environmental conflicts, and propose a defect detection system, method and device based on artificial intelligence image processing;

[0004] Therefore, a defect detection system, method and device based on artificial intelligence image processing are proposed.

[0005] In a first aspect of the implementation of the present invention, a defect detection system based on artificial intelligence image processing is first proposed, the system comprising: a data acquisition module, a data preprocessing module, a data enhancement module, a feature extraction module, a feature optimization module, a defect detection module, a model deployment module, a labeling and positioning module, a report generation module, an adaptive optimization module, a cloud collaboration module and a user interaction module;

[0006] The data acquisition module is used to obtain image data from industrial equipment, sensors and cameras, and perform preliminary formatting processing on the image data;

[0007] The data preprocessing module is used to automatically identify and repair damaged areas in the image to ensure the quality of the input data;

[0008] The data enhancement module is used to optimize data enhancement parameters and generate enhanced samples with high diversity and effectiveness;

[0009] The feature extraction module is used to extract feature information from the image;

[0010] The feature optimization module is used to select a feature set that meets the defect detection requirements from the extracted multiple feature information combinations;

[0011] The defect detection module is used to optimize the defect detection model, and classify and segment the input image through the model, and identify and mark the defect area;

[0012] The model deployment module is used to encapsulate the defect detection model, deploy and run it across platforms, and dynamically adjust computing resources through a resource scheduling mechanism;

[0013] The marking and positioning module is used to mark and spatially locate the detected defects;

[0014] The report generation module automatically generates test result reports in various formats;

[0015] The adaptive optimization module is used to monitor the system operation status in real time and dynamically optimize the full-process configuration of the system;

[0016] The cloud collaboration module is used for cloud storage and collaboration of detection data and models, and supports real-time sharing and analysis across teams;

[0017] The user interaction module is used to provide a test result display interface, and also supports users to manually adjust parameters, view test results and perform manual review.

[0018] As a further solution of the present invention, the specific steps of optimizing the data enhancement parameters by the data enhancement module are as follows:

[0019] S101: Randomly initialize the parameter set of data enhancement, including the rotation angle, scaling factor, shear coefficient and noise level parameters. Set the objective function based on the improvement of the deep learning model training performance by the sample. ,in Represents the objective function value, which indicates the comprehensive evaluation of the enhanced sample. represents the diversity index of the enhanced samples, represents the effectiveness index of the enhanced sample, as well as represents the weight coefficient, which is used to adjust the relative importance of diversity and effectiveness;

[0020] S102: Set initial temperature , termination temperature and cooling rate , , and based on the cooling strategy Perform cooling operation, Representative The temperature of the round iteration number, Represents the number of iterations, after which the current parameter set is randomly perturbed based on uniform distribution and Gaussian distribution, and a new parameter set is generated;

[0021] S103: Calculate the objective function value of the new parameter set, and then calculate the change in the objective function value of the new parameter set and the current parameter set ,like , then replace the current parameter set with the new parameter set, otherwise, Calculate the probability of accepting a worse solution , and choose to accept the new parameter set based on this probability;

[0022] S104: After each round of update, the current temperature is updated based on the cooling strategy For the next round of temperature , repeat the random perturbation and the new parameter set selection acceptance until the termination temperature is reached After the iteration, the optimal parameter set is output, and the preprocessed image data is enhanced based on the parameters in the optimal parameter set to generate an enhanced sample set.

[0023] As a further solution of the present invention, the specific steps of the feature optimization module selecting the required feature set from the extracted multiple feature information combinations are as follows:

[0024] S201: Create the initial root node , represents the set of all candidate features extracted by the feature extraction module , initialize the number of visits and cumulative evaluation values ​​of the root node, then start from the root node and gradually expand to new child nodes by randomly selecting feature information to build a search tree, where each child node represents a feature subset , the edge represents the selection from a feature subset to an extended subset, and the objective function is defined by the classification accuracy ,in represents a feature subset, and ;

[0025] S202: Starting from the root node, the UCB value of each child node is calculated by the upper confidence interval formula, and then the node with the highest UCB value is selected, and the next node is selected layer by layer until a node that has not been fully expanded is reached. On the current child node, a feature combination that has not been explored is randomly selected as a new child node, and the number of visits and cumulative evaluation values ​​of the new child node are initialized. The specific calculation formula of the upper confidence interval formula is as follows:

[0026]

[0027] In the formula, Representative Child nodes The upper confidence limit value of ; Representative Child nodes The cumulative assessed value of Representative Child nodes Number of visits; Represents the balance coefficient controlling exploration and exploitation; Representative Child nodes The number of visits to the parent node;

[0028] S203: Starting from the expanded child node, randomly select features and construct feature subsets, and use the machine learning model to train and evaluate the performance of the feature subsets. Calculate the objective function value of the child node, trace the simulation results back to all the nodes passed, and update the number of visits and cumulative evaluation values;

[0029] S204: Repeat the above steps S202 and S203 to gradually expand and optimize the search tree until the preset maximum number of iterations is reached, and then select the feature subset corresponding to the child node with the most visits from the root node as the final required feature set.

[0030] As a further solution of the present invention, the specific steps of the defect detection module to optimize the defect detection model are as follows:

[0031] S301: Collect the status of the current defect detection model training process and the executable training operations, and construct the state space of the defect detection model based on the two types of collected data and action space , calculated in the state Perform any training operation After transfer to state The state transition probability , based on the impact of this training operation on the training quality, define the reward function , calculated in the state Execute training operation The feedback obtained included: , , ;

[0032] S302: Randomly initialize the initial state of the defect detection model , and record the initial reward value , and set the total number of steps in the training process and the early stopping condition, and then Status of the moment Execute training operation Transfer to next state , and evaluate the performance of the model on the validation set, and calculate the reward value in the current state ;

[0033] S303: Based on Status of the moment , possible training actions , reward function and the transition probability , and calculate each state through the value iteration algorithm The value function To get the current state Take any training action The expected long-term return is calculated by recursively updating the value function using the Bellman equation to calculate each state The expected reward under each state is calculated and compared with each expected reward value to determine the training operation that should be taken in each state to maximize the expected reward value. ;

[0034] S304: Repeat the strategy update until the total number of steps in the preset training process is reached, or the model performance reaches the preset early stopping condition, that is, in each state The best action to choose , the strategy with the highest value function value ranked from large to small is selected, and the current defect detection model parameters are updated and trained based on the strategy.

[0035] In a second aspect of the present invention, a defect detection method based on artificial intelligence image processing is proposed, the method comprising the following steps:

[0036] Q1. Acquire and preprocess high-resolution multi-source image data, automatically select enhancement methods and parameter combinations, and generate multiple sets of enhanced image samples;

[0037] Q2. Extract key features from the image, use each group of extracted key features as a candidate feature set, and select the feature set that meets the defect detection requirements;

[0038] Q3. Based on the selected feature set, the defect detection model is used to classify and segment the image, and the parameter configuration is dynamically optimized according to the feedback detection results;

[0039] Q4. Accurately mark and spatially locate the detected defects, encapsulate defect detection models that meet the requirements, and deploy and run them across platforms;

[0040] Q5. After the inspection is completed, the results are analyzed and a visual report is output. The defect detection results are displayed in real time through a graphical interface, and manual review and secondary marking are supported.

[0041] As a further solution of the present invention, the specific steps of accurately marking and spatially locating the detected defects described in Q4 are as follows:

[0042] S401: deploying the defect detection model to the target hardware environment, initializing the model parameters, loading the trained weight file, adjusting the image size of the image to be detected, normalizing the adjusted image to be detected, inputting the preprocessed image to be detected, and extracting feature information through the defect detection model;

[0043] S402: Input the extracted feature information into the classification and localization network layers of the model. The classification network layer generates defect prediction results and the probability of each defect category. The localization network layer generates a bounding box of the defect area. The prediction results output by the model are then post-processed to generate defect detection results.

[0044] S403: Extract the geometric features of the defect area based on the detected bounding box parameters, and calculate the specific size of the defect. Calculate the position of the defect in the actual physical space based on the pixel coordinates in the image and the geometric calibration parameters of the camera. Use the camera intrinsic parameter matrix and extrinsic parameter matrix for projection transformation, convert the image coordinates into actual three-dimensional space coordinates, and then merge the defect category, area and three-dimensional coordinates into the image for annotation.

[0045] As a further solution of the present invention, the specific steps of the defect detection model that meets the requirements of Q4 are as follows:

[0046] S501: Select the trained defect detection model as the teacher model, build multiple groups of different lightweight models by reducing the number of network layer units and the network depth, and use them as student models, input the historical image data into the teacher model, and output the predicted probability distribution, and apply the temperature parameter on the predicted probability distribution of the teacher model to generate soft targets, and use the real label data as hard targets;

[0047] S502: Input the same historical image data as the teacher model into the student model. The student model processes the historical image data through the forward propagation algorithm and outputs the corresponding predicted probability distribution. The learning loss of the predicted probability distribution and the hard label is calculated through the cross entropy loss function. The learning loss of the predicted probability distribution and the soft target is calculated through the KL divergence. The learning loss of the hard target and the learning loss of the soft target are weighted and aggregated to obtain the corresponding distillation loss function.

[0048] S503: During the training process, the input data passes through the teacher model and the student model in turn, and the distillation loss function is used to calculate the corresponding distillation loss value. Then, the distillation loss value is passed layer by layer starting from the output layer of the student model according to the chain rule, and the gradient of the distillation loss value for each network layer of the model is calculated, and the student model parameters are updated using the Adam algorithm;

[0049] S504: Use the unused historical image data as a validation data set, evaluate the performance of the student model on the validation set, check whether the preset accuracy is achieved, and if the performance is insufficient, adjust the hyperparameters and retrain, encapsulate the trained student model as a Docker image, configure the container dependencies, and write a Dockerfile example;

[0050] S505: Monitor the CPU usage, GPU usage, and memory usage performance data of each container in real time and store them in the monitoring system. When the monitoring data is greater than the usage limit, increase the number of container instances and allocate more resources. Otherwise, reduce the number of container instances to save resources. Use the container orchestration tool to dynamically adjust the CPU, GPU, and memory quotas of the container instances, and horizontally expand and reduce the number of containers.

[0051] In a third aspect of the implementation of the present invention, a defect detection device based on artificial intelligence image processing is proposed as a further solution of the present invention, comprising an image sensor, a data acquisition interface, a memory, an image processing accelerator, a central processing unit, a deep learning acceleration unit, a data communication unit, a programmable memory, a display unit, and a power management unit;

[0052] The image sensor is responsible for converting optical signals into electrical signals for capturing high-resolution images; the data acquisition interface is responsible for transmitting the image data captured by the sensor to the processing unit;

[0053] The memory includes RAM and ROM, wherein the RAM is used to store temporary data and supports high-speed read and write operations, and the ROM is used to save the collected data and model files, which is convenient for long-term storage and fast reading;

[0054] The image processing accelerator is used to efficiently perform image preprocessing operations, the central processing unit is used to control the entire inspection process, coordinate data transmission, processing and storage, and perform some lightweight computing tasks, and the deep learning acceleration unit is used to quickly perform deep learning reasoning tasks, identify defect types and locations in images, and support model optimization and updating;

[0055] The data communication unit is used to transmit the detection results to external devices and systems; the programmable memory is used to store the device's operating system, defect detection algorithm and parameter configuration file; the display unit is used to display the detection results and the operating status of the device, and is composed of a touch screen to support user interaction; the power management unit provides a stable power supply for each electronic component in the device.

[0056] Beneficial effects of the present invention:

[0057] 1. The present invention dynamically selects the optimal combination of training parameters based on feedback during the training process, avoids redundant parameter debugging, shortens the model convergence time, and quickly skips training paths that are ineffective in improving model performance through a state transfer mechanism, thereby improving the utilization efficiency of training resources. It also gives higher learning priority to important training samples, enhances the generalization ability of the model, and simultaneously evaluates multiple potential optimization paths during the training process. The best training strategy is selected through long-term accumulated reward values, and these goals are comprehensively weighed through a multi-objective reward function to ensure the excellent performance of the model in diversified scenarios. When the defect type or detection task changes, the model's training strategy can be quickly adjusted to adapt to new task requirements without completely retraining, and automatically focuses on abnormal data or uncommon defect types to improve the performance of the model when processing rare scenarios. The boundary states that may lead to misjudgment are refined to ensure the robustness of the model in fine-grained classification tasks.

[0058] 2. The present invention trains a lightweight student model to inherit the performance of a large teacher model. It can significantly reduce the consumption of computing resources while maintaining high accuracy, ensure that the defect detection system can also run efficiently in resource-constrained devices, adapt to different hardware conditions, and expand the application scenario requirements of the system. The Docker container is used to encapsulate the model and all dependencies required for operation to form an independent and portable operating environment, isolate the dependency between the defect detection model and the host system, reduce operating problems caused by environmental conflicts, enhance the security of the system, avoid the impact of external malware attacks or resource leakage on system operation, and achieve rapid deployment of models on multiple operating systems and hardware platforms without repeated configuration of the environment. It is easy to expand and version manage. When the model needs to be updated or replaced, only a new container image needs to be deployed without reconfiguring the entire system. It also supports the parallel operation of multiple instances, and can achieve efficient load balancing between multiple detection tasks. It can quickly expand container instances to meet high-concurrency defect detection needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The present invention will be further described below in conjunction with the accompanying drawings.

[0060] Figure 1 A system block diagram of a defect detection system based on artificial intelligence image processing provided by an embodiment of the present invention;

[0061] Figure 2 A flowchart of a defect detection method based on artificial intelligence image processing provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0064] The embodiment of the present invention provides a defect detection system based on artificial intelligence image processing. Figure 1 , Figure 1 This is a system block diagram of a defect detection system based on artificial intelligence image processing provided by an embodiment of the present invention. The system includes the following modules:

[0065] The data acquisition module is used to acquire image data from industrial equipment, sensors, and cameras, and to perform preliminary formatting on the image data; the data preprocessing module is used to automatically identify and repair damaged areas in the image to ensure the quality of the input data; the data enhancement module is used to optimize data enhancement parameters to generate enhanced samples with high diversity and effectiveness.

[0066] Specifically, the parameter set of data enhancement is randomly initialized, including the rotation angle, scaling factor, shear coefficient, and noise level parameters. Based on the improvement of the sample on the training performance of the deep learning model, the objective function is set. ,in Represents the objective function value, which indicates the comprehensive evaluation of the enhanced sample. represents the diversity index of the enhanced samples, represents the effectiveness index of the enhanced sample, as well as Represents the weight coefficient, which is used to adjust the relative importance of diversity and effectiveness and set the initial temperature , termination temperature and cooling rate , , and based on the cooling strategy Perform cooling operation, Representative The temperature of the round iteration number, Represents the number of iterations, then randomly perturbs the current parameter set based on uniform distribution and Gaussian distribution, generates a new parameter set, calculates the objective function value of the new parameter set, and then calculates the change in the objective function value of the new parameter set and the current parameter set ,like , then replace the current parameter set with the new parameter set, otherwise, Calculate the probability of accepting a worse solution , and based on this probability, choose to accept the new parameter set. After each round of update, update the current temperature based on the cooling strategy For the next round of temperature , repeat the random perturbation and the new parameter set selection acceptance until the termination temperature is reached After the iteration, the optimal parameter set is output, and the preprocessed image data is enhanced based on the parameters in the optimal parameter set to generate an enhanced sample set.

[0067] The feature extraction module is used to extract feature information from the image; the feature optimization module is used to select a feature set that meets the defect detection requirements from the extracted multiple feature information combinations.

[0068] Specifically, create the initial root node , represents the set of all candidate features extracted by the feature extraction module , initialize the number of visits and cumulative evaluation values ​​of the root node, then start from the root node and gradually expand to new child nodes by randomly selecting feature information to build a search tree, where each child node represents a feature subset , the edge represents the selection from a feature subset to an extended subset, and the objective function is defined by the classification accuracy ,in represents a feature subset, and ,Starting from the root node, calculate the UCB value of each child node through the upper confidence interval formula, then select the node with the highest UCB value from large to small, and select the next node layer by layer until reaching the node that has not been fully expanded. On the current child node, randomly select an unexplored feature combination as a new child node, and initialize the number of visits and cumulative evaluation values ​​of the new child node. Starting from the expanded child node, randomly select features and construct feature subsets, and use the machine learning model to train and evaluate the performance of the feature subsets. Through the objective function Calculate the objective function value of the child node, trace the simulation results back to all the nodes passed, update the number of visits and cumulative evaluation values, repeat selection, expansion, simulation and backtracking to gradually expand and optimize the search tree until the preset maximum number of iterations is reached, and then select the feature subset corresponding to the child node with the most visits from the root node as the final required feature set.

[0069] In this embodiment, the specific calculation formula of the upper confidence interval formula is as follows:

[0070]

[0071] In the formula, Representative Child nodes The upper confidence limit value of ; Representative Child nodes The cumulative assessed value of Representative Child nodes Number of visits; Represents the balance coefficient controlling exploration and exploitation; Representative Child nodes The number of visits to the parent node.

[0072] The defect detection module is used to optimize the defect detection model, and use the model to classify and segment the input image, identify and mark the defect area.

[0073] Specifically, the status of the current defect detection model training process and the executable training operations are collected, and the state space of the defect detection model is constructed based on the two types of collected data. and action space , calculated in the state Perform any training operation After transfer to state The state transition probability , based on the impact of this training operation on the training quality, define the reward function , calculated in the state Execute training operation The feedback obtained included: , , , randomly initialize the initial state of the defect detection model , and record the initial reward value , and set the total number of steps in the training process and the early stopping condition, and then Status of the moment Execute training operation Transfer to next state , and evaluate the performance of the model on the validation set, and calculate the reward value in the current state ,based on Status of the moment , possible training actions , reward function and the transition probability , and calculate each state through the value iteration algorithm The value function To get the current state Take any training action The expected long-term return is calculated by recursively updating the value function using the Bellman equation to calculate each state The expected reward under each state is calculated and compared with each expected reward value to determine the training operation that should be taken in each state to maximize the expected reward value. , repeat the strategy update until the total number of steps in the preset training process is reached, or the model performance reaches the preset early stopping condition, that is, at each state The best action to choose , the strategy with the highest value function value ranked from large to small is selected, and the current defect detection model parameters are updated and trained based on the strategy.

[0074] The model deployment module is used to encapsulate the defect detection model, deploy and run it across platforms, and dynamically adjust computing resources through the resource scheduling mechanism;

[0075] The annotation and positioning module is used to annotate and spatially locate detected defects; the report generation module automatically generates inspection result reports in multiple formats; the adaptive optimization module is used to monitor the system operation status in real time and dynamically optimize the system's full-process configuration; the cloud collaboration module is used for cloud storage and collaboration of inspection data and models, and supports real-time sharing and analysis across teams; the user interaction module is used to provide an inspection result display interface, and also supports users to manually adjust parameters, view inspection results, and perform manual review.

[0076] The embodiment of the present invention also provides a defect detection method based on artificial intelligence image processing, such as Figure 2 As shown, the method comprises the following steps:

[0077] Acquire and preprocess high-resolution multi-source image data, automatically select enhancement methods and parameter combinations, and generate multiple groups of enhanced image samples.

[0078] The key features in the image are extracted, and each group of extracted key features is used as a candidate feature set, and the feature sets that meet the defect detection requirements are screened out.

[0079] Based on the screened feature set, the defect detection model is used to classify and segment the image, and the parameter configuration is dynamically optimized according to the feedback of the detection results.

[0080] The detected defects are accurately marked and spatially located, and the defect detection model that meets the requirements is encapsulated and deployed and run across platforms.

[0081] Specifically, the defect detection model is deployed in the target hardware environment, the model parameters are initialized, the trained weight file is loaded, the image size of the image to be detected is adjusted, and the adjusted image to be detected is normalized. The preprocessed image to be detected is input, and the feature information is extracted through the defect detection model. The extracted feature information is input into the classification and positioning network layer of the model. The classification network layer generates the prediction results of the defects and the probability of each defect category. The positioning network layer generates the bounding box of the defect area. The prediction results output by the model are then post-processed to generate defect detection results. According to the detected bounding box parameters, the geometric features of the defect area are extracted, and the specific size of the defect is calculated. According to the pixel coordinates in the image and the geometric calibration parameters of the camera, the position of the defect in the actual physical space is calculated. The camera intrinsic parameter matrix and extrinsic parameter matrix are used for projection transformation, and the image coordinates are converted into actual three-dimensional space coordinates. Then, the defect category, area and three-dimensional coordinates are merged into the image for annotation.

[0082] Specifically, a trained defect detection model is selected as the teacher model, and multiple groups of different lightweight models are constructed by reducing the number of network layer units and the network depth, and used as student models. The historical image data is input into the teacher model, and the predicted probability distribution is output. The temperature parameter is applied to the predicted probability distribution of the teacher model to generate a soft target, and the real label data is used as the hard target. The historical image data same as the teacher model is input into the student model. The student model processes the historical image data through the forward propagation algorithm and outputs the corresponding predicted probability distribution. The learning loss of the predicted probability distribution and the hard label is calculated through the cross entropy loss function, and the learning loss of the predicted probability distribution and the soft target is calculated through the KL divergence. The learning loss of the hard target and the learning loss of the soft target are weightedly aggregated to obtain the corresponding distillation loss function. During the training process, the input data passes through the teacher model and the student model in turn, and the distillation loss function is used to calculate the corresponding distillation loss value. , then the distillation loss value is passed layer by layer starting from the output layer of the student model according to the chain rule, and the gradient of the distillation loss value for each network layer of the model is calculated. The student model parameters are updated using the Adam algorithm. The unused historical image data is used as the verification data set. The performance of the student model is evaluated on the verification set to check whether the preset accuracy is achieved. If the performance is insufficient, the hyperparameters are adjusted and retrained. The trained student model is encapsulated as a Docker image, and the container dependencies are configured. A Dockerfile example is written to monitor the CPU usage, GPU usage, and memory usage performance data of each container in real time and store them in the monitoring system. When the monitoring data is greater than the usage limit, the number of container instances is increased and more resources are allocated. Otherwise, the number of container instances is reduced to save resources. The container orchestration tool is used to dynamically adjust the CPU, GPU, and memory quotas of the container instances, and to scale out and reduce the number of containers.

[0083] After the inspection is completed, the results are analyzed and a visual report is output. The defect detection results are displayed in real time through a graphical interface, and manual review and secondary marking are supported.

[0084] In another embodiment provided by the present invention, a defect detection device based on artificial intelligence image processing is also provided, including an image sensor, a data acquisition interface, a memory, an image processing accelerator, a central processing unit, a deep learning acceleration unit, a data communication unit, a programmable memory, a display unit and a power management unit.

[0085] The image sensor is responsible for converting light signals into electrical signals for capturing high-resolution images; the data acquisition interface is responsible for transmitting the image data captured by the sensor to the processing unit;

[0086] The memory includes RAM and SSD. RAM is used to store temporary data and supports high-speed read and write operations. ROM is used to save collected data and model files for long-term storage and fast reading.

[0087] The image processing accelerator is used to efficiently perform image preprocessing operations. The central processing unit is used to control the entire inspection process, coordinate data transmission, processing and storage, and perform some lightweight computing tasks. The deep learning acceleration unit is used to quickly perform deep learning reasoning tasks, identify the defect type and location in the image, and support model optimization and updating.

[0088] The data communication unit is used to transmit the test results to external devices and systems; the programmable memory is used to store the device's operating system, defect detection algorithm and parameter configuration files; the display unit is used to display the test results and the operating status of the device, and is composed of a touch screen to support user interaction; the power management unit provides a stable power supply for each electronic component in the device.

[0089] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. The defect detection system based on artificial intelligence image processing is characterized by: include: Data acquisition module, data preprocessing module, data enhancement module, feature extraction module, feature optimization module, defect detection module, model deployment module, annotation and positioning module, report generation module, adaptive optimization module, cloud collaboration module and user interaction module; The data acquisition module is used to obtain image data from industrial equipment, sensors and cameras, and perform preliminary formatting processing on the image data; The data preprocessing module is used to automatically identify and repair damaged areas in the image to ensure the quality of input data; The data enhancement module is used to optimize data enhancement parameters and generate enhanced samples with high diversity and effectiveness; The feature extraction module is used to extract feature information from the image; The feature optimization module is used to select a feature set that meets the defect detection requirements from the extracted multiple feature information combinations; The defect detection module is used to optimize the defect detection model, and classify and segment the input image through the model, and identify and mark the defect area; The model deployment module is used to encapsulate the defect detection model, deploy and run it across platforms, and dynamically adjust computing resources through a resource scheduling mechanism; The marking and positioning module is used to mark and spatially locate the detected defects; The report generation module automatically generates test result reports in various formats; The adaptive optimization module is used to monitor the system operation status in real time and dynamically optimize the full-process configuration of the system; The cloud collaboration module is used for cloud storage and collaboration of detection data and models, and supports real-time sharing and analysis across teams; The user interaction module is used to provide a test result display interface, and supports users to manually adjust parameters, view test results and perform manual review; The specific steps of optimizing the data enhancement parameters by the data enhancement module are as follows: S101: Randomly initialize the parameter set of data enhancement, including the rotation angle, scaling factor, shear coefficient and noise level parameters. Set the objective function based on the improvement of the deep learning model training performance by the sample. ,in Represents the objective function value, which indicates the comprehensive evaluation of the enhanced sample. represents the diversity index of the enhanced samples, represents the effectiveness index of the enhanced sample, as well as represents the weight coefficient, which is used to adjust the relative importance of diversity and effectiveness; S102: Set initial temperature , termination temperature and cooling rate , , and based on the cooling strategy Perform cooling operation, Representative The temperature of the round iteration number, Represents the number of iterations, after which the current parameter set is randomly perturbed based on uniform distribution and Gaussian distribution, and a new parameter set is generated; S103: Calculate the objective function value of the new parameter set, and then calculate the change in the objective function value of the new parameter set and the current parameter set ,like , then replace the current parameter set with the new parameter set, otherwise, Calculate the probability of accepting a worse solution , and choose to accept the new parameter set based on this probability; S104: After each round of update, the current temperature is updated based on the cooling strategy For the next round of temperature , repeat the random perturbation and the new parameter set selection acceptance until the termination temperature is reached , after the iteration, the optimal parameter set is output, and the preprocessed image data is enhanced based on the parameters in the optimal parameter set to generate an enhanced sample set; The specific steps of the feature optimization module selecting the required feature set from the extracted multiple feature information combinations are as follows: S201: Create the initial root node , represents the set of all candidate features extracted by the feature extraction module , initialize the number of visits and cumulative evaluation values ​​of the root node, then start from the root node and gradually expand to new child nodes by randomly selecting feature information to build a search tree, where each child node represents a feature subset , the edge represents the selection from a feature subset to an extended subset, and the objective function is defined by the classification accuracy ,in represents a feature subset, and ; S202: Starting from the root node, the UCB value of each child node is calculated by the upper confidence interval formula, and then the node with the highest UCB value is selected, and the next node is selected layer by layer until a node that has not been fully expanded is reached. On the current child node, a feature combination that has not been explored is randomly selected as a new child node, and the number of visits and cumulative evaluation values ​​of the new child node are initialized. The specific calculation formula of the upper confidence interval formula is as follows: In the formula, Representative Child nodes The upper confidence limit value of ; Representative Child nodes The cumulative assessed value of Representative Child nodes Number of visits; Represents the balance coefficient controlling exploration and exploitation; Representative Child nodes The number of visits to the parent node; S203: Starting from the expanded child node, randomly select features and construct feature subsets, and use the machine learning model to train and evaluate the performance of the feature subsets. Calculate the objective function value of the child node, trace the simulation results back to all the nodes passed, and update the number of visits and cumulative evaluation values; S204: Repeat the above steps S202 and S203 to gradually expand and optimize the search tree until the preset maximum number of iterations is reached, and then select the feature subset corresponding to the child node with the most visits from the root node as the final required feature set.

2. The defect detection system based on artificial intelligence image processing according to claim 1 is characterized in that: The specific steps of the defect detection module to optimize the defect detection model are as follows: S301: Collect the status of the current defect detection model training process and the executable training operations, and construct the state space of the defect detection model based on the two types of collected data and action space , calculated in the state Perform any training operation After transfer to state The state transition probability , based on the impact of this training operation on the training quality, define the reward function , calculated in the state Execute training operation The feedback obtained included: , , ; S302: Randomly initialize the initial state of the defect detection model , and record the initial reward value , and set the total number of steps in the training process and the early stopping condition, and then Status of the moment Execute training operation Transfer to next state , and evaluate the performance of the model on the validation set, and calculate the reward value in the current state ; S303: Based on Status of the moment , possible training actions , reward function and the transition probability , and calculate each state through the value iteration algorithm The value function To get the current state Take any training action The expected long-term return is calculated by recursively updating the value function using the Bellman equation to calculate each state The expected reward under each state is calculated and compared with each expected reward value to determine the training operation that should be taken in each state to maximize the expected reward value. ; S304: Repeat the strategy update until the total number of steps in the preset training process is reached, or the model performance reaches the preset early stopping condition, that is, in each state The best action to choose , the strategy with the highest value function value ranked from large to small is selected, and the current defect detection model parameters are updated and trained based on the strategy.

3. A defect detection method based on artificial intelligence image processing, used to implement a defect detection system based on artificial intelligence image processing as claimed in any one of claims 1 to 2, characterized in that: The following steps are involved: Q1. Acquire and preprocess high-resolution multi-source image data, automatically select enhancement methods and parameter combinations, and generate multiple sets of enhanced image samples; Q2. Extract key features from the image, use each group of extracted key features as a candidate feature set, and select the feature set that meets the defect detection requirements; Q3. Based on the selected feature set, the defect detection model is used to classify and segment the image, and the parameter configuration is dynamically optimized according to the feedback detection results; Q4. Accurately mark and spatially locate the detected defects, encapsulate defect detection models that meet the requirements, and deploy and run them across platforms; Q5. After the inspection is completed, the results are analyzed and a visual report is output. The defect detection results are displayed in real time through a graphical interface, and manual review and secondary marking are supported.

4. The defect detection method based on artificial intelligence image processing according to claim 3 is characterized in that: The specific steps for accurately marking and spatially locating the detected defects described in Q4 are as follows: S401: deploying the defect detection model to the target hardware environment, initializing the model parameters, loading the trained weight file, adjusting the image size of the image to be detected, normalizing the adjusted image to be detected, inputting the preprocessed image to be detected, and extracting feature information through the defect detection model; S402: Input the extracted feature information into the classification and localization network layers of the model. The classification network layer generates defect prediction results and the probability of each defect category. The localization network layer generates a bounding box of the defect area. The prediction results output by the model are then post-processed to generate defect detection results. S403: Extract the geometric features of the defect area based on the detected bounding box parameters, and calculate the specific size of the defect. Calculate the position of the defect in the actual physical space based on the pixel coordinates in the image and the geometric calibration parameters of the camera. Use the camera intrinsic parameter matrix and extrinsic parameter matrix for projection transformation, convert the image coordinates into actual three-dimensional space coordinates, and then merge the defect category, area and three-dimensional coordinates into the image for annotation.

5. The defect detection method based on artificial intelligence image processing according to claim 3 is characterized in that: The specific steps of the defect detection model that meets the requirements of Q4 are as follows: S501: Select the trained defect detection model as the teacher model, build multiple groups of different lightweight models by reducing the number of network layer units and the network depth, and use them as student models, input the historical image data into the teacher model, and output the predicted probability distribution, and apply the temperature parameter on the predicted probability distribution of the teacher model to generate soft targets, and use the real label data as hard targets; S502: Input the same historical image data as the teacher model into the student model. The student model processes the historical image data through the forward propagation algorithm and outputs the corresponding predicted probability distribution. The learning loss of the predicted probability distribution and the hard label is calculated through the cross entropy loss function. The learning loss of the predicted probability distribution and the soft target is calculated through the KL divergence. The learning loss of the hard target and the learning loss of the soft target are weighted and aggregated to obtain the corresponding distillation loss function. S503: During the training process, the input data passes through the teacher model and the student model in turn, and the distillation loss function is used to calculate the corresponding distillation loss value. Then, the distillation loss value is passed layer by layer starting from the output layer of the student model according to the chain rule, and the gradient of the distillation loss value for each network layer of the model is calculated, and the student model parameters are updated using the Adam algorithm; S504: Use the unused historical image data as a validation data set, evaluate the performance of the student model on the validation set, check whether the preset accuracy is achieved, and if the performance is insufficient, adjust the hyperparameters and retrain, encapsulate the trained student model as a Docker image, configure the container dependencies, and write a Dockerfile example; S505: Monitor the CPU usage, GPU usage, and memory usage performance data of each container in real time and store them in the monitoring system. When the monitoring data is greater than the usage limit, increase the number of container instances and allocate more resources. Otherwise, reduce the number of container instances to save resources. Use the container orchestration tool to dynamically adjust the CPU, GPU, and memory quotas of the container instances, and horizontally expand and reduce the number of containers.

6. A defect detection device based on artificial intelligence image processing, used to implement the defect detection system based on artificial intelligence image processing according to any one of claims 1-2, characterized in that: It includes an image sensor, a data acquisition interface, a memory, an image processing accelerator, a central processing unit, a deep learning acceleration unit, a data communication unit, a programmable memory, a display unit, and a power management unit; The image sensor is responsible for converting optical signals into electrical signals for capturing high-resolution images; The data acquisition interface is responsible for transmitting the image data captured by the sensor to the processing unit; The memory includes RAM and ROM, wherein the RAM is used to store temporary data and supports high-speed read and write operations, and the ROM is used to save the collected data and model files, which is convenient for long-term storage and fast reading; The image processing accelerator is used to efficiently perform image preprocessing operations, the central processing unit is used to control the entire inspection process, coordinate data transmission, processing and storage, and perform some lightweight computing tasks, and the deep learning acceleration unit is used to quickly perform deep learning reasoning tasks, identify defect types and locations in images, and support model optimization and updating; The data communication unit is used to transmit the detection results to external devices and systems; the programmable memory is used to store the device's operating system, defect detection algorithm and parameter configuration file; the display unit is used to display the detection results and the operating status of the device, and is composed of a touch screen to support user interaction; the power management unit provides a stable power supply for each electronic component in the device.

Images