Defect detection method and electronic equipment

Abstract

The invention discloses a defect detection method and electronic equipment, and relates to the technical field of industrial assembly quality detection, and the method comprises the steps: obtaining rotation data of assembly equipment on a target assembly station, inputting the rotation data into an equipment fault early warning model, and carrying out the equipment fault detection, and obtaining an appearance image of an assembly object, inputting the appearance image into an appearance defect detection model for assembly object defect detection, determining a defect risk value of a target assembly station according to the defect severity and the equipment fault risk degree output by the model, and triggering defect and fault linkage alarm when the defect risk value is greater than an alarm threshold. According to the invention, linkage detection of the assembly object defect and the assembly equipment fault based on double-model cooperation and a multi-modal data dynamic fusion mechanism is realized, and the defect detection precision of the assembly object and the early warning timeliness of the assembly equipment fault are improved. The technical problems that the defect omission ratio is high and faults are not found in time when appearance detection and equipment fault detection are carried out manually in the assembly link can be solved.

Description

Technical Field

[0001] This application relates to the field of industrial assembly quality inspection technology, and in particular to a defect detection method and electronic equipment. Background Technology

[0002] In the field of intelligent server manufacturing, quality control in the assembly process directly determines the operational stability and lifespan of the server. Defects in key processes such as motherboard interface alignment, screw tightening, and cable connection (such as interface misalignment, missing screws, and loose cables), as well as sudden failures of assembly equipment (such as screw machines and interface plugging / unplugging machines), are the core issues leading to server factory repairs and rising maintenance costs.

[0003] However, in related technologies, the method of relying on manual visual inspection of server appearance defects and equipment malfunctions on server assembly lines has the problems of high defect miss rate and untimely fault detection. Summary of the Invention

[0004] This application provides a defect detection method and electronic device to at least solve the problems of high defect omission rate and untimely fault detection in the related art when manual assembly process is used for appearance inspection and equipment fault detection.

[0005] This application provides a defect detection method, including: Acquire rotation data of the assembly equipment at the target assembly station and an image of the assembly object's appearance; The appearance image is input into a pre-trained appearance defect detection model, and the defect severity output by the appearance defect detection model is obtained. The rotation data is input into a pre-trained equipment fault early warning model, and the equipment fault risk level output by the equipment fault early warning model is obtained. Based on the defect severity and the equipment failure risk, the defect risk value corresponding to the target assembly station is determined; If the defect risk value is greater than the alarm threshold, a defect and fault linkage alarm will be triggered.

[0006] This application also provides a defect detection device, including: The first acquisition module is used to acquire rotation data of the assembly equipment at the target assembly station and appearance images of the assembly object. The second acquisition module is used to input the appearance image into a pre-trained appearance defect detection model and acquire the defect severity output by the appearance defect detection model. The third acquisition module is used to input the rotation data into a pre-trained equipment fault early warning model and acquire the equipment fault risk level output by the equipment fault early warning model. The risk determination module is used to determine the defect risk value corresponding to the target assembly station based on the defect severity and the equipment failure risk. The alarm module is used to trigger a defect and fault linkage alarm when the defect risk value is greater than the alarm threshold.

[0007] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described defect detection methods.

[0008] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described defect detection methods.

[0009] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described defect detection methods.

[0010] This application trains a visual defect detection model and an equipment fault early warning model separately. Rotation data of the assembly equipment at the target assembly station is input into the equipment fault early warning model for equipment fault detection to obtain the equipment fault risk level. Similarly, visual images of the assembly object are input into the visual defect detection model for assembly object defect detection to obtain the defect severity. Based on the defect severity and equipment fault risk level, the defect risk value corresponding to the target assembly station is determined. When the defect risk value exceeds the alarm threshold, a linked alarm for defects and faults is triggered. This achieves linked detection of assembly object defects and assembly equipment faults based on a dual-model collaboration and multi-modal data dynamic fusion mechanism. It forms a linkage mechanism between assembly equipment management and assembly object defect detection, breaking the limitation of data separation between assembly object defects and assembly equipment. This enables automated real-time detection of assembly object defects and assembly equipment faults, improving the accuracy of assembly object defect detection and the timeliness of assembly equipment fault early warning. Therefore, it solves the technical problems of high defect miss rates and untimely fault detection in manual visual inspection and equipment fault detection during assembly, achieving the technical effect of improving defect detection accuracy and fault early warning timeliness. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1A schematic diagram of the application environment architecture of a defect detection method according to an exemplary embodiment of this application is shown; Figure 2 A schematic flowchart of a defect detection method provided for an exemplary embodiment of this application; Figure 3 A flowchart illustrating a defect detection method provided as another exemplary embodiment of this application; Figure 4 A flowchart illustrating a defect detection method provided in yet another exemplary embodiment of this application; Figure 5 A schematic flowchart of a defect detection method provided in yet another exemplary embodiment of this application; Figure 6 This is a schematic diagram of the structure of a defect detection device provided in an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0014] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0015] Currently, the quality inspection of assembly objects (such as servers) and the management of assembly equipment still face multiple challenges. Taking server assembly as an example, traditional server assembly lines rely on manual visual inspection of the appearance defects of the servers being assembled. Server assembly involves more than 200 key components, and manual inspection of each component takes 3-5 minutes per unit. When the production line capacity increases to more than 30 units per hour, manual inspection is prone to missed inspections due to fatigue (such as traces of stripped screws or tiny interface gaps). The defect missed inspection rate is generally higher than 5%. Moreover, manual inspection cannot predict potential failures of assembly equipment (such as the gradual decrease of screw machine torque). It can only be dealt with passively after defects such as loose screws appear in batches, resulting in an average of more than 20 defective products per failure, causing waste of raw materials and labor time.

[0016] To address the problems with manual inspection, some fault detection solutions based on machine learning exist. These solutions rely on a single model, typically employing deep learning models (such as the ResNet series) for appearance defect detection. They use industrial cameras to capture images of assembled components on servers, process the image data using traditional fixed-threshold segmentation techniques, and then input the data into the model for defect identification. However, for assembly equipment management, there is no linkage mechanism with defect detection; equipment operation is monitored solely through manual recording or simple sensor data. Some solutions attempt to deploy traditional deep learning models (such as ResNet-50) on edge devices (such as ordinary industrial control computers) or rely on cloud inference to automate inspection. Furthermore, for different server models (such as rack-mount 2U / 4U and tower servers), a full retraining of the model is used to adapt to differences in component layout.

[0017] It is evident that existing inspection solutions still have shortcomings, including: server assembly involves more than 200 key components, and manual inspection of each component is time-consuming, inefficient, and difficult to adapt to high-volume production demands. Furthermore, human fatigue can easily lead to missed defects (such as stripped screw threads or tiny interface gaps), with a defect miss rate generally exceeding 5%. Additionally, it cannot predict potential equipment failures (such as the gradual decrease in screw machine torque), allowing only reactive handling after defects appear in batches, resulting in wasted raw materials and labor. The adaptability of single-model inspection solutions is also weak. Metal components (screws, interface shells, heat sinks) account for over 60% of server assembly, and industrial cameras are prone to localized high-brightness reflections (pixel values ​​close to 255) due to changes in workshop lighting angles. Traditional fixed-threshold segmentation techniques cannot effectively suppress these reflections, causing the model to mistakenly identify reflective areas as interface gaps or conceal loose screws, thus revealing defects. With a false detection rate exceeding 8%, these solutions only focus on assembly defects that have already occurred, without linking them to the operational data of the assembly equipment. When a defect is detected, it is impossible to trace the root cause of the fault (e.g., interface misalignment may be caused by positioning deviation of the interface machine or human error), requiring additional time for investigation, which affects the efficiency of the production line. The obstacles to their implementation are also quite obvious. Traditional deep learning models (such as ResNet-50 with 25.6 million parameters) have high inference time, and the inference speed on edge devices (such as ordinary industrial control computers) is only 5-8fps, which cannot meet the real-time requirements. If cloud inference is relied upon, fluctuations in the workshop network can easily cause detection delays of more than 100ms, missing the opportunity to correct defects in time. In addition, the server models are iterated quickly, and traditional models need to be fully retrained for new models. Training requires professional algorithm personnel to operate, which is difficult to adapt to the rapid model changeover requirements of the production line, resulting in a detection coverage rate of less than 70% in the early stage of new model production.

[0018] Effective server assembly quality inspection and fault early warning hinges on the ability to accurately identify assembly defects, predict equipment failures in advance, and adapt to the real-time requirements of edge computing scenarios and the flexible iteration needs of multiple server models. To address some or all of the problems of traditional solutions and improve the accuracy, real-time performance, and scenario adaptability of server assembly inspection, this application proposes a multimodal data fusion and edge intelligent collaboration technical solution. This solution integrates IoT device data collection, dual-model collaborative optimization, and dynamic weight fusion into a unified detection framework. Employing IoT collaboration and dual-model collaboration technologies, it achieves accurate defect detection and introduces an attention mechanism to achieve adaptive feature fusion, making it highly effective in anomaly detection and real-time fault detection. Unlike traditional single-model detection schemes, the method in this application is deeply adapted to the industrial edge scenario of server assembly. This scheme designs a dual-model collaboration, dynamic weight fusion, and lightweight incremental learning mechanism, which makes it significantly different from similar traditional solutions in the field of server assembly inspection. By using a multimodal dynamic fusion mechanism to break the limitation of the separation between defect and equipment data, and combining edge deployment optimization and incremental learning technology, compared with traditional methods, it not only improves the accuracy of defect detection and the timeliness of fault early warning, but also meets the needs of real-time inference at the edge and flexible adaptation to multiple models. Through the deep integration of IoT devices, deep learning models, and edge intelligence technology, this application can discover assembly defects faster, predict equipment failures earlier, and reduce the adaptation cost of new models, thereby meeting the high-quality and high-efficiency production requirements in the complex environment of intelligent server manufacturing.

[0019] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] The specific application environment architecture or specific hardware architecture on which the defect detection method depends is described here.

[0021] Figure 1 A schematic diagram of the application environment architecture of a defect detection method according to an exemplary embodiment of this application is shown, such as... Figure 1 As shown, the architecture includes multiple assembly stations and an edge server. Each assembly station deploys IoT devices to collect and preprocess visual video streams of the assembled object (e.g., a server) and operational data (e.g., torque / speed) of the assembly equipment. The edge server ensures low-latency response detection. The edge server integrates the defect detection method of this application. By executing this defect detection method, the visual video streams and operational data transmitted from the IoT devices at each assembly station are processed in real time to detect defects and equipment malfunctions. Compared to cloud servers, using an edge server to execute the defect detection method of this application ensures the timeliness of defect detection.

[0022] Embodiments of this application provide a defect detection method, which can be executed by a defect detection device provided in the embodiments of this application. This device can be implemented in software and / or hardware and can be integrated into an electronic device, such as... Figure 1 The edge server in the process. The method will be described in detail below, taking into account the execution flow of the defect detection method.

[0023] Figure 2 A flowchart illustrating a defect detection method provided in an exemplary embodiment of this application is shown below. Figure 2 As shown, the defect detection method may include the following steps: Step 101: Obtain the rotation data of the assembly equipment at the target assembly station and the appearance image of the assembly object.

[0024] For key assembly stations on the assembly line (such as motherboard assembly, cable connection stations, etc.) or all assembly stations, IoT devices can be deployed at these stations. These IoT devices include a 4K industrial AI camera (25fps frame rate, 12mm lens focal length, focusing on key areas such as motherboard interfaces and screws) and a torque / speed sensor (8Hz sampling frequency). The 4K industrial AI camera is used to capture video streams of the appearance of the objects being assembled (e.g., servers) at the assembly station. The torque / speed sensor can be either a torque sensor or a speed sensor, used to collect the torque or speed of the assembly equipment at the assembly station. The target assembly station can be any of the aforementioned assembly stations. The system acquires rotational data of the assembly equipment at the target assembly station (e.g., torque collected by the torque sensor or speed collected by the speed sensor), and obtains an appearance image of the object being assembled at the target assembly station. The appearance image is a video frame from the appearance video stream captured by the 4K industrial AI camera at the target assembly station.

[0025] It should be noted that in this embodiment, the acquired appearance image and rotation data are bound by timestamp (accurate to the second) and workstation number (such as motherboard assembly workstation 01) to ensure that the data in the same scene correspond.

[0026] Step 102: Input the appearance image into the pre-trained appearance defect detection model and obtain the defect severity output by the appearance defect detection model.

[0027] The appearance defect detection model is pre-trained. This trained model can detect defects based on input data and output the severity of defects in the assembled object, where the severity is quantified as a value between 0 and 1. Understandably, the appearance defect detection model can also be trained to output other data, such as specific defect types, as needed.

[0028] As an example, several labeled images can be collected as training samples, including negative samples with defects and positive samples without defects. The commonly used deep learning model is trained, and the model loss value is calculated based on the severity of defects output by the model and the severity of defects labeled. When the model loss value is less than the preset value or the model accuracy verified by the validation set reaches a certain value, the training is completed, and a trained appearance defect detection model is obtained.

[0029] As an example, a lightweight ResNet model with an assembly feature attention module can be constructed and trained to obtain an appearance defect detection model. Specifically, based on the ResNet-18 residual connection network, a new region-part dual attention module can be designed after the conv3 layer, and a lightweight convolutional neural network architecture, MobileNetV2, can be connected after the region-part dual attention module to construct the initial model. Region attention targets key areas at the workstation level (such as the motherboard interface area), while part attention targets part-level details (such as a single screw head). The two work together to improve feature extraction accuracy. Region attention strengthens the weights of key areas such as interfaces and screws, while part attention focuses on defect-prone parts such as connectors and heat sinks. The inverse residual structure of MobileNetV2 compresses the number of parameters to 2.8 million, significantly reducing the model's network parameters. Appearance images of assembled objects (such as servers) at different assembly stages are collected, including normal images and images with defects. For example, the training dataset contains 6000 labeled images (including 8 types of defects such as interface misalignment and missing screws), and the dataset is divided in a 7:2:1 ratio. The initial model described above was trained using the collected training dataset. To ensure stable learning and avoid overfitting, the AdamW optimizer was used for model training, with an initial learning rate of 8e. -5The model employs a focal loss function with defect sample weights α=0.3 and a loss ratio γ=2.5. It is trained for 60 epochs with the following training objectives: validation set accuracy > 98.5% and Intersection over Union (IOU) > 0.8. During training, the model parameters are adjusted and optimized by calculating the model's loss value. The loss value is calculated based on the defect type output by the model and the defect type labeled in the samples. The model also outputs the defect severity based on the predicted defect type (e.g., a severity of 0.9 for a severe defect and 0.4 for a mild defect). Training is complete when the expected training objectives are achieved, resulting in a trained appearance defect detection model capable of detecting appearance defects and outputting the defect type and severity.

[0030] In this embodiment, the acquired appearance image is input into the appearance defect detection model, which then performs appearance defect detection on the assembly object in the appearance image and outputs the predicted defect severity.

[0031] In one optional embodiment of this application, before inputting the appearance image into the appearance defect detection model, the appearance image can be preprocessed with operations such as grayscale conversion, metal reflection suppression (adaptive threshold segmentation), and edge enhancement to improve the accuracy of defect detection.

[0032] Step 103: Input the rotation data into the pre-trained equipment fault early warning model and obtain the equipment fault risk level output by the equipment fault early warning model.

[0033] The equipment fault early warning model is pre-trained. This trained model can predict equipment faults based on input data and output the equipment fault risk level of the assembly equipment at the assembly station. The equipment fault risk level is quantified as a value between 0 and 1. Understandably, the model can also be trained to output its data capabilities, such as fault types and their probability distributions (e.g., the probability of abnormal torque in a screw machine is 85%), and the fault type is used to label the associated equipment number in the report generation, achieving a seamless connection between fault early warning, risk assessment, and decision support.

[0034] As an example, a TCN-LSTM hybrid model combining Temporal Convolutional Network (TCN) and Long Short-Term Memory (LSTM) can be optimized. The TCN layer is designed with two convolutional blocks (with a kernel size of 5 and dilation coefficients of 1 and 3) to extract short-cycle fluctuation features of torque / speed data. The LSTM layer contains two hidden layers (80 units per layer) to capture long-cycle trends of torque / speed data, enabling fault warning 45 minutes in advance. Torque or speed data from each assembly unit is collected as training data. The training data includes 1200 normal samples and 600 fault samples (fault types such as loose screws, loose connections, etc.). Each sample includes multiple speed / torque data points collected within a time period (e.g., 15 seconds). 25-dimensional features are extracted in 15-second windows. For example, if the sampling frequency of the torque or speed sensor is 8Hz, then one sample contains data collected within 15 seconds, resulting in 15 × 8 = 120 torque / speed values. A Stochastic Gradient Descent (SGD) optimizer (learning rate 4e) is used. -3 The training process consisted of 35 rounds, with the training objectives set as follows: test set accuracy > 97% and false alarm rate < 3%. A TCN-LSTM hybrid model was trained using the collected training data. During training, the model parameters were adjusted and optimized by calculating the model loss. The loss calculation was based on the fault type, fault risk value, and labeled fault type and risk value in the model output. Training was completed when the expected training objective was achieved, resulting in the equipment fault warning model. Testing showed that the trained equipment fault warning model could provide fault warnings 45 minutes in advance.

[0035] In this embodiment, the rotation data of the acquired assembly equipment is input into the equipment fault early warning model. The equipment fault early warning model analyzes the input rotation data and predicts whether the equipment has a fault, and outputs the predicted equipment fault risk level.

[0036] In one optional embodiment of this application, before inputting the acquired rotation data into the device fault early warning model, the rotation data can be standardized to the [0,1] interval, and outliers can be removed using the 3σ principle to improve the accuracy of fault detection.

[0037] Step 104: Determine the defect risk value corresponding to the target assembly station based on the defect severity and equipment failure risk.

[0038] In this embodiment, the defect severity output by the appearance defect detection model and the equipment failure risk level output by the equipment failure early warning model are obtained. Further, the defect risk value corresponding to the target assembly station is determined based on the defect severity and equipment failure risk level, realizing the fusion of multimodal data. The existence of risk is judged based on the multimodal data fusion result, and the fused data comes from the model output results in the aforementioned steps, ensuring that the data source is traceable.

[0039] As an example, the sum, mean, or larger of the defect severity and equipment failure risk can be determined, and the determined sum, mean, or larger value can be used as the defect risk value corresponding to the target assembly station.

[0040] As an example, corresponding weights can be set in advance for defect severity and equipment failure risk, and the two can be weighted and summed based on the weights. The result can be used as the defect risk value corresponding to the target assembly station.

[0041] Step 105: If the defect risk value is greater than the alarm threshold, trigger a defect and fault linkage alarm.

[0042] The specific value of the alarm threshold can be set according to actual needs. The alarm threshold may also vary depending on the method used to determine the defect risk value. For example, for a scenario where the defect risk value is determined by weighted summation, the alarm threshold can be set to 0.7.

[0043] In this embodiment, the defect risk value is compared with the alarm threshold. If the defect risk value is greater than the alarm threshold, a defect and fault linkage alarm is triggered. For example, the defect and fault linkage alarm may include at least one of the following alarm methods: synchronously pushing defect images to the assembly inspection management web system, notifying technicians via SMS, and displaying repair screen instructions on the display screen of the target assembly station.

[0044] The defect detection method in this application embodiment trains an appearance defect detection model and an equipment fault early warning model separately. It obtains rotation data of the assembly equipment at the target assembly station and inputs it into the equipment fault early warning model to detect equipment faults and obtain the equipment fault risk level. It also obtains appearance images of the assembly object and inputs them into the appearance defect detection model to detect defects in the assembly object and obtain the defect severity. Then, based on the defect severity and equipment fault risk level, it determines the defect risk value corresponding to the target assembly station. When the defect risk value exceeds the alarm threshold, it triggers a linked alarm for defects and faults. Thus, it achieves linked detection of assembly object defects and assembly equipment faults based on a dual-model collaboration and multi-modal data dynamic fusion mechanism, forming a linkage mechanism between assembly equipment management and assembly object defect detection. This breaks the limitation of data separation between assembly object defects and assembly equipment, realizing automated real-time detection of assembly object defects and assembly equipment faults. It improves the accuracy of assembly object defect detection and the timeliness of assembly equipment fault early warning. Therefore, it can solve the technical problems of high defect miss rates and untimely fault detection in manual appearance inspection and equipment fault detection during the assembly process, achieving the technical effect of improving defect detection accuracy and fault early warning timeliness.

[0045] In one alternative embodiment of this application, such as Figure 3 As shown, based on the aforementioned embodiments, the step of obtaining the appearance image of the assembly object in step 101 may include the following sub-steps: Step 201: Obtain the original image of the assembly object captured by the industrial camera deployed at the target assembly station.

[0046] In this embodiment, an industrial camera is installed on the target assembly station to collect images or video streams of the assembled objects at the station. When the industrial camera collects an image, the image collected by the industrial camera is used as the original image. When the industrial camera collects a video stream, the most recent video frame in the video stream is used as the original image.

[0047] Step 202: Convert the original image to grayscale to obtain a grayscale image.

[0048] In this embodiment, the acquired original image is converted to grayscale, transforming the original color image into a grayscale image. The grayscale conversion can be implemented using commonly used methods such as the maximum value method, the average value method, and the weighted average method; this application does not limit the specific grayscale conversion method used.

[0049] Step 203: Perform metal reflection suppression processing on the grayscale image to obtain a reflection-suppressed image.

[0050] In this embodiment, after obtaining the grayscale image, the grayscale image can be processed to suppress metallic reflection, resulting in a reflection-suppressed image.

[0051] As an example, pixels with grayscale values ​​exceeding a threshold in a grayscale image can be suppressed by reducing their grayscale values ​​to a range close to normal metallic grayscale (typically 180-200), resulting in a reflection-suppressed image. Industrial cameras often produce localized bright reflections (pixel values ​​close to 255) due to variations in workshop lighting angles. Therefore, in this example, suppressing grayscale values ​​exceeding the threshold to a range close to normal metallic grayscale can suppress metallic reflections to some extent.

[0052] As an example, to improve the effect of suppressing metallic reflections, the grayscale image can be divided into blocks. A threshold is determined for each block, and reflection suppression is performed based on this threshold. Compared to suppressing reflections across the entire grayscale image using a single threshold, this method effectively improves the reflection suppression effect. Specifically, the grayscale image can be divided into multiple image blocks. Based on the pixel grayscale values ​​of multiple pixels in each image block, a local pixel mean is determined for each block. That is, for each image block, the local pixel mean is determined based on the pixel grayscale values ​​of each pixel within that block, and each image block has its own local pixel mean. Then, based on the local pixel mean and a preset offset, a grayscale threshold is determined for each image block. That is, for each image block, its own grayscale threshold is determined using the local pixel mean and the preset offset. Grayscale threshold = local pixel mean - preset offset. The specific value of the preset offset can be set according to actual needs, for example, setting a preset offset... The quantity is 8; identify target pixels in multiple image blocks whose pixel grayscale values ​​are greater than the corresponding grayscale threshold, and adjust the pixel grayscale values ​​of the target pixels to the target grayscale range to obtain multiple reflection suppression image blocks. That is, for each image block, compare the pixel grayscale values ​​of each pixel in the image block with the grayscale threshold of the image block itself, and adjust the pixel grayscale values ​​of pixels whose pixel grayscale values ​​are greater than the grayscale threshold to any grayscale value within the target grayscale range. After traversing all pixels in the image block, a reflection suppression image block is obtained. The target grayscale range is determined based on the normal metallic grayscale. Set the target grayscale range to a grayscale range close to the normal metallic grayscale, for example, set the target grayscale range to 180-200; generate a reflection suppression image based on multiple reflection suppression image blocks.

[0053] For example, assuming the grayscale image size is 224×224, it is divided into multiple sub-blocks in 8×8 format. Each sub-block has its own grayscale threshold calculated by subtracting a preset offset from the local pixel mean (e.g., 230-8=222 for reflective areas and 150-8=142 for non-reflective areas). Then, reflection is located and grayscale is adjusted. If the pixel grayscale value exceeds the grayscale threshold of the sub-block, it is determined to be a reflection, and its grayscale is reduced to 180-200 (close to normal metallic grayscale). If it does not exceed the threshold, it remains unchanged.

[0054] In this embodiment, by dividing the image into blocks to obtain multiple image blocks, determining the exclusive threshold for each image block, and adjusting the gray values ​​of pixels in each image block that exceed their exclusive threshold to a target gray range that is close to the normal metallic gray value, the metallic reflection of the image is suppressed, the reflection suppression effect is improved, a high-quality data source is provided for defect detection, and the foundation for accurate defect detection is laid.

[0055] Step 204: Perform edge enhancement processing on the reflection suppression image to obtain the appearance image.

[0056] In this embodiment, the obtained reflection-suppressed image can be further processed with edge enhancement, and the processed image can be used as the appearance image to ensure the image quality after reflection suppression.

[0057] For example, the obtained reflection-suppressed image can be smoothed at the edges and details restored. Gaussian blurring can be used to eliminate threshold jumps in the blocks, and contrast stretching ([100,200] mapped to [50,220]) can be used to restore defect details such as screw gaps. This algorithm has low complexity and does not affect real-time monitoring. The processed image data (e.g., if the original grayscale image is 224×224 pixels, the processed image will still be 224×224 pixels, with reflections suppressed) can be directly used as input to the appearance defect detection model.

[0058] The defect detection method of this application embodiment obtains a grayscale image by performing grayscale processing on the acquired original image, and then performs metal reflection suppression processing on the grayscale image to obtain a reflection suppression image. Finally, the reflection suppression image is subjected to edge enhancement processing to obtain an appearance image. This results in the final appearance image not only suppressing reflection but also enhancing edges, ensuring the image quality of the appearance image, providing a high-quality data source for the defect detection model, and helping to improve the accuracy of defect detection.

[0059] In one alternative embodiment of this application, such as Figure 4 As shown, based on the foregoing embodiments, step 104 may include the following sub-steps: Step 301: Determine the assembly defect weight based on the defect severity.

[0060] As an example, different severity ranges can be preset, with each severity range corresponding to an assembly defect weight. For the defect severity output by the appearance defect detection model, the corresponding assembly defect weight can be determined based on the severity range to which it belongs.

[0061] As an example, a severity threshold can be pre-set according to actual needs. The severity of defects output by the appearance defect detection model is compared with this severity threshold. If the defect severity is greater than or equal to the severity threshold, the assembly defect weight is determined as the first weight. If the defect severity is less than the severity threshold, the assembly defect weight is determined as the second weight. The first weight is greater than the second weight. Thus, dynamic weight allocation is achieved through defect severity classification. Higher weights are assigned to defects with higher severity, making the multimodal data fusion focus more on defects. Dynamically allocating corresponding weights according to defect severity improves the flexibility of the solution and the rationality of weight selection.

[0062] Step 302: Determine the equipment failure weight based on the assembly defect weight.

[0063] In this embodiment, after determining the assembly defect weight, the equipment failure weight can be determined based on the assembly defect weight, where the equipment failure weight = 1 - assembly defect weight. For example, when a serious defect such as interface misalignment is detected, the assembly defect weight corresponding to the defect severity is set to 0.8, and the equipment failure weight is set to 0.2; when a minor defect such as loose cable is detected, the assembly defect weight is adjusted to 0.5, and the equipment failure weight is adjusted accordingly to 0.5.

[0064] Step 303: Based on the defect severity, assembly defect weight, equipment failure risk degree and equipment failure weight, perform a weighted summation to obtain the defect risk value corresponding to the target assembly station.

[0065] In this embodiment, the defect risk value corresponding to the target assembly station is equal to the weighted value of the defect severity and assembly defect weight, and the sum of the weighted values ​​of the equipment failure risk and equipment failure weight. That is, the defect risk value corresponding to the target assembly station = defect severity × assembly defect weight + equipment failure risk × equipment failure weight.

[0066] The defect detection method in this application determines the assembly defect weight based on the defect severity and the equipment failure weight based on the assembly defect weight. Then, it performs a weighted summation based on the defect severity, assembly defect weight, equipment failure risk level, and equipment failure weight to obtain the defect risk value corresponding to the target assembly station. Thus, a dynamic weighting mechanism based on defect severity to determine the corresponding weight is realized, and the equipment failure weight is adjusted accordingly. This achieves deep fusion of multimodal data based on the dynamic weighting mechanism, making the fused defect risk value more focused on the factors with greater influence, which helps to improve the rationality of the defect risk value and trigger timely alarms.

[0067] In one optional embodiment of this application, the network structure of the appearance defect detection model includes a residual neural network, a region attention module, a component attention module, and a lightweight convolutional neural network model. The residual neural network can be ResNet-18. The region attention module and the component attention module can be connected after the conv3 layer of the residual neural network. The region attention module enhances the extraction of features from defect-prone areas such as interfaces and screws, while the component attention module focuses on defect-prone components such as connectors and heat sinks. The lightweight convolutional neural network model can be MobileNetV2. The input layer of MobileNetV2 is connected to the outputs of the region attention module and the component attention module. The inverse residual structure of MobileNetV2 can compress the number of parameters to 2.8 million, thereby balancing detection accuracy and edge-end inference efficiency. Therefore, in this embodiment, when inputting the appearance image into a pre-trained appearance defect detection model and obtaining the defect severity output by the appearance defect detection model, the appearance image is first input into a residual neural network for convolution processing to obtain the appearance feature map corresponding to the appearance image; then, the appearance feature map is input into a region attention module for feature extraction to obtain workstation-level region features, and the appearance feature map is input into a component attention module for feature extraction to obtain component-level detail features; then, the workstation-level region features and component-level detail features are input into a lightweight convolutional neural network model for defect detection to obtain the defect severity output by the lightweight convolutional neural network model.

[0068] In this embodiment, by connecting a region attention module and a component attention module after the residual neural network, the region attention module strengthens the feature extraction of easily defective areas such as interfaces and screws, while the component attention module focuses on easily defective components such as connectors and heat sinks. The two work together to improve the feature extraction accuracy, thereby ensuring the accuracy of feature extraction and improving the accuracy of defect detection results. Furthermore, by using a lightweight convolutional neural network model to perform defect detection on the extracted workstation-level region features and component-level detail features, the number of model parameters can be compressed to a large extent, thereby improving defect detection efficiency. This allows the appearance defect detection model to balance detection accuracy and edge-end inference efficiency.

[0069] In one optional embodiment of this application, the defect detection model can also output specific defect types. For component omission defects, defect tracing can be performed based on the designed dual-model collaborative detection framework and defect-fault tracing mechanism to reduce the recurrence of the same type of defect. For example, when the appearance defect detection model detects a screw omission defect, the equipment fault early warning model will backtrack the torque / speed data of the screw machine at that station for the past 10 minutes. If an abnormality is found where the torque is below a threshold, the equipment fault is automatically marked as the root cause of the defect. Thus, as... Figure 5 As shown, based on the foregoing embodiments, the defect detection method of this application may further include the following steps: Step 401: Obtain the defect type output by the appearance defect detection model.

[0070] In this embodiment of the application, the trained appearance defect detection model can not only output the severity of the defect, but also the specific defect type. Thus, after inputting the appearance image into the appearance defect detection model, the defect type output by the appearance defect detection model can be obtained.

[0071] Step 402: In the case of defect type of missing component, obtain the target rotation data of the assembly equipment within the first preset time period.

[0072] The specific value of the first preset time period can be set according to actual needs, for example, it can be set to 10 minutes.

[0073] In this embodiment, when the defect type output by the appearance defect detection model is missing component, the historical rotation data of the assembly equipment within the first preset time period (referred to as target rotation data) is obtained to trace the source of defects and faults.

[0074] It is understandable that the appearance image is bound to the assembly station number. When a defect of missing part is detected in an input appearance image, the corresponding assembly station can be determined based on the assembly station number bound to the external image. Then, the historical rotation data of the assembly equipment at that assembly station can be traced back to detect whether the root cause of the missing part is a failure of the assembly equipment.

[0075] Step 403: Input the target rotation data into the equipment fault early warning model and obtain the fault root cause confidence score output by the equipment fault early warning model. The fault root cause confidence score is used to characterize the credibility that the abnormal rotation data of the assembly equipment is the direct cause of the missing parts.

[0076] In this embodiment of the application, the target rotation data is input into the equipment fault early warning model. The equipment fault early warning model processes and analyzes the target rotation data and outputs the confidence level of the fault root cause. The higher the confidence level of the fault root cause, the more likely the component omission fault is caused by abnormal rotation data of the assembly equipment.

[0077] As an example, after inputting the target rotation data into the equipment fault early warning model, the model first cleans the backtracked target rotation data, removing abnormal noise such as equipment start-up and shutdown, signal interference, etc., and retaining the effective torque / speed sample values. Next, time-series feature extraction is performed, where TCN is responsible for capturing local trends in the target rotation data (such as sudden drops or persistently low values), and LSTM is responsible for mining long-term dependencies (whether the torque has gradually decreased within the last 10 minutes). Then, abnormal pattern matching is performed. The model compares the data sequence formed by the target rotation data with historical labeled samples such as normal assembly, minor equipment faults, and serious equipment faults to determine which abnormal pattern the current data belongs to. The model also verifies whether the anomaly is temporally related to the missing component (such as a screw) (e.g., the torque anomaly occurs before the missing component defect), avoiding misjudging unrelated equipment fluctuations. The initial threshold for anomaly pattern matching is generated from historical data statistics. Specifically, it is based on the rotation data of the assembly equipment during normal assembly at this workstation, calculating the mean value with fluctuations of up to and below two standard deviations as the initial baseline threshold. The TCN-LSTM model adjusts the threshold according to real-time assembly scenarios (such as different screw specifications, different operators, and equipment aging). For example, after six months of equipment use, the normal torque range may shift, and the model will automatically correct the threshold range. For critical defects such as missing screws, the model will appropriately reduce the threshold sensitivity (i.e., more easily judged as an anomaly) to avoid missing the root cause of the fault. For non-critical defects, the threshold is increased to reduce false alarms. The equipment fault early warning model analyzes the target rotation data and outputs anomaly identification indicators, including no anomaly, persistently low torque, and torque rejection fluctuations. It also outputs the fault root cause confidence level (between 0 and 1) to quantify the credibility that the rotation data anomaly is the direct cause of the missing component.

[0078] Step 404: If the confidence level of the root cause of the fault is greater than the confidence level threshold, a device maintenance instruction is pushed out. The device maintenance instruction is used to instruct the user to perform maintenance on the assembled device.

[0079] The confidence threshold can be set according to actual needs; for example, the confidence threshold can be set to 0.8.

[0080] In this embodiment, after obtaining the confidence level of the fault root cause output by the equipment fault early warning model, the confidence level of the fault root cause can be compared with the confidence level threshold. If the confidence level of the fault root cause is greater than the confidence level threshold, it can be determined that the abnormal rotation data of the assembly equipment is the direct cause of the missing parts, thereby pushing the equipment maintenance instruction (instead of just triggering an alarm). For example, the equipment maintenance instruction can be pushed to the assembly inspection management web system, or it can be pushed to the cloud server to instruct the user (the maintenance personnel of the assembly equipment) to perform maintenance on the assembly equipment.

[0081] In one optional embodiment of this application, multiple confidence thresholds of different values ​​can be set to trigger different processing strategies. For example, when the confidence level of the root cause of the fault is greater than or equal to 0.8, it is determined that the equipment fault has caused the defect, and the equipment repair instruction is pushed to the cloud first; when the confidence level of the root cause of the fault is between 0.5 and 0.8, it is determined to be a suspected correlation, and a defect alarm and equipment inspection prompt are pushed at the same time; when the confidence level of the root cause of the fault is less than 0.5, a normal alarm is issued.

[0082] In one optional embodiment of this application, the equipment fault early warning model can also output abnormal feature details, including the abnormal start time, minimum torque value, abnormal duration, and whether it conforms to the characteristics of gradual torque decay before missing parts are installed. Confidence level and abnormal features can distinguish between true and false correlations, and abnormal features can also provide assistance in subsequent fault processing.

[0083] The defect detection method of this application embodiment obtains the target rotation data of the assembly equipment within a first preset time period when the defect type output by the appearance defect detection model is a missing component. The target rotation data is then input into the equipment fault early warning model. The equipment fault early warning model outputs the confidence level of the root cause of the fault based on the target rotation data. When the confidence level of the root cause of the fault is greater than the confidence level threshold, a maintenance instruction is pushed to instruct the user to perform maintenance. In this way, the root cause of the missing component defect is traced, and a closed-loop tracing from the defect result to the fault cause is realized, thereby reducing the recurrence of the same type of defect.

[0084] In one optional embodiment of this application, the defect location information output by the appearance defect detection model and the fault type output by the equipment fault early warning model within the second preset time period can also be obtained. The duration of the second preset time period can be set according to actual needs, for example, the second preset time period can be set to 8 minutes, that is, a defect and fault detection report is generated every 8 minutes; the fault equipment identifier of the assembly equipment that detected the fault is determined based on the fault type; and then, a defect and fault detection report is generated based on the defect location information and the fault equipment identifier.

[0085] In this embodiment, the appearance defect detection model can also output defect location information, and the equipment fault early warning model can also output specific fault types (e.g., abnormal torque of a screw machine). It can statistically analyze all predicted defect location information and fault types within a second preset time period, and determine the fault equipment identifier of the assembly equipment that detected the fault based on the fault type. For example, if the identified fault type is abnormal torque of a screw machine, then the assembly equipment that detected the fault can be identified as a screw machine, and the equipment identifier of the screw machine can be obtained as the fault equipment identifier. Then, all defect location information and fault equipment identifiers are integrated to generate a defect and fault detection report. Optionally, the defect and fault detection report can also include an assessment of the impact of the detected defects and faults on subsequent assembly, assisting production decisions. Thus, a defect and fault detection report for a preset time period is generated, providing convenience for maintenance personnel to view the phased assembly quality.

[0086] In one optional embodiment of this application, the generated defect and fault detection report can also be transmitted to the assembly inspection management web system for management and storage, facilitating traceability analysis and archiving by later maintenance personnel.

[0087] In this embodiment, an assembly inspection management web system adaptable to various server models can also be developed. The web interface includes a model switching function (adapting to rack / tower servers). Users can upload new defect samples in batches through the web interface (≥40 samples of a single type, with defect category and location to be labeled). The system automatically verifies the sample format and label integrity, triggering incremental training of the external defect detection model. During incremental learning, a feature transfer and a small number of parameter update strategies are adopted—freezing the underlying ResNet residual blocks and convolutional blocks (accounting for 90% of the model parameters), and only updating the top-level classification layer (5%) and attention module (5%). At the same time, sample distillation technology is introduced to fuse the 40 newly uploaded samples with historical samples of the same type to generate distilled samples (preserving key features and reducing data volume), which shortens the incremental training time from 2 hours of traditional full training to 15 minutes, adapting to the needs of rapid iteration in the production line. Therefore, in one optional embodiment of this application, the defect detection method may further include: acquiring defect samples uploaded by users through a browser page, wherein the first model of the assembly object contained in the defect sample is different from the second model of the assembly object contained in the historical samples used to train the appearance defect detection model; generating distilled samples based on the defect samples and historical samples, for example, extracting common features from historical samples and newly uploaded defect samples of the same defect type, removing duplicates of the common features in the new defect samples, and retaining only the features different from those in historical samples of the same type to obtain distilled samples; incrementally training the appearance defect detection model using the distilled samples to obtain an appearance defect detection model for detecting defects in assembly objects of the first and second models; wherein, during incremental training, the model parameters of the residual neural network at the bottom layer of the appearance defect detection model are frozen, and only the model parameters of the region attention module, the component attention module, and the top-level classification module are updated. Testing shows that the incrementally trained appearance defect detection model achieves an accuracy rate of over 96% in identifying new defects. As a result, users can upload defect samples of new server types through a web page, which improves the adaptability of the solution to different types of servers, thereby enabling rapid iterative adaptation of multi-model server assembly detection. Furthermore, by training the appearance defect detection model through an incremental learning mechanism, freezing the parameters of the model's bottom feature extraction layer, and training only the top classification layer and attention module, the model training efficiency is improved.

[0088] In one optional embodiment of this application, the trained lightweight ResNet model containing the assembly feature attention module can be optimized for inference using TensorRT 8.0. First, the PyTorch model is converted to Open Neural Network Exchange (ONNX) format. Then, TensorRT's layer fusion and precision calibration (INT8 quantization) functions are used to reduce computational redundancy, ultimately increasing the model's inference speed from 8fps to 25fps, with single-frame detection time controlled within 40ms, meeting the real-time requirement of the server assembly line to complete the detection of one component every 2 seconds. The model in this application is deployed on edge nodes and directly connected to industrial cameras via the Peripheral Component Interconnect Express (PCIe) interface, achieving a local closed loop of image data acquisition-inference-result output, avoiding cloud transmission delays and ensuring the real-time performance of defect detection. An application programming interface (API) is used to implement the TCN-LSTM equipment fault warning model. The interface (API) is encapsulated using the FastAPI framework to build interface services (supporting POST requests to input device timing data and returning fault risk values ​​and warning levels). It is then containerized using Docker, configured with a base image (Ubuntu 20.04 + Python 3.8 + PyTorch 1.12) and resource limits (2 CPU cores, 4GB memory) to ensure environmental consistency across different edge nodes and facilitate migration between them. After container deployment, load balancing is achieved through an Nginx reverse proxy, supporting 80 concurrent calls per second and simultaneously connecting to sensor data from 5-8 assembly stations, outputting real-time device fault warning results. Edge nodes employ a dynamic power consumption adjustment strategy. During low-production periods on the assembly line, device power consumption is automatically reduced from 20W to 10W, and inference speed is reduced from 25fps to 15fps (still meeting detection requirements). During high-production periods (e.g., 30 units per hour), high-power, high-frame-rate mode is automatically restored, achieving a dynamic balance between real-time performance and energy consumption, reducing overall energy costs in the workshop. Through model optimization deployment, system function development, and iterative mechanism design, the system achieves real-time and modular operation of server assembly and testing, while adapting to the production needs of multiple server models and ensuring long-term adaptability to production line iterations.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0090] Embodiments of this application also provide a defect detection device, which can be implemented in software and / or hardware and can be integrated into an electronic device.

[0091] Figure 6 This is a schematic diagram of the structure of a defect detection device provided in an embodiment of this application, as shown below. Figure 6 As shown, the defect detection device 50 includes: a first acquisition module 510, a second acquisition module 520, a third acquisition module 530, a risk determination module 540, and an alarm module 550.

[0092] The first acquisition module 510 is used to acquire rotation data of the assembly equipment at the target assembly station and appearance image of the assembly object. The second acquisition module 520 is used to input the appearance image into a pre-trained appearance defect detection model and acquire the defect severity output by the appearance defect detection model. The third acquisition module 530 is used to input rotation data into a pre-trained equipment fault early warning model and acquire the equipment fault risk level output by the equipment fault early warning model. The risk determination module 540 is used to determine the defect risk value corresponding to the target assembly station based on the defect severity and equipment failure risk. The alarm module 550 is used to trigger a defect and fault linkage alarm when the defect risk value is greater than the alarm threshold.

[0093] Optionally, the first acquisition module 510 is also used for: Acquire raw images of the assembly objects captured by industrial cameras deployed at the target assembly station; The original image is converted to grayscale to obtain a grayscale image; Metal reflection suppression processing is applied to the grayscale image to obtain a reflection-suppressed image; Edge enhancement processing is performed on the reflection suppression image to obtain the appearance image.

[0094] Optionally, the first acquisition module 510 is also used for: Divide the grayscale image into blocks to obtain multiple image blocks; Based on the pixel grayscale values ​​of multiple pixels in multiple image blocks, determine the local pixel mean value corresponding to each of the multiple image blocks; Based on the local pixel mean and preset offset of each image patch, determine the grayscale threshold corresponding to each image patch. Identify target pixels in multiple image blocks whose pixel grayscale values ​​are greater than the corresponding grayscale thresholds, and adjust the pixel grayscale values ​​of the target pixels to the target grayscale range to obtain multiple reflection-suppressed image blocks. A reflection-suppressed image is generated based on multiple reflection-suppressed image blocks.

[0095] Optionally, the network structure of the appearance defect detection model includes a residual neural network, a region attention module, a component attention module, and a lightweight convolutional neural network model; the second acquisition module 520 is also used for: The appearance image is input into the residual neural network for convolution processing to obtain the appearance feature map corresponding to the appearance image. The appearance feature map is input into the region attention module for feature extraction to obtain the workstation-level region features. The appearance feature map is input into the component attention module for feature extraction to obtain component-level detail features; Workstation-level regional features and component-level detailed features are input into a lightweight convolutional neural network model for defect detection to obtain the defect severity.

[0096] Optionally, the defect detection device 50 further includes: The fourth acquisition module is used to acquire the defect types output by the appearance defect detection model; The fifth acquisition module is used to acquire the target rotation data of the assembly equipment within a first preset time period when the defect type is missing part assembly. The sixth acquisition module is used to input the target rotation data into the equipment fault early warning model and acquire the fault root cause confidence level output by the equipment fault early warning model. The fault root cause confidence level is used to characterize the credibility that the abnormal rotation data of the assembly equipment is the direct cause of the missing parts. The instruction push module is used to push equipment maintenance instructions when the confidence level of the fault root cause is greater than the confidence threshold. The equipment maintenance instructions are used to instruct users to perform maintenance on the assembled equipment.

[0097] Optionally, the defect detection device 50 further includes: The seventh acquisition module is used to acquire the defect location information output by the appearance defect detection model and the fault type output by the equipment fault early warning model within the second preset time period; The determination module is used to determine the faulty equipment identifier of the assembly equipment that has detected a fault based on the fault type; The report generation module is used to generate defect and fault detection reports based on defect location information and faulty equipment identification.

[0098] Optionally, the defect detection device 50 further includes: The eighth acquisition module is used to acquire defect samples uploaded by users through the browser page. The first model of the assembly object contained in the defect sample is different from the second model of the assembly object contained in the historical samples used to train the appearance defect detection model. The sample generation module is used to generate distilled samples based on defective samples and historical samples; The training module is used to incrementally train the appearance defect detection model using distilled samples to obtain an appearance defect detection model for detecting defects in assembly objects of the first and second models. During incremental training, the model parameters of the residual neural network at the bottom layer of the appearance defect detection model are frozen, and only the model parameters of the region attention module, the component attention module, and the top-level classification module are updated.

[0099] For a description of the features in the embodiment corresponding to the defect detection device, please refer to the relevant description of the embodiment corresponding to the defect detection method, which will not be repeated here.

[0100] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described defect detection method embodiments.

[0101] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described defect detection method embodiments when it is run.

[0102] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0103] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described defect detection method embodiments.

[0104] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described defect detection method embodiments.

[0105] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] The above provides a detailed description of a defect detection method and electronic device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A defect detection method, characterized in that, include: Acquire rotation data of the assembly equipment at the target assembly station and an image of the assembly object's appearance; The appearance image is input into a pre-trained appearance defect detection model, and the defect severity output by the appearance defect detection model is obtained. The rotation data is input into a pre-trained equipment fault early warning model, and the equipment fault risk level output by the equipment fault early warning model is obtained. Based on the defect severity and the equipment failure risk, the defect risk value corresponding to the target assembly station is determined; If the defect risk value is greater than the alarm threshold, a defect and fault linkage alarm will be triggered.

2. The defect detection method according to claim 1, characterized in that, Obtain the appearance image of the assembly object, including: Acquire the original image of the assembly object captured by the industrial camera deployed at the target assembly station; The original image is converted to grayscale to obtain a grayscale image; The grayscale image is subjected to metal reflection suppression processing to obtain a reflection-suppressed image; The reflection suppression image is subjected to edge enhancement processing to obtain the appearance image.

3. The defect detection method according to claim 2, characterized in that, The process of performing metallic reflection suppression processing on the grayscale image to obtain a reflection-suppressed image includes: The grayscale image is divided into blocks to obtain multiple image blocks; Based on the pixel grayscale values ​​of multiple pixels in the multiple image blocks, determine the local pixel mean value corresponding to each of the multiple image blocks; Based on the local pixel mean and preset offset corresponding to the plurality of image blocks respectively, determine the grayscale threshold corresponding to the plurality of image blocks respectively; Identify target pixels in the plurality of image blocks whose pixel grayscale values ​​are greater than the corresponding grayscale threshold, and adjust the pixel grayscale values ​​of the target pixels to the target grayscale range to obtain a plurality of reflection suppression image blocks; The reflection suppression image is generated based on the plurality of reflection suppression image blocks.

4. The defect detection method according to claim 1, characterized in that, The step of determining the defect risk value corresponding to the target assembly station based on the defect severity and the equipment failure risk includes: Based on the severity of the defects, the assembly defect weights are determined; Based on the assembly defect weights, determine the equipment failure weights; The defect risk value corresponding to the target assembly station is obtained by weighted summation based on the defect severity, assembly defect weight, equipment failure risk degree, and equipment failure weight.

5. The defect detection method according to claim 4, characterized in that, The determination of assembly defect weights based on the defect severity includes: The severity of the defect is compared with a preset severity threshold. If the severity of the defect is greater than or equal to the severity threshold, the assembly defect weight is determined to be the first weight. If the severity of the defect is less than the severity threshold, the assembly defect weight is determined to be the second weight. Wherein, the first weight is greater than the second weight.

6. The defect detection method according to claim 1, characterized in that, The network structure of the appearance defect detection model includes a residual neural network, a region attention module, a component attention module, and a lightweight convolutional neural network model. The step of inputting the appearance image into a pre-trained appearance defect detection model and obtaining the defect severity output by the appearance defect detection model includes: The appearance image is input into the residual neural network for convolution processing to obtain the appearance feature map corresponding to the appearance image; The appearance feature map is input into the region attention module for feature extraction to obtain workstation-level region features; The appearance feature map is input into the component attention module for feature extraction to obtain component-level detail features; The workstation-level region features and the component-level detail features are input into the lightweight convolutional neural network model for defect detection to obtain the defect severity.

7. The defect detection method according to claim 1, characterized in that, The method further includes: Obtain the defect type output by the appearance defect detection model; In the case where the defect type is missing component, the target rotation data of the assembly equipment within a first preset time period is obtained; The target rotation data is input into the equipment fault early warning model, and the fault root cause confidence score output by the equipment fault early warning model is obtained. The fault root cause confidence score is used to characterize the credibility that the abnormal rotation data of the assembly equipment is the direct cause of the missing component. If the confidence level of the fault root cause is greater than the confidence threshold, a device maintenance instruction is pushed out, which is used to instruct the user to perform maintenance on the assembly equipment.

8. The defect detection method according to any one of claims 1-7, characterized in that, The method further includes: Obtain the defect location information output by the appearance defect detection model and the fault type output by the equipment fault early warning model within the second preset time period; The fault equipment identifier of the assembly equipment that detected the fault is determined based on the fault type. Based on the defect location information and the faulty device identifier, a defect and fault detection report is generated.

9. The defect detection method according to any one of claims 1-7, characterized in that, The method further includes: Obtain defect samples uploaded by users through browser pages, wherein the first model of the assembly object contained in the defect sample is different from the second model of the assembly object contained in the historical samples used to train the appearance defect detection model; Based on the defective sample and the historical sample, a distillation sample is generated; The appearance defect detection model is incrementally trained using the distilled sample to obtain an appearance defect detection model for detecting defects in assembly objects of the first and second models. During incremental training, the model parameters of the residual neural network at the bottom layer of the appearance defect detection model are frozen, and only the model parameters of the region attention module, the component attention module, and the top-level classification module are updated.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the defect detection method as described in any one of claims 1 to 9.

Images