New energy motor power assembly visual defect detection system and method based on deep learning, storage medium and computer program product

Abstract

The invention provides a new energy motor power assembly visual defect detection system and method based on deep learning, a storage medium and a computer program product. The method comprises the following steps: acquiring an image of a key area of a motor power assembly workpiece; constructing and deploying a deep learning large model for defect detection of the new energy motor power assembly; and performing defect detection and result judgment on the motor power assembly workpiece based on the acquired image and the model deployed by the deep learning large model training and deployment module. The problems that existing manual detection efficiency is low, and traditional machine vision flexibility is poor are solved. The motor power assembly key area image quality is ensured through multi-angle shooting and adaptive illumination, the defect classification, positioning and segmentation precision is improved through a multi-task learning head, high-confidence-coefficient defects are inferred and screened in real time, automatic judgment and unqualified product sorting are achieved through a linkage execution unit, mistaken judgment and missing judgment are reduced, the detection efficiency and the automation level are improved, and the detection efficiency is improved. And the quality control requirements of industrial continuous production are met.

Description

Technical Field

[0001] This invention belongs to the field of industrial artificial intelligence and machine vision technology, specifically relating to a deep learning-based visual defect detection system, method, storage medium, and computer program product for the powertrain of an 800V platform flat wire motor (whose stator core stack thickness includes 60, 100, and 120 mm) using hair-pin winding technology. Background Technology

[0002] With the rapid development of the new energy vehicle industry, the quality and reliability requirements for its core component—the electric motor powertrain—are becoming increasingly stringent. Surface defects in components such as stator and rotor, including exposed copper in the stator coating, coating cracks, stator core lamination, damaged insulation paper, residual foreign matter (such as high-temperature tape), and broken copper wires, are key factors affecting product performance and safety.

[0003] Currently, mainstream production line quality inspection mainly relies on the following two methods: 1. Manual visual inspection; this involves quality inspectors making judgments based on their experience. This method suffers from problems such as low efficiency, high labor intensity, susceptibility to subjective factors, high rates of missed and false inspections, inability to achieve full inspection, and difficulty in standardizing inspection criteria.

[0004] 2. Traditional machine vision inspection: This method uses pre-defined rules and algorithms (such as edge detection, template matching, and threshold segmentation) for defect identification. This approach is sensitive to changes in lighting and has poor adaptability to the diversity, complexity, and uncertainty of defects. It requires meticulous algorithm design and adjustment for each type of defect, resulting in insufficient generalization ability and flexibility. When product models or defect morphologies change slightly, the algorithm often needs to be redeveloped, leading to high maintenance costs.

[0005] Especially for 800V high-voltage platform flat wire motors (hair-pin) widely used in the drive systems of new energy vehicles, their stators are characterized by large stack thickness (such as 60, 100, 120), dense copper wire arrangement in the slots, and extremely high insulation requirements. Detecting defects on the surface of the enameled wire (flat wire), hair-pin solder joints, the ends of the high-stack iron core, and the insulating paper in the slots of such motors faces unique challenges, including more confined spaces, complex light reflection, and more subtle defect morphologies. Traditional methods or general testing solutions are difficult to effectively address these challenges. Summary of the Invention

[0006] To address the shortcomings of existing technologies in detecting complex defects, their lack of flexibility, and their reliance on manual labor, this invention proposes a deep learning-based visual defect detection system, method, storage medium, and computer program product for new energy motor powertrains.

[0007] A visual defect detection system for new energy motor powertrains based on deep learning, which achieves one of the objectives of this invention, includes: Image acquisition system module: used to acquire images of key areas of the motor powertrain workpiece; Deep Learning Large Model Training and Deployment Module: Used to build and deploy a deep learning large model for defect detection in new energy motor powertrains; Online detection and feedback execution module: Based on the images acquired by the image acquisition system construction module and the deep learning large model deployed by the deep learning large model training and deployment module, it performs defect detection and result judgment on the motor powertrain workpiece.

[0008] The motor powertrain is a flat wire motor using hair-pin winding technology, with a working voltage platform of 800V and a stator core stack thickness of 60, 100, or 120.

[0009] Furthermore, the image acquisition system module includes: imaging devices respectively positioned at side and endoscopic angles, used to acquire images of key areas on the side and inside of the motor powertrain workpiece; the key areas of the motor powertrain workpiece include one or more combinations of the enameled wire surface, the iron core end, and the insulating paper mounting location; different key areas of the motor powertrain workpiece can be detected separately; or different key areas of the motor powertrain workpiece can be detected simultaneously. The technical effects include: targeted acquisition of side and internal key area images through multi-angle imaging devices ensures no defects are missed, improving the comprehensiveness and accuracy of image acquisition.

[0010] Furthermore, the image acquisition system construction module also includes a standardized lighting unit and an automated triggering device; The standardized lighting unit includes a ring light source, a coaxial light source, and a backlight source, with the light source combination adapted to the surface characteristics of different parts of the motor powertrain workpiece. Among them, the copper wire part is adapted to the light source combination that suppresses reflection, and the insulating paper part is adapted to the light source combination that highlights the matte feature. It adopts a single-channel mode and is wrapped with an iron sheet structure. When the motor powertrain workpiece enters the preset photography area, the entrance and exit doors of the standardized lighting unit close. The automated triggering device is connected to the production line PLC and includes a proximity switch; when the motor powertrain workpiece is transported by the production line conveyor to the preset image capture position, the proximity switch detects the workpiece signal and triggers image acquisition.

[0011] The technical benefits include: standardized lighting adapts to different materials, ensuring image quality; automated triggering and linkage with production lines improves acquisition efficiency and stability, and adapts to continuous industrial production.

[0012] Furthermore, the deep learning large model training and deployment module includes a model training unit, a model optimization unit, and a deployment execution unit; The model training unit is used to train large deep learning models based on a selected base model; The model optimization unit is used to optimize the performance of large deep learning models that have been trained. The deployment execution unit is used to deploy the optimized deep learning large model to edge computing devices or built-in industrial control computers; it includes an annotation terminal, a training terminal, a tool terminal, and a deploy terminal; the annotation terminal is used for image management, manual annotation, and intelligent assisted annotation; the training terminal is used to load sample data for offline model training and performance testing; the tool terminal is used for sample screening and flexible data processing; and the deploy terminal is used to link with production line vision software to achieve online model detection.

[0013] The technical benefits include: through training, optimization, and deployment unit collaboration, the model performance has been improved and adapted to edge devices or industrial control computers, meeting the real-time detection needs of industrial scenarios.

[0014] Furthermore, the model training unit is configured with a multi-task learning head, which includes a shared feature layer, a defect classification submodule, a target localization submodule, and a pixel-level segmentation submodule. The shared feature layer reuses the multi-scale feature map output by the base model. The defect classification submodule includes a two-layer fully connected network and a Softmax activation function to output a defect category probability distribution. The defect category probability distribution is a multi-dimensional probability vector corresponding to preset defect categories such as exposed copper, cracking, warping, breakage, and residual tape. The real-time inference unit selects the category corresponding to the dimension with the highest probability in the probability distribution as the final defect type and uses this highest probability as the defect type. The target localization submodule includes a 3×3 convolutional kernel layer and a bounding box regression layer, used to output defect coordinates and localization confidence; the pixel-level segmentation submodule includes a transposed convolutional layer and a sigmoid activation function, used to output a binary segmentation mask representing defect pixels or background pixels, where a mask value of 1 corresponds to a defect pixel and 0 corresponds to a background pixel; the total loss of the model training unit is a weighted fusion of classification loss, localization loss and segmentation loss, where the classification loss is cross-entropy loss, the localization loss is GIoU loss, and the segmentation loss is Dice loss, and the weight coefficients α, β, and γ of the three are defaulted to 0.3, 0.4, and 0.3.

[0015] The technical benefits include: multi-task learning heads share features and output multi-dimensional results, simultaneously achieving defect classification, localization, and segmentation, thus improving detection accuracy and information completeness.

[0016] Furthermore, the online detection and feedback execution module includes a real-time inference unit and a result determination unit; The real-time inference unit receives images of the motor powertrain workpiece acquired by the image acquisition system construction module, calls the optimized deep learning model to perform defect analysis on the workpiece image, and outputs corresponding defect information, including defect type, defect confidence level, and defect location coordinates. When the defect confidence level in the defect information is higher than a preset threshold, the corresponding defect type, defect confidence level, and defect location coordinates are transmitted to the result judgment unit in the online detection and feedback execution module. The result determination unit is used to receive defect information transmitted by the real-time inference unit and to determine the motor powertrain workpiece according to preset rules. The preset rules include the number of defects to be determined and the determination criteria. The result determination unit is configured with a visual identifier. When the determination is qualified, a preset OK-RGB color identifier box is output. When the determination is unqualified, a preset NG-RGB color identifier box is output. The over-kill rate and escape rate of the defects are recorded simultaneously.

[0017] The technical benefits include: real-time reasoning to screen high-confidence defects, accurate judgment by the result determination unit according to rules, reduced misjudgments, and improved reliability of quality inspection.

[0018] Furthermore, the online detection and feedback execution module also includes a linkage execution unit, used to achieve linkage control with the production line control system based on the judgment result of the result judgment unit. When the judgment result is qualified, a workpiece flow signal is sent to the production line PLC to control the conveyor line to transfer the workpiece to the next process. When a product is judged to be unqualified, defect information is recorded and a report is generated, an alarm signal is sent to the production line PLC, triggering the production line alarm light to start and pneumatic push rods and other actuators to sort unqualified workpieces. The model inference results (defect type, confidence level, location coordinates) are sent to the decision engine. At the same time, the defect image and defect information are packaged and uploaded to the production line vision software via the MQTT protocol. Meanwhile, the MES generates a unique quality traceability ID to complete the closed-loop management of production quality data. The technical benefits include: closed-loop detection and production process of the linkage execution unit, automatic control of workpiece flow and sorting of defective products, and improved efficiency of production line automation and quality control.

[0019] Furthermore, the online detection and feedback execution module also includes a data traceability unit, used to construct workpiece quality archives and realize traceable management of detection data; it receives the judgment results given by the result judgment unit and the defect information uploaded by the linkage execution unit, generates a quality traceability ID uniquely corresponding to the workpiece, and records the following data: workpiece detection time, defect type, defect location coordinates, defect confidence level, defect image, and judgment result; the above data is synchronously uploaded to the MES system to form a production quality database, providing data support for the optimization of motor powertrain production processes.

[0020] The technical benefits include: the data traceability unit constructs quality profiles, generates unique traceability IDs, records all test data and synchronizes it to the MES, achieving data traceability, providing support for production process optimization, and improving the level of precision in quality control.

[0021] A second objective of this invention is a deep learning-based visual defect detection method for new energy motor powertrains, comprising: Acquire images of key areas of the electric motor powertrain component; Construct and deploy a large-scale deep learning model for defect detection in new energy motor powertrains; Based on the images acquired by the image acquisition system construction module and the deep learning large model deployed by the deep learning large model training and deployment module, defect detection and result judgment are performed on the motor powertrain workpiece.

[0022] The motor powertrain is a flat wire motor powertrain using hair-pin winding technology, with an operating voltage of 800V, and can be adapted to different models of products with stator core stack thicknesses of 60, 100, and 120.

[0023] A non-transitory computer-readable storage medium for achieving the third objective of the present invention stores a computer program thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the deep learning-based visual defect detection method for new energy motor powertrains.

[0024] A computer program product for achieving the fourth objective of the present invention includes a computer program / instruction, which, when executed by a processor, implements the steps of the deep learning-based visual defect detection method for new energy motor powertrains.

[0025] The beneficial effects of this invention include: This invention solves the problems of low efficiency in existing manual inspections and poor flexibility in traditional machine vision. Specifically targeting the unique defects of high-thickness stators in 800V high-voltage flat-wire motors (hair-pin), such as poor hair-pin welding, damaged insulation paper in deep slots, and warped core end faces, it achieves accurate and efficient automatic inspection. Multi-angle imaging and adaptive lighting ensure image quality of key areas of the motor powertrain; a multi-task learning head improves the accuracy of defect classification, location, and segmentation; real-time reasoning filters high-confidence defects; and a linked execution unit enables automatic judgment and sorting of non-conforming products, reducing false positives and false negatives, improving inspection efficiency and automation levels, and adapting to the quality control needs of continuous industrial production. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the system described in this invention. Detailed Implementation

[0027] The following detailed embodiments are provided to explain the technical solutions of the present invention, so that those skilled in the art can understand the present invention. The scope of protection of the present invention is not limited to the following specific embodiments. Any modifications or improvements made by those skilled in the art that incorporate the technical solutions of the present invention but differ from the following detailed embodiments are also within the scope of protection of the present invention.

[0028] A deep learning-based visual defect detection system for new energy electric motor powertrains, such as... Figure 1 As shown, it is particularly suitable for flat wire motor power assemblies with a working voltage of 800V and hair-pin winding technology, and can be adapted to different models of products with stator core stack thickness of 60, 100 and 120, including: image acquisition system construction module, deep learning large model training and deployment module, and online detection and feedback execution module.

[0029] Image acquisition system building module: used to achieve blind-spot-free, high-definition image acquisition of key parts of the motor powertrain workpiece; including: multi-angle high-resolution industrial camera array, standardized lighting unit and processing unit; A multi-angle, high-resolution industrial camera array is deployed at the production line inspection station, including at least two Daheng Imaging MER2-1220-32U3M industrial cameras with a maximum global shutter resolution of 12 megapixels. These cameras are configured as a side-view camera and an endoscope camera, respectively. The side-view camera is used to acquire images of key areas on the side of the motor powertrain workpiece; the endoscope camera is used to acquire images of key areas inside the motor powertrain workpiece. These key areas include the surface of the enameled wire, the end of the iron core, and the location where the insulation paper is installed. The industrial camera array is rotated and calibrated using tray positioning pins, and preset depth-of-field parameters are configured to constrain the image acquisition effect. Due to the narrow space and large stacking thickness (e.g., 120) within the slots of the hair-pin flat wire motor, the endoscope camera requires a lens with a specific focal length and depth of field to ensure clear capture of the insulation paper and enameled wire surface inside the deep slots.

[0030] The standardized lighting unit integrates a ring light source, a coaxial light source, and a backlight source, all of which are LED light sources. The standardized lighting unit adopts a single-channel mode and is externally encased in a sheet metal structure. When the motor powertrain workpiece enters the preset imaging area, the entrance and exit doors of the standardized lighting unit close, achieving uniform and stable lighting through the combination of multiple light sources, highlighting defect features, and suppressing ambient light interference. The types of light sources in the multi-light source combination are adapted to the surface characteristics of different parts of the motor powertrain workpiece; for example, a light source combination that suppresses reflection is suitable for copper wire parts, while a light source combination that highlights matte characteristics is suitable for insulating paper parts.

[0031] The processing unit uses an industrial workstation with a built-in NVIDIA RTX A5000 GPU to run the deep learning model corresponding to visual defect detection of new energy motor powertrains. It receives image data collected by multi-angle high-resolution industrial camera arrays and performs real-time image processing. At the same time, it receives trigger signals from automated triggering devices and sends image acquisition commands to the multi-angle high-resolution industrial camera arrays.

[0032] The automated triggering device is connected to the production line PLC (Programmable Logic Controller). The automated triggering device includes a proximity switch. When the motor powertrain workpiece is transported by the production line conveyor to the preset photo-taking position (proximity switch), the proximity switch detects the workpiece signal and transmits it to the automated triggering device. The automated triggering device sends an image acquisition trigger command to the processing unit to realize the automated control of image acquisition.

[0033] The image acquisition module ensures that the acquired images of the motor powertrain workpiece have no blind spots and clear defect features through multi-angle synchronous acquisition by a multi-angle high-resolution industrial camera array, adaptive light source combination of standardized lighting units, and precise triggering by an automated triggering device. Moreover, the image acquisition process is synchronized with the production line cycle, meeting the real-time requirements of online inspection.

[0034] Deep Learning Large Model Training and Deployment Module: Used to build and train a deep learning large model for defect detection in new energy motor powertrains and deploy it on the production line. It provides inference for defect classification, localization and segmentation of workpiece images acquired by the image acquisition module, meeting the accuracy and real-time requirements of online detection; including: data processing unit, model training unit, model optimization unit and deployment execution unit. The data processing unit is used to collect, label, and enhance defect sample data. The collected defect sample data specifically comes from production images of 800V platform hairpin flat wire motors (stator stack thickness 60 / 100 / 120). In one embodiment, the defect sample data consists of real production images containing various defects such as exposed copper, cracks, warping, breakage, and residual adhesive tape in the 800V platform stator assembly. The labeling method uses a fine labeling method combining bounding box and pixel-level segmentation labeling. Data enhancement techniques include rotation, flipping, brightness adjustment, and noise addition to improve model robustness. In one embodiment, the number of model training iterations is set to 300-1000 times, the number of images in a single training iteration is a multiple of 2, the specific number is determined according to the detection accuracy requirements, and the confidence threshold for model inference is preset to 0.25.

[0035] The model training unit is used to train a large deep learning model based on a selected base model. In one embodiment, the selected base model includes: YOLOv8 is selected as the base model because most of the surface defects of the motor powertrain are small targets with irregular shapes. Its CSPDarknet backbone network performs well in multi-scale feature extraction. At the same time, an attention module (SE Block) is introduced to enhance the ability to perceive small defects (such as exposed copper in enameled wire).

[0036] In one embodiment, the model training unit is further configured with a loss function optimization strategy and a multi-task learning head; the loss function optimization strategy includes: if the model is underfitting, optimization is achieved by increasing model complexity, reducing regularization strength, extending training time, and verifying the effectiveness of data features; if the model is overfitting, optimization is achieved by increasing sample data, enhancing regularization, simplifying model structure, and adopting an early stopping mechanism; if the model training exhibits oscillations or NaN, optimization is achieved by reducing the learning rate, increasing the batch size, adopting gradient pruning, and adding batch normalization; the multi-task learning head is used to simultaneously complete defect classification (identifying defect types), target localization (determining defect locations), and pixel-level segmentation (extracting the precise shape of defects).

[0037] In one embodiment, the multi-task learning head is deeply adapted to the base model (such as improved YOLOv8, VisionTransformer, etc.). By reusing features from the base model and using an architecture that splits task branches, defect classification, target localization, and pixel-level segmentation are completed simultaneously. This design avoids building separate models for different tasks, thus avoiding the efficiency loss caused by multiple inferences in traditional single-task models. Furthermore, feature reuse ensures consistency in the understanding of defect features across the three types of tasks.

[0038] Specifically, the multi-task learning head first uses multi-scale feature maps output by the base model to construct the core feature input layer. For example, for a model with YOLOv8 as the base model, it directly uses feature maps of different resolutions such as 8×8, 16×16, and 32×16 generated by its CSPDarknet backbone network. These feature maps, from low to high resolution, correspond to the global contour and local details of the defect, respectively. For a model with VisionTransformer as the base model, it uses a high-dimensional feature layer output by the Multi-HeadAttention module. This layer can better capture the correlation features between the defect and the background. This layer, which directly inherits the base model and provides a unified input for all subsequent tasks, is defined as the shared feature layer. It is used to centrally extract key features of the defect, avoiding repetitive feature extraction for each task. This reduces computational consumption and ensures that the classification, localization, and segmentation tasks are based on the same set of feature logic reasoning, reducing error conflicts between tasks.

[0039] Based on the shared feature layer, the multi-task learning head is divided into three independent but collaborative task sub-modules: defect classification module, target localization module, and pixel-level segmentation module.

[0040] The defect classification module adopts a two-layer fully connected network structure. First, global pooling is performed on the feature map output by the shared feature layer to transform the high-dimensional features into vectors of fixed dimensions. Then, the probability distribution of preset defect categories such as exposed copper, cracks, and warping is output through the Softmax activation function. For example, for 10 common defects, a 10-dimensional probability vector is generated, and the category corresponding to the dimension with the highest probability is the defect type.

[0041] The target localization module further extracts local features through a convolutional layer with a 3×3 kernel, and then calculates and outputs the precise coordinates of the defect region through the bounding box regression layer, including the x and y coordinates of the bounding box center and the width w and height h. It also outputs a location confidence value to evaluate the reliability of the bounding box localization.

[0042] The pixel-level segmentation module uses a transposed convolutional layer to upsample the low-resolution feature map output by the shared feature layer to the same resolution as the original acquired image. Then, it generates a binary segmentation mask through the Sigmoid activation function: pixels with a value of 1 in the mask correspond to the defect area, and pixels with a value of 0 correspond to the background area, thereby accurately outlining the actual shape and edge of the defect.

[0043] To ensure that the three tasks are optimized synchronously and without interference during training, a weighted fusion loss function is designed for the learning head. Considering the different optimization objectives of each task—the defect classification module needs to accurately distinguish defect categories, the target localization module needs bounding boxes to highly overlap with actual defects, and the pixel-level segmentation module needs pixel-level matching—the total loss is composed of three proportionally weighted parts: the classification loss uses cross-entropy loss to calculate the error between the predicted category value and the true label; the localization loss uses GIoU loss to evaluate the accuracy of the bounding box; and the segmentation loss uses Dice loss to measure the overlap between the segmentation mask and the actual defect region. The weight coefficients (α, β, γ) are determined through hyperparameter search, with default settings of α=0.3, β=0.4, and γ=0.3. This emphasizes the priority of the localization task for production line detection while ensuring that the accuracy of classification and segmentation is not sacrificed, allowing the model to improve the performance of all three tasks simultaneously during training. The weights α, β, and γ are optimized on the validation set through grid search, with the optimization objective being to maximize mAP@0.5:0.95. Experiments show that when the localization task weight is slightly higher (β=0.4), the model can improve the overlap between the localization box and the real defect by about 5% while maintaining classification accuracy.

[0044] In the actual inference phase, the multi-task learning head only needs to complete one forward propagation to synchronously output the defect category label, bounding box position, and segmentation mask from the shared feature layer.

[0045] Compared to the traditional approach of using three single-task models for separate inference, this design reduces the time consumption and can meet the real-time detection needs of new energy motor powertrain production lines. At the same time, the three types of results can corroborate each other. For example, if the segmentation mask highly overlaps with the bounding box region, it can further reduce the probability of misjudgment in a single task and improve the overall reliability of detection.

[0046] The model optimization unit is used to optimize the performance of the trained deep learning model. The optimization methods include model pruning and model quantization. The model optimization unit analyzes the model prediction results through confidence threshold, confusion matrix and bounding box (BBOX) to calculate the category, number and AP accuracy of defect targets, so as to verify whether the detection performance of the optimized model meets the production line requirements.

[0047] The deployment execution unit comprises four sub-modules: an annotation module, a training module, a tool module, and a deploy module. It supports deployment on high-performance edge computing devices (such as the NVIDIA Jetson AGX Orin) or industrial control computers to achieve low-latency real-time inference. Specifically, the annotation module is used for image management, manual annotation, and intelligent assisted annotation to improve annotation efficiency; the training module loads sample data for offline model training and performance testing; the tool module handles sample selection and flexible data processing; and the deploy module integrates with production line vision software to achieve online model deployment and real-time defect detection. The edge computing device is the NVIDIA Jetson AGX Orin, and the industrial control computer is an industrial workstation with a built-in NVIDIA RTX A5000 GPU for the image acquisition module.

[0048] In one embodiment, the model deployment process of the deployment execution unit is as follows: after the annotation end completes the image annotation, the training end imports the annotated training images for model training; the online test is conducted to assess the model's detection performance; and the deploy end deploys the qualified model to edge computing devices or industrial control computers to achieve real-time linkage with the production line.

[0049] In one embodiment, the model training unit also supports dynamically adjusting sample weights, reducing the weighting of ordinary quality samples, so that the model prioritizes learning the features of defect samples that are difficult to detect, and further improves the model's generalization ability to complex defects.

[0050] The online detection and feedback execution module is used to realize real-time defect analysis, detection result judgment, production line linkage control, and closed-loop traceability of quality data for motor powertrain workpieces, supporting the fully automated online detection process of the production line. It includes a real-time inference unit, a result judgment unit, a linkage execution unit, and a data traceability unit. The real-time inference unit, the result judgment unit, the linkage execution unit, and the data traceability unit are connected in sequence. The real-time inference unit communicates with the deployment execution unit of the deep learning large model training and deployment module. The linkage execution unit communicates with the production line PLC (programmable logic controller) and MES (manufacturing execution system).

[0051] The real-time inference unit receives images of the motor powertrain workpiece acquired by the image acquisition module, calls the optimized deep learning large model deployed by the deep learning large model training and deployment module, and performs real-time defect analysis on the workpiece image. The output results include defect type (such as exposed copper, cracking, warping, breakage, and residual tape), defect confidence, and defect location coordinates. When the defect confidence is higher than a preset threshold, the defect information is transmitted to the result judgment unit. In this embodiment, the confidence threshold is 0.75, which is determined based on the balance point of precision and recall in the validation set PR curve. The overlap threshold is 80%, which is derived from the IoU distribution statistics of 1000 sets of labeled samples to ensure that the positioning accuracy is >95%.

[0052] The result determination unit is used to determine the workpiece and output the result based on the defect information output by the real-time inference unit; the determination criteria include: The preset number of defects to be judged: most defects correspond to a single judgment, while a few complex defects correspond to multiple judgments; Defect Judgment Criteria: Any positive integer can be filled in. When the number of judgment criteria for different numbered judgment items is the same, the defect needs to satisfy all judgment items at the same time; when the number of judgment criteria for different numbered judgment items is different, the defect only needs to satisfy one of the judgment items.

[0053] The result judgment unit is also configured with result visualization indicators: when the judgment result is qualified, a preset OK-RGB color indicator box is output; when the judgment result is unqualified, a preset NG-RGB color indicator box is output, and the over-Killrate and escaperate of the defect are recorded simultaneously (the specific values ​​are set according to the production severity requirements of the corresponding test items of the motor powertrain).

[0054] The linkage execution unit is used to achieve linkage control with the production line control system based on the judgment result of the result judgment unit. When the judgment result is qualified, the linkage execution unit sends a workpiece flow signal to the production line PLC to control the conveyor line to transfer the workpiece to the next process. When the judgment result is unqualified, the linkage execution unit sends an alarm signal to the production line PLC through the TCP / IP protocol, triggering the production line alarm light to start and the pneumatic push rod and other actuators to act (for sorting unqualified workpieces). At the same time, the defect image and defect information are packaged and uploaded to the production line vision software through the MQTT protocol.

[0055] The linkage execution unit is used to achieve linkage control with the production line control system based on the judgment result of the result judgment unit. When the judgment result is qualified, the linkage execution unit sends a workpiece flow signal to the production line PLC, controlling the conveyor line to transfer the workpiece to the next process. When a product is judged to be unqualified, the system automatically records the defect information and generates a report. The linkage execution unit sends an alarm signal to the production line PLC via TCP / IP protocol, triggering the production line alarm light to start and pneumatic push rods and other actuators to sort unqualified workpieces. The model inference results (defect type, confidence level, location coordinates) are sent to the decision engine. At the same time, the defect image and defect information are packaged and uploaded to the production line vision software via MQTT protocol. Meanwhile, the MES generates a unique quality traceability ID to complete the closed-loop management of production quality data.

[0056] The data traceability unit is used to build a digital archive of workpiece quality and realize traceable management of inspection data; it receives the judgment results given by the result judgment unit and the defect information uploaded by the linkage execution unit, generates a quality traceability ID (assigned by the MES system) that is uniquely associated with the workpiece, and records the following data: workpiece inspection time, defect type, defect location coordinates, defect confidence level, defect image, and judgment result; and synchronously uploads the above data to the MES system to form a production quality database, providing data support for the optimization of motor powertrain production processes.

[0057] The workflow of the online inspection and feedback execution module is as follows: the real-time inference unit receives workpiece images and calls a deep learning large model for analysis; the result judgment unit determines whether the workpiece is qualified or unqualified based on defect information and preset rules and outputs a visual label; the linkage execution unit sends control signals to the PLC / MES according to the judgment result and triggers corresponding actions; and the data traceability unit generates a quality traceability ID and records and uploads the inspection data, forming a closed-loop process of inspection, judgment, execution and traceability, which meets the automation and traceability inspection requirements of the new energy motor powertrain production line.

[0058] This invention also provides a deep learning-based visual defect detection method for new energy motor powertrains, particularly suitable for flat wire motor powertrains operating at 800V and employing hair-pin winding technology, and adaptable to different models with stator core stack thicknesses of 60, 100, and 120 mm, including: Acquire images of key components of the electric motor powertrain; Construct and deploy a large-scale deep learning model for defect detection in new energy motor powertrains; Based on the images acquired by the image acquisition system construction module and the deep learning large model deployed by the deep learning large model training and deployment module, defect detection and result judgment are performed on the motor powertrain workpiece.

[0059] In one embodiment, the method for detecting insulation paper damage defects in a hair-pin flat wire motor stator assembly with an 800V platform and a stack thickness of 120 is as follows: The multi-angle high-resolution industrial camera array (model DahengImagingMER2-1220-32U3M) of the image acquisition system module acquires images of the insulation paper area of ​​the stator assembly. The image resolution is set to 1920×1080 pixels and the storage format is BMP. The standardized lighting unit adopts a combination of ring light source and backlight to adapt to the light source selection requirements that the insulation paper area needs to highlight the matte feature, ensuring that the contrast between the damaged area of ​​the insulation paper and the background is ≥30dB, providing clear image data for subsequent inspection.

[0060] The model training unit uses the improved YOLOv8 as the base model, and its multi-task learning head output parameters are as follows: The defect coordinates output by the target localization submodule are (x=800 pixels, y=500 pixels, w=120 pixels, h=80 pixels), where x and y are the center coordinates of the bounding box, and w and h are the width and height of the bounding box. The upper left corner coordinates of the rectangular region (denoted as region A) corresponding to these coordinates are (740 pixels, 460 pixels), and the lower right corner coordinates are (860 pixels, 540 pixels). The total number of pixels covered by region A is 120×80=9600 pixels. The binarized segmentation mask output by the pixel-level segmentation submodule is consistent with the resolution of the acquired image (1920×1080 pixels). The pixels with a value of 1 in the mask (denoted as region B) constitute the actual outline of the damaged insulation paper. According to statistics, the total number of pixels in region B is 6200 pixels, and the pixel distribution range is the upper left corner (750 pixels, 470 pixels) and the lower right corner (850 pixels, 530 pixels).

[0061] The real-time inference unit of the online detection and feedback execution module calls a preset overlap calculation algorithm, using the intersection-over-union (IoU) ratio as the overlap metric. The calculation steps are as follows: The pixel coordinates of region A and region B are compared pixel by pixel, and the number of pixels in the intersection region (denoted as region C) that simultaneously belongs to the two regions is counted. In this embodiment, the number of pixels in region C is 5890 pixels. The number of pixels in the union region (denoted as region D) is calculated as follows: number of pixels in region D = number of pixels in region A + number of pixels in region B - number of pixels in region C, i.e., 9600 + 6200 - 5890 = 9910 pixels; The overlap is calculated as IoU = number of pixels in region C / number of pixels in region D. In this embodiment, IoU = 5890 / 9910 ≈ 0.594.

[0062] The real-time inference unit has a preset overlap threshold of 80% for positioning accuracy (this threshold is calibrated based on the detection data of 1000 sets of insulation paper damage defect samples), and the processing logic is as follows: If the overlap is ≥80%, the defect coordinates of the target positioning submodule are determined to be accurate. The defect coordinates (x=800, y=500, w=120, h=80) along with the defect type (damaged insulation paper) and defect confidence level (0.92) are transmitted to the result judgment unit and the workpiece qualification judgment process is initiated. If the overlap is less than 80% (e.g., 59.4% in this embodiment), the defect coordinates are determined to be inaccurate. The real-time inference unit marks the positioning result as pending verification and uploads the defect image, defect coordinates, segmentation mask, and overlap value (59.4%) to the production line vision software via the MQTT protocol. At the same time, the data traceability unit records it in the file corresponding to the quality traceability ID of the stator assembly. Subsequently, the quality inspector manually reviews it, and this set of data is used for subsequent iterative optimization of the model training unit (adjusting the 3×3 convolution kernel weights of the target positioning submodule).

[0063] This invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the various steps of the method described in this invention.

[0064] This invention also provides a non-transitory computer-readable storage medium storing a computer program. The computer program includes program instructions that, when executed by a processor, implement the various steps of the method described in this invention, which will not be elaborated further here.

[0065] The computer-readable storage medium can be the data transmission apparatus or the internal storage unit of a computer device provided in any of the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be the external storage device of the computer device, such as the plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the computer device.

[0066] Furthermore, the computer-readable storage medium may include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that is to be output or has already been output.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A visual defect detection system for new energy motor powertrains based on deep learning, characterized in that, include: Image acquisition system module: used to acquire images of key areas of the motor powertrain workpiece; Deep Learning Large Model Training and Deployment Module: Used to build and deploy a deep learning large model for defect detection in new energy motor powertrains; Online detection and feedback execution module: Based on the images acquired by the image acquisition system construction module and the deep learning large model deployed by the deep learning large model training and deployment module, it performs defect detection and result judgment on the motor powertrain workpiece.

2. The deep learning-based visual defect detection system for new energy motor powertrains as described in claim 1, characterized in that, The image acquisition system module includes: imaging devices respectively set at the side viewing angle and the endoscopic angle, used to acquire images of key areas on the side and key areas inside the motor powertrain workpiece; the key areas include one or more combinations of the enameled wire surface, the iron core end, the insulating paper, and the hair-pin solder joint.

3. The deep learning-based visual defect detection system for new energy motor powertrains as described in claim 1 or 2, characterized in that, The image acquisition system module also includes a standardized lighting unit and an automated triggering device; The standardized lighting unit includes a ring light source, a coaxial light source, and a backlight source, and the combination of light sources is adapted according to the material surface characteristics of different parts of the motor powertrain workpiece; when the motor powertrain workpiece enters the preset photography area, the entrance and exit doors of the standardized lighting unit are closed; The automated triggering device communicates with the production line PLC and includes a proximity switch. When the motor powertrain workpiece is transported to the preset photo-taking position, the proximity switch detects the workpiece and triggers image acquisition.

4. The deep learning-based visual defect detection system for new energy motor powertrains as described in claim 1, characterized in that, The deep learning large model training and deployment module includes a model training unit, a model optimization unit, and a deployment execution unit; The model training unit is used to train large deep learning models based on a selected base model; The model optimization unit is used to optimize the performance of large deep learning models that have been trained. The deployment execution unit is used to deploy optimized deep learning large models to edge computing devices or built-in industrial control computers.

5. The deep learning-based visual defect detection system for new energy motor powertrains as described in claim 4, characterized in that, The model training unit is configured with a multi-task learning head, which includes a shared feature layer, a defect classification submodule, a target localization submodule, and a pixel-level segmentation submodule. The shared feature layer reuses the multi-scale feature map output by the basic model, and uses the multi-scale feature map as the common input of the defect classification submodule, the target localization submodule, and the pixel-level segmentation submodule; The defect classification submodule consists of a two-layer fully connected network and a Softmax activation function, which is used to output the probability distribution of defect categories; The target localization submodule includes a 3×3 convolutional kernel layer and a bounding box regression layer, which are used to output defect coordinates and localization confidence. The pixel-level segmentation submodule includes a transposed convolutional layer and a Sigmoid activation function, which are used to output a binary segmentation mask representing defects or background.

6. The deep learning-based visual defect detection system for new energy motor powertrains as described in claim 4, characterized in that, The online detection and feedback execution module includes a real-time inference unit and a result determination unit; The real-time inference unit is used to receive the workpiece image of the motor powertrain acquired by the image acquisition system construction module, call the optimized deep learning large model to perform defect analysis on the workpiece image, and output the corresponding defect information. When the defect confidence level in the defect information is higher than the preset threshold, the corresponding defect type, defect confidence level and defect location coordinates are transmitted to the result judgment unit. The result determination unit is used to receive defect information transmitted by the real-time inference unit and to determine the motor powertrain workpiece according to preset rules.

7. The deep learning-based visual defect detection system for new energy motor powertrains as described in claim 6, characterized in that, The online detection and feedback execution module also includes a linkage execution unit, which is used to achieve linkage control with the production line control system based on the judgment result of the result judgment unit. When the judgment result is qualified, a workpiece flow signal is sent to the production line PLC to control the conveyor line to transfer the workpiece to the next process. When a product is judged to be unqualified, the defect information is recorded and a report is generated. An alarm signal is sent to the production line PLC to trigger the production line alarm light to start and the pneumatic push rod and other actuators to sort the unqualified workpieces.

8. A deep learning-based visual defect detection method for new energy motor powertrains according to the system described in claim 1, characterized in that, include: Acquire images of key areas of the electric motor powertrain component; Construct and deploy a large-scale deep learning model for defect detection in new energy motor powertrains; Based on the images acquired by the image acquisition system construction module and the deep learning large model deployed by the deep learning large model training and deployment module, defect detection and result judgment are performed on the motor powertrain workpiece.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the deep learning-based visual defect detection method for new energy motor powertrains as described in claim 8.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the deep learning-based visual defect detection method for new energy motor powertrains as described in claim 8.

Images