Training method of migration target detection model from GPU to NPU and target detection method

Abstract

This invention discloses a training method and object detection method for a GPU-to-NPU transfer target detection model, belonging to the field of object detection technology. In each training process, the transfer model P1 is first trained on the NPU using an object detection training set for the first stage. After a preset iteration epoch E1, the test descent rate of the current training is calculated. If the test descent rate of the current training is less than the preset rate, it indicates that the model may have difficulty converging. Therefore, a subset of the object detection training set is used to train the transfer model P1 for a preset iteration epoch E2 for the second stage, allowing the model to learn common target features and acquire basic feature extraction capabilities before starting the next training iteration. This invention can improve the training speed and stability of training a GPU-to-NPU transfer target detection model on the NPU in a simple and efficient manner.

Description

Technical Field

[0001] This invention belongs to the field of object detection technology, and more specifically, relates to a training method and object detection method for a GPU-to-NPU transfer object detection model. Background Technology

[0002] Current deep learning technology has reached a relatively mature stage in object detection, effectively extracting and recognizing complex features within objects. With the rapid development of deep learning, the hardware platforms for training and inference of object detection models are becoming increasingly diverse. Among them, Graphics Processing Units (GPUs) and Neural Processing Units (NPUs), due to their efficient parallel computing capabilities, can accelerate model training and inference in object detection tasks. However, the development of existing object detection models largely relies on GPU platforms, but the trend in object detection model development is gradually evolving towards cross-platform compatibility. Therefore, researching a training method for transferring object detection models from GPUs to NPUs is of great significance.

[0003] Existing methods for training transferable object detection models from GPUs to NPUs often involve training the model directly on the NPU. This transferable object detection model is essentially a model developed on a GPU platform and then migrated to the NPU. However, different hardware platforms have their own data formats and standards, leading to incompatible model formats. Often, the model's GPU data format and application programming interface (API) are forcibly converted to NPU mode to achieve model transfer. While this method accomplishes most of the code conversion, it heavily relies on general-purpose API libraries. Especially for custom APIs, API incompatibility issues frequently arise. When dealing with extremely large datasets, directly training the transferable object detection model on the NPU results in poor training stability. The resulting model often fails to converge or fit data, and the training speed is slow and highly unstable, with gradients exhibiting vanishing and exploding behavior. To address the aforementioned issues, other approaches involve manual intervention. This involves instrumenting specific problematic APIs to optimize API affinity, thereby enabling the smooth training of transfer target detection models from GPUs to NPUs directly on the NPU. However, this method is extremely labor-intensive for complex models, with long debugging cycles and low efficiency, failing to meet the demands of rapid iteration. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a training method and object detection method for a GPU-to-NPU transfer target detection model. The purpose is to improve the training speed and stability of the training process of training a GPU-to-NPU transfer target detection model on an NPU in a simple and efficient manner.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for training a transfer model from GPU to NPU, comprising:

[0006] S1. Let the number of training iterations t = 1; initialize the transfer model P1 as the transfer target detection model to be trained; where the transfer target detection model is the model after transferring the target detection model developed on the GPU platform to the NPU;

[0007] S2. On the NPU, the transfer model P1 is trained for the tth time using the object detection training set; after a preset iteration round E1, the test descent rate of the tth training is calculated; where the test descent rate is the rate of decrease of the object detection loss value of the transfer model P1 within the preset iteration round E1.

[0008] S3. Determine whether the test descent rate during the t-th training session is less than the preset rate. If so, record the computation graph g of the transfer model P1 during the t-th training session. t If the training session ends, proceed to S4; otherwise, proceed to S5.

[0009] S4. When the preset conditions are met, train the transfer model P1 on the NPU using a training subset for a preset number of iterations E2 to obtain the transfer model P2. Update the transfer model P1 to the transfer model P2, let t = t + 1, and go to S2. The preset conditions include: t = 1, or t > 1. Simultaneously calculate graph g. t The complexity is less than that of computing graph g. t-1 The complexity; the training subset is a subset of the object detection training set; the number of image samples in the training subset is less than the preset number;

[0010] S5. On the NPU, use the object detection training set to continue training the transfer model P1 in subsequent iterations until the transfer model P1 converges, thus completing the model training.

[0011] More preferably, the test descent rate during the t-th training iteration is:

[0012]

[0013] Where Loss1 is the object detection loss value of the transfer model P1 in the first iteration of the t-th training cycle; Loss E1 Let be the target detection loss value of the transfer model P1 in the E1th iteration of the t-th training.

[0014] More preferably, the transferred object detection model is the model developed on the GPU platform and then transferred to the NPU using an automatic transfer tool.

[0015] More preferably, the transferred target detection model is a binary compilation optimized model; wherein, the binary compilation optimization is: when the shapes of the input tensor and output tensor of the target detection model developed on the GPU platform are fixed, the compilation mode in the automatic transfer tool is set to the fixed shape compilation mode;

[0016] When the shapes of the input and output tensors of the object detection model developed on the GPU platform are not fixed, the compilation mode in the automatic migration tool is set to dynamic shape compilation mode.

[0017] More preferably, during the training of the transfer model P1 on the NPU using the object detection training set, during the data loading phase, the image sample data from the object detection training set is allocated to the locked memory on the NPU.

[0018] More preferably, during the training of the transfer model P1 on the NPU using the object detection training set, during the model training phase, the image sample data in the object detection training set is transferred from the host memory to the NPU device memory using a non-blocking data delivery method.

[0019] More preferably, the migrating target detection model further includes a gradient scaler.

[0020] More preferably, the above-mentioned target detection training set is: a set of labeled PCB image training samples; the labels include: whether there are defects, and the location information of the defects.

[0021] More preferably, the method for obtaining the PCB image training sample set includes:

[0022] Obtain the original PCB image set; the original PCB image set includes only images of defective PCBs;

[0023] Each PCB image in the original PCB image set is cut according to a preset overlap ratio to obtain the first image set;

[0024] Select multiple defective PCB images and multiple flawless PCB images from the first image set, making their ratio a preset ratio, and mark the areas where the defects are located in the defective PCB images to obtain the second image set;

[0025] Perform one or more of the following operations on the defective PCB images in the second image set: scaling, stretching, rotation, color transformation, brightness and contrast enhancement, and adding Gaussian noise.

[0026] Thirdly, the present invention provides an NPU-based target detection method, comprising:

[0027] The image to be classified is input into a pre-trained transfer target detection model on the NPU for classification.

[0028] The transferred target detection model is a model developed on the GPU platform and then transferred to the NPU; the pre-trained transferred target detection model is obtained by pre-training using the training method provided in the first aspect of this invention.

[0029] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0030] 1. This invention provides a training method for a transfer model from GPU to NPU. Through a two-stage training strategy, it helps the model find a better convergence starting point on complex loss graphs. In each training process, the transfer model P1 is first trained on the NPU using an object detection training set. After a preset iteration epoch E1, the test descent rate of the current training is calculated. If the test descent rate of the current training is less than the preset rate, it indicates that the model may have difficulty converging. Therefore, a subset of the object detection training set is used to train the transfer model P1 for a preset iteration epoch E2 in the second stage, allowing the model to learn common target features and obtain a good initial weight value, enabling the model to have basic feature extraction capabilities before starting the next training. Furthermore, starting from the second training, after the first stage of training has passed a preset iteration epoch E1, the complexity of the computation graph under this training is monitored to ensure real-time monitoring of the training status. Through the above operations, the transfer model P1 can converge faster and more stably on the object detection training set. The whole process is simple to operate and can improve the training speed and stability of training a transfer object detection model from GPU to NPU in a simple and efficient way.

[0031] 2. Furthermore, the GPU-to-NPU transfer model training method provided by this invention transfers the target detection model to a binary compiled and optimized model, which enables the NPU's graph compiler to perform deeper operator fusion and optimization, generating binary files with better execution performance.

[0032] 3. Furthermore, the GPU-to-NPU transfer model training method provided by this invention allocates image sample data from the target detection training set to paged memory on the NPU during the data loading stage, ensuring that the data resides in physical memory and avoiding frequent page swapping by the operating system, thereby significantly accelerating the data transfer from the CPU to the NPU and further improving training efficiency.

[0033] 4. Furthermore, the GPU-to-NPU transfer model training method provided by this invention, during the model training stage, adopts a non-blocking data delivery method to transfer image sample data in the target detection training set from the host memory to the NPU device memory, thereby realizing parallel execution between CPU data preprocessing and NPU model calculation and reducing waiting time.

[0034] 5. Furthermore, the GPU-to-NPU transfer model training method provided by the present invention further includes a gradient scaler to prevent gradient overflow or underflow, thereby ensuring the stability of mixed precision training.

[0035] 6. Furthermore, the GPU-to-NPU transfer model training method provided by this invention performs a series of data augmentation processes on the PCB image when handling PCB defect detection tasks, including cutting, filtering, and transformation operations, generating a rich variety of training samples. This not only increases the scale and diversity of the dataset but also improves the model's ability to identify defects under different conditions, thereby enhancing the model's robustness and generalization performance. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the training method for the GPU-to-NPU transfer model provided in an embodiment of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0038] To achieve the above objectives, in a first aspect, the present invention provides a method for training a transfer model from GPU to NPU, such as... Figure 1 As shown, it includes:

[0039] S1. Let the number of training iterations t = 1; initialize the transfer model P1 as the transfer target detection model to be trained; where the transfer target detection model is the model after transferring the target detection model developed on the GPU platform to the NPU;

[0040] S2. On the NPU, the transfer model P1 is trained for the tth time using the object detection training set; after a preset iteration round E1, the test descent rate of the tth training is calculated; where the test descent rate is the rate of decrease of the object detection loss value of the transfer model P1 within the preset iteration round E1.

[0041] S3. Determine whether the test descent rate during the t-th training session is less than the preset rate. If so, record the computation graph g of the transfer model P1 during the t-th training session. t If the training session ends, proceed to S4; otherwise, proceed to S5.

[0042] S4. When the preset conditions are met, train the transfer model P1 on the NPU using a training subset for a preset number of iterations E2 to obtain the transfer model P2. Update the transfer model P1 to the transfer model P2, let t = t + 1, and go to S2. The preset conditions include: t = 1, or t > 1. Simultaneously calculate graph g. t The complexity is less than that of computing graph g. t-1 The complexity; the training subset is a subset of the object detection training set; the number of image samples in the training subset is less than the preset number;

[0043] It should be noted that if the preset conditions are not met, the training will fail and the operation will end.

[0044] S5. On the NPU, use the object detection training set to continue training the transfer model P1 in subsequent iterations until the transfer model P1 converges, thus completing the model training.

[0045] It should be noted that the above object detection model can be any existing object detection model, such as the R-CNN model, the YOLO series models (such as YOLOV1, YOLOV3, YOLOV5, YOLOV9, etc.), the SSD model, etc., without any limitation.

[0046] It should be noted that there are multiple ways to calculate the test descent rate mentioned above, and no specific method is used here. For example, the test descent rate for the t-th training iteration is:

[0047]

[0048] Among them, Loss i Let Loss be the target detection loss value of the transfer model P1 in the i-th iteration of the t-th training cycle; j Let be the target detection loss value of the transfer model P1 in the j-th iteration of the t-th training; the i-th iteration and the j-th iteration are any two different iterations from the first iteration to the E1-th iteration in the t-th training, where i is less than j.

[0049] Preferably, in one optional implementation, the test descent rate during the t-th training iteration is:

[0050]

[0051] Where Loss1 is the object detection loss value of the transfer model P1 in the first iteration of the t-th training cycle; Loss E1 Let be the target detection loss value of the transfer model P1 in the E1th iteration of the t-th training.

[0052] In one alternative implementation, the transferred object detection model is the model developed on the GPU platform and then migrated to the NPU using an automatic migration tool. It should be noted that any existing automatic migration tool can be used, such as the PyTorch GPU2Ascend migration tool, etc., and there is no limitation here.

[0053] In one optional implementation, the transferred object detection model is a binary-compiled and optimized model. It should be noted that there are various methods of binary compilation optimization, which are not limited here. Preferably, in one optional implementation, the binary compilation optimization is as follows: when the shapes of the input and output tensors of the object detection model developed on the GPU platform are fixed, the compilation mode in the automatic migration tool is set to fixed-shape compilation mode; when the shapes of the input and output tensors of the object detection model developed on the GPU platform are not fixed, the compilation mode in the automatic migration tool is set to dynamic-shape compilation mode.

[0054] In one alternative implementation, during the training of the transfer model P1 on the NPU using the object detection training set, during the data loading phase, the image sample data from the object detection training set is allocated to the paged memory on the NPU.

[0055] In one alternative implementation, during the training of the transfer model P1 on the NPU using the object detection training set, during the model training phase, the image sample data in the object detection training set is transferred from the host memory to the NPU device memory using a non-blocking data delivery method.

[0056] In one alternative implementation, the moving target detection model further includes a gradient scaler.

[0057] It should be noted that the target detection tasks that this invention can be applied to are of various types, such as panoramic scene segmentation, key point detection, vehicle detection, etc., and are not limited here; in one optional implementation, the above-mentioned target detection task is PCB defect detection. In this case, the above-mentioned target detection training set is: a set of labeled PCB image training samples; the labels include: whether there is a defect, and the location information of the defect.

[0058] It should be noted that the aforementioned PCB image training sample set can be any existing PCB image training sample set. Preferably, in an optional implementation, the method for obtaining the PCB image training sample set includes:

[0059] Obtain the original PCB image set; the original PCB image set includes only images of defective PCBs;

[0060] Each PCB image in the original PCB image set is cut according to a preset overlap ratio to obtain the first image set;

[0061] Select multiple defective PCB images and multiple flawless PCB images from the first image set, making their ratio a preset ratio, and mark the areas where the defects are located in the defective PCB images to obtain the second image set;

[0062] Perform one or more of the following operations on the defective PCB images in the second image set: scaling, stretching, rotation, color transformation, brightness and contrast enhancement, and adding Gaussian noise.

[0063] Example 1

[0064] The following example illustrates the invention in detail using the scenario of training the YOLOv9 object detection model on an Ascend NPU with a very large object detection training set.

[0065] After porting the open-source YOLOv9 PyTorch code to the Ascend NPU platform using an automatic migration tool, it was directly trained on a very large object detection training set (70k samples). The following problems were encountered: training speed was only 1 / 10 of that of the GPU, the loss curve oscillated wildly and failed to converge, and frequent NaN (Not a Number) gradient errors occurred. The main reasons are as follows:

[0066] Extremely large datasets imply an extremely complex loss graph. Training models from randomly initialized states makes it difficult to find an effective convergence path. The automatic differentiation mechanism on the NPU may differ from native frameworks, leading to unstable gradient calculations. In dynamic graph frameworks like PyTorch, the CPU is responsible for dispatching computational operators to the NPU for execution one by one. Ideally, the NPU's computational pipeline should be continuously fully loaded. However, if the CPU's dispatch speed cannot keep up with the NPU's execution speed, the NPU will idle, wasting computational power. Conversely, if the CPU dispatches too quickly, the bottleneck lies in the NPU's execution efficiency. The core of performance optimization lies in reducing the CPU's operator dispatch time (host-side) and the NPU's operator execution time (device-side).

[0067] To address the aforementioned issues, this embodiment provides a training method for a transfer model from GPU to NPU. Considering the difficulty of direct training on extremely large datasets, in each training process, the aforementioned extremely large object detection training set is first used to train the transfer YOLOv9 model. After a preset iteration cycle E1 (in this embodiment, the preset iteration cycle E1 is 30 poch), the test descent rate of the current training iteration is calculated. When the test descent rate of the current training iteration is less than a preset rate (in this embodiment, the preset rate is 0.075), the computation graph g of the transfer model P1 under the t-th training iteration is recorded. t Furthermore, when preset conditions are met, the transfer YOLOV9 model is trained using the standard dataset COCO128 (a subset of the object detection training set) for a preset iteration round E2 (in this embodiment, the preset iteration round E2 is 30 poch). This stage aims to allow the model to learn common target features and obtain a good initial weight value. The above process is repeated (at this point, since the model already possesses basic feature extraction capabilities, it can converge faster and more stably on the large object detection training set) until the test descent rate of the current training is greater than or equal to the preset rate. Then, the transfer model P1 is trained on the NPU using the aforementioned large object detection training set for subsequent iteration rounds until the transfer YOLOV9 model converges, thus completing the model training. If the preset conditions are not met during the above process, it indicates training failure, and the operation ends. The preset conditions include: t = 1, or t > 1, and simultaneously calculating graph g. t The complexity is less than that of computing graph g. t-1 The complexity is determined by the following: the training subset is a subset of the object detection training set; the number of image samples in the training subset is less than a preset number (in this embodiment, the preset number is 128 images). In one optional implementation, the complexity of the computation graph is obtained through an adjacency list: O(|V|+|E|); where the computation graph is treated as a directed acyclic graph G=(V,E); V is a node in the directed acyclic graph, representing an operator in the computation graph; E is an edge in the directed acyclic graph, representing a variable in the computation graph; and O is the complexity factor representation.

[0068] The transfer target detection model in this embodiment also includes a gradient scaler to prevent gradient overflow or underflow and ensure the stability of mixed-precision training. During training, the gradient scaler is used to adjust the gradient calculation of the transfer target detection model optimizer to maintain gradient stability. In this embodiment, when using mixed-precision training, a gradient scaler (torch_npu.amp.GradScaler) is configured for the optimizer (e.g., AdamW) in the training script. By dynamically adjusting the scaling factor (scale_factor), the loss is multiplied by this factor before backpropagation to amplify the gradient value and prevent it from underflowing to zero in FP16; before updating the weights, the gradient is divided by this factor to restore its original size, thereby effectively preventing gradient vanishing and exploding.

[0069] This embodiment configures the training system to optimize operator compilation and data transmission. The configuration includes enabling a fixed-shape operator compilation mode, using paged memory during data loading, and transmitting data to the NPU in a non-blocking manner.

[0070] Specifically, considering that the input image size of the YOLOv9 model is typically fixed during training and inference (e.g., 640x640), we set SOC_VERSION=Ascend910B (example version) in the system environment variable or compilation options and configure a fixed shape mode (i.e., fixed-shape compilation mode). The NPU's Graph Engine performs more aggressive operator fusion during the graph compilation phase, merging multiple small operators into a large fusion operator, reducing kernel startup overhead and memory interactions, thus improving performance. For scenarios where the model input and output tensor shapes are fixed, we enable the "Fixed Shape" compilation mode. This allows the NPU's Graph Engine to perform deeper operator fusion and optimization, generating binaries with better execution performance.

[0071] During the data loading phase of the training process, image sample data from the object detection training set is allocated to paged memory on the NPU. Specifically, when defining PyTorch's DataLoader, pin_memory is set to True. Paged memory ensures that data resides in physical memory, avoiding frequent page swapping by the operating system, thus significantly accelerating data transfer from the CPU to the NPU.

[0072] During the model training phase of the training process, a non-blocking data delivery method (non_blocking=True) is used to migrate image sample data from the host memory to the NPU device memory in the object detection training set. Specifically, in the training loop, when transferring data from the CPU to the NPU, `.to('npu:0',non_blocking=True)` is called. This non-blocking data transfer method enables parallel execution between CPU data preprocessing and NPU model computation, reducing waiting time.

[0073] The above design makes the data preprocessing and loading pipeline smoother, and the CPU can prepare the next batch of data in parallel with the NPU to compute the current batch of data, which significantly reduces the NPU idle time caused by data waiting.

[0074] Through the above operations, the training performance of the YOLOv9 model on the Ascend NPU was fundamentally improved: In terms of training speed, compared to the initial transfer version, the overall training speed was increased by 8-10 times, approaching the training speed on a high-performance GPU. In terms of training stability, the loss curve decreased smoothly, the gradient NaN problem was completely eliminated, and the model successfully converged on a very large dataset. In terms of object detection accuracy, the final mAP (mean Average Precision) metric achieved by the model met the expected requirements. This demonstrates the effectiveness and superiority of the multi-level model training method proposed in this invention. This invention can significantly improve the training speed and stability of the model on the NPU, reduce the engineering complexity of optimization and debugging, and has significant application value.

[0075] Example 2

[0076] To further illustrate the training method of the GPU-to-NPU transfer model provided by this invention, the following description is based on a specific target detection task. In this embodiment, the target detection task is a PCB image defect detection task.

[0077] In the printed circuit board (PCB) manufacturing process, automated optical inspection (AOI) systems are indispensable tools for ensuring product quality and reliability. AOI systems capture PCB images using high-resolution cameras and employ image processing algorithms to identify potential defects or flaws, such as poor solder joints, missing components, or misaligned placements. Traditional AOI systems primarily rely on predefined rules and template matching for detection, but this approach is poorly suited to the complex and varied types of defects. With the development of machine learning and deep learning technologies, image-based defect detection algorithms have been widely applied, bringing significant improvements to automated production and quality control.

[0078] This embodiment considers that most existing improvements focus on optimizing the detection model architecture itself, such as introducing more complex network structures, employing pre-trained model transfer learning, or designing specific loss functions to improve model performance. However, these methods often require significant computational resources and may not fully overcome the challenges posed by the limitations of the training dataset itself. In reality, obtaining a sufficient number of high-quality defective samples is a challenge. Furthermore, improper processing of flawless background images can lead to an imbalance between positive and negative samples, affecting the model's learning effect and generalization ability. Therefore, this embodiment focuses on how to effectively expand and optimize the training dataset to improve the accuracy and robustness of the detection model.

[0079] In this embodiment, data augmentation was first performed on the original PCB image set; the original PCB image set only included: defective PCB images;

[0080] For a pre-raw PCB image set x0, the original PCB images of arbitrary size in x0 are cut into k*k square sub-images, and the overlap rate of the sub-images is selected as λ1. The resulting new dataset is x1. In this embodiment, k is 512 or 640, and λ1 is 0.1.

[0081] All flawed sub-images in x1 are selected to form image set x2. Flawless sub-images in x1 are selected, and a portion are randomly discarded, leaving a set of images x3 with a number λ² times the number of images in image set x2. This set serves as the background images used during training. In this embodiment, a value of λ² is preferably 1. These operations ensure a balance between positive and negative samples, avoiding learning bias caused by sample imbalance and contributing to improved model learning performance and accuracy.

[0082] To scale, stretch, and rotate the images in image set x2, the specific steps are: enlarge the image set to an area λ3 and rotate it 180 degrees to obtain image set x. 4-1 The image set x is obtained by reducing it to an area of ​​1 / λ³ and rotating it by 90 degrees. 4-2 Shrink the image to a size of λ and rotate it 270 degrees to obtain the image set x. 4-3 . Transfer the image set x 4-1 x 4-2 x 4-3 Combined, we get image set x4. In this embodiment, λ3 is 4 and λ4 is 2 / 3.

[0083] To perform color transformation on the images in image set x2, the specific steps are as follows: Take an image t, and for each pixel value [r, g, b] of t, treat it as a vector v, and let v = vw, where w is a 3x3 matrix, to obtain a new image t'. Perform the above operation on each image to obtain image set x5 from image set x2.

[0084] Furthermore, a preferred approach is to perform the above transformation five times and combine the results of the five transformations to obtain the final image set x5. The w matrix is ​​different for each of the five transformations, as follows: 1 1 0 0 1 1 1 0 0 0 1 0 0 0 1

[0086] 0 0 1, 1 0 0, 0 1 1, 1 0 1, 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[0088] Brightness and contrast enhancement is performed on the images in image set x2 by increasing the brightness of the images by λ5 times and increasing the contrast value of the images by λ6, resulting in image set x6.

[0089] Furthermore, it is preferred that λ5 and λ6 have a value of 1.5.

[0090] Gaussian noise is added to the images in image set x2. For each image t in image set x2, for each pixel value [r,g,b] of t, let y = y + ηN(0,1), where N(0,1) is a standard normal distribution and η is a parameter. This yields image set x7. In this embodiment, η is preferably set to 0.25.

[0091] The images in image set x3 are randomly rotated as follows: for each image in image set x3, there is a 1 / 4 chance of rotating it by 90 degrees, a 1 / 4 chance of rotating it by 180 degrees, and a 1 / 4 chance of rotating it by 270 degrees, thus obtaining image set x8.

[0092] The images in image set x8 undergo random color transformation. Specifically, for each image in image set x8, there is a probability of λ7 of random color transformation. The specific method for this random color transformation is as follows: for each image, a 3x3 matrix w is randomly generated, and then the same color transformation as described above is performed on that image. This yields image set x9. In this embodiment, λ7 is preferably set to 1 / 5.

[0093] The images x2, x4, x5, x6, x7, and x9 are combined to form a new dataset, which is the final PCB image training sample set.

[0094] The data augmentation methods described above were used to generate a high-quality, diverse training sample set of PCB images, ensuring that each image input into the YOLOv9 transfer model met the expected format and size requirements. Specifically, the images in the original PCB image set were cut into 512*512 square sub-images with an overlap rate of 0.1. Flawless images were randomly discarded until the remaining number was equal to the number of flawed images. Then, 1 / 5 of these images were randomly recolored, and all images were randomly rotated before being integrated into the final dataset. Flawed images were processed without further processing, recolored to red, blue, cyan, orange, and purple, respectively, with Gaussian noise of intensity 0.25 added, brightness and contrast increased by 1.5 times, rotated, and then enlarged by 4 times, reduced by 4 times, or shrunk to 2 / 3 of their original size before being integrated into the final dataset. This resulted in a rich and diverse augmented dataset.

[0095] Then, the YOLOv9 model configuration was adjusted according to the hardware characteristics of the Ascend NPU910b, in preparation for training. Specifically, the configuration of the transfer YOLOv9 model was adjusted based on the hardware characteristics, including but not limited to hyperparameter settings such as batch size and learning rate scheduling strategy, as well as necessary operator transfer analysis and tuning. The loss function of the transfer YOLOv9 model was replaced with the Inner-EIoU loss function, focusing on improving the accuracy of bounding box prediction, especially in small object detection. The transfer YOLOv9 model in this embodiment is the model after being automatically transferred to the Ascend NPU910b platform using the Ascend Toollist automatic transfer tool.

[0096] The model was trained using the GPU-to-NPU transfer model training method provided in Embodiment 1 of this invention. During training, the Inner-EIoU loss function was used to guide the model's learning process to ensure the accuracy of the detection boxes. Simultaneously, the Ascend PyTorchProfiler tool was used to closely monitor the training progress and comprehensively analyze the model's performance during training. The model configuration was adjusted in a timely manner to achieve optimal performance by upgrading high-performance libraries, using paged memory, employing the NPU affinity optimizer, implementing non-blocking data delivery, optimizing operator binary compilation, and scanning the NPU affinity API. Through a series of optimization measures, the model was able to run normally on the Ascend NPU910b. The model was exported as an ONNX file, enabling it to run model inference on a 2-core 8GHz CPU. The related technical solutions are the same as in Embodiment 1 of this invention and will not be repeated here.

[0097] After training, the performance of the trained model is evaluated using a test set to check its ability to detect various types of PCB defects. Information such as the type, quantity, location, and confidence level of the defects output are recorded to determine the effectiveness of the model.

[0098] After thorough training and validation, the model was successfully deployed to Huawei Cloud, enabling real-time and efficient detection of PCB board defects, which greatly improved the level of automation and quality control in production.

[0099] In summary, this embodiment aims to address the problems of insufficient training data, lack of sample diversity, and insufficient model ability to identify defects under complex conditions in existing technologies. Through a series of data augmentation strategies and optimized YOLOv9 detection model configuration, this invention has the following significant advantages: (1) improved dataset diversity and representativeness, and enhanced feature expression; (2) high detection efficiency and accuracy; (3) adaptability to dedicated hardware platforms, capable of running on Ascend NPU910b. This embodiment significantly improves the robustness and accuracy of PCB board defect detection, promotes the development of the electronics manufacturing industry, and enhances the quality control level of the electronics manufacturing industry.

[0100] Thirdly, the present invention provides an NPU-based target detection method, comprising:

[0101] The image to be classified is input into a pre-trained transfer target detection model on the NPU for classification.

[0102] The transferred target detection model is a model developed on the GPU platform and then transferred to the NPU; the pre-trained transferred target detection model is obtained by pre-training using the training method provided in the first aspect of this invention.

[0103] The related technical solutions are the same as the training method provided in the first aspect of this invention, and will not be described in detail here.

[0104] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for a transfer model from GPU to NPU, characterized in that, include: S1, Let the number of training iterations t = 1; Initialize the transfer model P1 as the transfer target detection model to be trained; The transferred target detection model is a model developed on the GPU platform and then transferred to the NPU. S2. The transfer model P1 is trained for the tth time on the NPU using the object detection training set; After a preset number of iterations E1, calculate the test descent rate of the t-th training iteration; The test descent rate is the descent rate of the target detection loss value of the migration model P1 within the preset iteration round E1; S3. Determine whether the test descent rate during the t-th training session is less than the preset rate. If so, record the computation graph g of the transfer model P1 during the t-th training session. t If the training session ends, proceed to S4; otherwise, proceed to S5. S4. When the preset conditions are met, the transfer model P1 is trained on the NPU using a training subset for a preset number of iterations E2 to obtain the transfer model P2; the transfer model P1 is updated to the transfer model P2, let t = t + 1, and then proceed to S2; the preset conditions include: t = 1, or t > 1, and graph g is calculated simultaneously. t The complexity is less than that of computation graph g. t-1 The complexity; the training subset is a subset of the target detection training set; the number of image samples in the training subset is less than a preset number; S5. On the NPU, the transfer model P1 is trained in subsequent iterations using the object detection training set until the transfer model P1 converges, thereby completing the model training.

2. The training method according to claim 1, characterized in that, The test descent rate during the t-th training iteration is: Where Loss1 is the object detection loss value of the transfer model P1 in the first iteration of the t-th training cycle; Loss E1 Let be the target detection loss value of the transfer model P1 in the E1th iteration of the t-th training.

3. The training method according to claim 1, characterized in that, The transferred target detection model is the model developed on the GPU platform and then transferred to the NPU using an automatic transfer tool.

4. The training method according to claim 3, characterized in that, The migration target detection model is a binary compiled and optimized model; The binary compilation optimization is as follows: when the shapes of the input and output tensors of the target detection model developed on the GPU platform are fixed, the compilation mode in the automatic migration tool is set to fixed shape compilation mode. When the shapes of the input and output tensors of the object detection model developed on the GPU platform are not fixed, the compilation mode in the automatic migration tool is set to dynamic shape compilation mode.

5. The training method according to claim 1, characterized in that, During the training of the transfer model P1 on the NPU using the object detection training set, in the data loading phase, the image sample data in the object detection training set is allocated to the locked memory on the NPU.

6. The training method according to any one of claims 1-5, characterized in that, During the training of the transfer model P1 on the NPU using the object detection training set, image sample data from the host memory to the NPU device memory is transferred in a non-blocking data delivery manner during the model training phase.

7. The training method according to any one of claims 1-5, characterized in that, During the training of the transfer model P1 on the NPU using the object detection training set, image sample data from the host memory to the NPU device memory is transferred in a non-blocking data delivery manner during the model training phase.

8. The training method according to any one of claims 1-5, characterized in that, The moving target detection model also includes: gradient scaler.

9. The training method according to any one of claims 1-5, characterized in that, The target detection training set is a set of labeled PCB image training samples; the labels include: whether there are defects, and the location information of the defects; The method for obtaining the PCB image training sample set includes: Obtain the original PCB image set; the original PCB image set includes only images of defective PCBs; Each PCB image in the original PCB image set is cut according to a preset overlap ratio to obtain the first image set; Select multiple defective PCB images and multiple flawless PCB images from the first image set, making their ratio a preset ratio, and mark the areas where the defects are located in the defective PCB images to obtain the second image set; Perform one or more of the following operations on the defective PCB images in the second image set: scaling, stretching, rotation, color transformation, brightness and contrast enhancement, and adding Gaussian noise.

10. A target detection method based on NPU, characterized in that, include: The image to be classified is input into a pre-trained transfer target detection model on the NPU for classification. The transferred target detection model is a model developed on the GPU platform and then transferred to the NPU; the pre-trained transferred target detection model is obtained by pre-training using the training method described in any one of claims 1-9.

Images