Training method of migration target detection model from GPU to NPU and target detection method

Abstract

The invention discloses a training method of a migration target detection model from a GPU to an NPU and a target detection method, and belongs to the technical field of target detection. In each training process, firstly, a target detection training set is adopted on the NPU to carry out first-stage training on the migration model P1, after a preset iteration round E1, the test descent rate of the current training is calculated, when the test descent rate of the current training is smaller than a preset rate, it is indicated that the model is difficult to converge, and when the test descent rate of the current training is smaller than the preset rate, it is indicated that the model is difficult to converge. Therefore, a subset of a target detection training set is firstly adopted to carry out second-stage training of a preset iteration round E2 on a migration model P1, so that the model learns universal target features and has basic feature extraction capability, and then next training is started; according to the method, the training speed of training the migration target detection model migrated from the GPU to the NPU on the NPU and the stability of the training process can be improved in a simple and efficient mode.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target detection, and more particularly relates to a training method of a GPU-to-NPU migrated target detection model and a target detection method. BACKGROUND

[0002] The current deep learning technology has reached a relatively mature stage in the research of target detection tasks, and it can well realize the extraction and recognition of complex features in the target. With the rapid development of deep learning, the hardware platforms for training and reasoning of target detection models are becoming more and more diversified. Among them, the graphic processing unit (GPU) and the neural network processing unit (NPU) can accelerate the model training and reasoning in the target detection task due to their efficient parallel computing capability. However, the development of existing target detection models mostly depends on the GPU platform, but the development trend of target detection models is gradually evolving towards cross-platform, so it is of great significance to study a training method of a GPU-to-NPU migrated target detection model.

[0003] The existing training method of a GPU-to-NPU migrated target detection model often directly trains the migrated target detection model on the NPU. The migrated target detection model is a model developed on the GPU platform and then migrated to the NPU. However, different hardware platforms have their own data formats and standards, and the model formats are incompatible. Therefore, the data format and application interface (API) on the GPU are often forcibly converted to the NPU mode to realize model migration. Although this method can complete most of the code conversion, it mostly depends on the general API library, especially for custom APIs, which often have API incompatibility problems. When facing a super large-scale data set, directly training the migrated target detection model from the GPU to the NPU on the NPU will result in poor stability of the training process, and the obtained model is often difficult to converge or fit, and the training speed is slow, the training process is extremely unstable, and the gradient disappears or explodes at times. In order to solve the above problems, some other methods use manual intervention to analyze the specific API that has problems through instrumentation, so as to optimize the affinity of the API, and thus make the process of directly training the migrated target detection model from the GPU to the NPU on the NPU proceed smoothly. However, for complex models, this method has a huge amount of engineering and a long debugging period, and is low in efficiency, which cannot meet the demand for rapid iteration. SUMMARY

[0004] In view of the above defects or improvement needs of the prior art, the application provides a training method of a GPU-to-NPU migrated target detection model and a target detection method, which aims to improve the training speed and stability of the training process of the migrated target detection model from the GPU to the NPU on the NPU in a simple and efficient manner.

[0005] To achieve the above object, in a first aspect, the application provides a training method of a GPU-to-NPU migration model, comprising:

[0006] S1, let the training times t = 1; initialize the migration model P1 as a migration target detection model to be trained; wherein the migration target detection model is a model after migrating a target detection model developed on a GPU platform to an NPU;

[0007] S2, start the tth training of the migration model P1 on the NPU using a target detection training set; after a preset iteration round E1, calculate the test drop rate of the tth training; wherein the test drop rate is the drop rate of the target detection loss value of the migration model P1 within the preset iteration round E1;

[0008] S3, judge whether the test drop rate of the tth training is less than a preset rate, if yes, record the computational graph g t of the migration model P1 of the tth training, terminate the tth training, and go to S4; otherwise, go to S5;

[0009] S4, when a preset condition is met, train the migration model P1 for a preset iteration round E2 on the NPU using a training subset, obtain a migration model P2, update the migration model P1 to the migration model P2, let t = t + 1, and go to S2; wherein the preset condition includes: t = 1, or t > 1 and the complexity of the computational graph g t is less than the complexity of the computational graph g t-1 ; the training subset is a subset of the target detection training set; the number of image samples in the training subset is less than a preset number;

[0010] S5, continue the subsequent iteration round training of the migration model P1 on the NPU using the target detection training set until the migration model P1 converges, thereby completing the model training.

[0011] Further preferably, the test drop rate of the tth training is:

[0012]

[0013] wherein Loss1 is the target detection loss value of the migration model P1 in the first iteration round of the tth training; Loss E1 is the target detection loss value of the migration model P1 in the E1th iteration round of the tth training.

[0014] Further preferably, the migration target detection model is a model after migrating a target detection model developed on a GPU platform to an NPU through an automatic migration tool.

[0015] Further preferably, the migration target detection model is a binary-compiled optimized model; wherein, when the shapes of the input tensor and the output tensor of the target detection model developed on the GPU platform are fixed, the compilation mode in the automatic migration tool is set to a fixed shape compilation mode;

[0016] When the shapes of the input tensor and the output tensor of the target detection model developed on the GPU platform are not fixed, the compilation mode in the automatic migration tool is set to a dynamic shape compilation mode.

[0017] Further preferably, in the process of training the migration model P1 using the target detection training set on the NPU, in the data loading stage, the image sample data in the target detection training set is allocated to the lock page memory on the NPU.

[0018] Further preferably, in the process of training the migration model P1 using the target detection training set on the NPU, in the model training stage, the image sample data in the target detection training set is migrated from the host memory to the NPU device memory using a non-blocking data delivery method.

[0019] Further preferably, the migration target detection model further comprises a gradient scaler.

[0020] Further preferably, the above-mentioned target detection training set is a labeled PCB image training sample set; the label includes whether there is a defect and the location information of the defect.

[0021] Further preferably, the method for obtaining the PCB image training sample set comprises:

[0022] Obtaining an original PCB image set; wherein, the original PCB image set only includes defect PCB images;

[0023] Cutting each PCB image in the original PCB image set according to a preset overlap rate to obtain a first image set;

[0024] Selecting a plurality of defect PCB images and a plurality of non-defect PCB images from the first image set so that the ratio of the two is a preset ratio, and labeling the area where the defect is located in the defect PCB image to obtain a second image set;

[0025] Performing one or more of scaling operation, stretching operation, rotation operation, color transformation operation, brightness and contrast enhancement operation, and adding Gaussian noise operation on the defect PCB images in the second image set.

[0026] In a third aspect, the present application provides a target detection method based on NPU, comprising:

[0027] inputting an image to be classified into a pre-trained migration target detection model on an NPU for classification;

[0028] The migration target detection model is a model obtained by migrating a target detection model developed on a GPU to an NPU.

[0029] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0030] 1. The present application provides a training method for a migration model from a GPU to an NPU. Through a two-stage training strategy, the model can find a better convergence starting point on a complex loss graph. In each training process, first, the migration model P1 is trained in the first stage using a target detection training set on the NPU. After a preset iteration E1, the test drop rate of the current training is calculated. When the test drop rate of the current training is less than the preset rate, it indicates that the model may not converge. Therefore, the migration model P1 is trained in the second stage using a subset of the target detection training set for a preset iteration E2. The model first learns general target features and obtains a good weight initial value, so that the model has basic feature extraction capability, and then the next training begins. From the second training, after the first stage of training for a preset iteration E1, the complexity of the computation graph under the current training is monitored to monitor the training state in real time. Through the above operation, the migration model P1 can converge faster and more stably on the target detection training set. The entire process is simple and can improve the training speed and stability of the training process of the migration target detection model migrated from the GPU to the NPU on the NPU in a simple and efficient manner.

[0031] 2. Further, the training method for a migration model from a GPU to an NPU provided by the present application is a binary-compiled optimized model, which enables the NPU graph compiler to perform deeper operator fusion and optimization to generate a binary file with better execution performance.

[0032] 3. Further, the training method for a migration model from a GPU to an NPU provided by the present application, in the data loading stage, image sample data in the target detection training set is allocated to the lock page memory on the NPU, ensuring that the data resides in the physical memory and avoiding frequent page swapping in and out of the operating system, thereby significantly accelerating the data transmission from the CPU to the NPU and further improving the training efficiency.

[0033] 4、Further, the GPU to NPU migration model training method provided by the application adopts a non-blocking data issuing mode to migrate image sample data in the target detection training set from the host memory to the NPU device memory in the model training stage, realizes parallel execution between CPU data preprocessing and NPU model calculation, and reduces the waiting time.

[0034] 5、Further, the GPU to NPU migration model training method provided by the application, the migration target detection model further comprises a gradient scaler to prevent gradient overflow or underflow and ensure the stability of the mixed precision training.

[0035] 6、Further, the GPU to NPU migration model training method provided by the application, when processing the PCB defect detection task, a series of data enhancement processes are performed on the PCB image, including cutting, screening, transformation and other operations, to generate a variety of training samples. This not only increases the size and diversity of the data set, but also improves the recognition ability of the model to defects under different conditions, thereby enhancing the robustness and generalization performance of the model. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 The GPU to NPU migration model training method flowchart provided by the embodiments of the application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0038] In order to achieve the above-mentioned purpose, in a first aspect, the application provides a GPU to NPU migration model training method, as shown in Figure 1 , comprising:

[0039] S1, let the training times t = 1; initialize the migration model P1 as the migration target detection model to be trained; wherein the migration target detection model is a model obtained by migrating the target detection model developed on the GPU platform to the NPU;

[0040] S2, start the tth training of the migration model P1 using the target detection training set on the NPU; after a preset iteration E1, calculate the test drop rate of the tth training; wherein the test drop rate is the drop rate of the target detection loss value of the migration model P1 in the preset iteration E1;

[0041] S3, determining whether a test drop rate of the tth training is less than a preset rate, if yes, recording a calculation graph g of the tth training migration model P1 t , terminating the tth training, and turning to S4; otherwise, turning to S5;

[0042] S4, when a preset condition is met, training the migration model P1 for a preset iteration number E2 on the NPU using a training subset, obtaining a migration model P2, updating the migration model P1 to the migration model P2, setting t = t + 1, and turning to S2; wherein the preset condition includes: t = 1, or t > 1 and the complexity of the calculation graph g t is less than the complexity of the calculation graph g t-1 ; the training subset is a subset of the target detection training set; and the number of image samples in the training subset is less than a preset number;

[0043] It should be noted that when the preset condition is not met, the training fails, and the operation ends.

[0044] S5, continuing to train the migration model P1 for a subsequent iteration number on the NPU using the target detection training set until the migration model P1 converges, thereby completing the model training.

[0045] It should be noted that the above target detection model can be any existing target detection model, such as R-CNN model, YOLO series model (such as YOLOV1, YOLOV3, YOLOV5, YOLOV9, etc.), SSD model, etc., which is not limited here.

[0046] It should be noted that there are many ways to calculate the test drop rate, which is not limited here, such as the test drop rate of the tth training is:

[0047]

[0048] Wherein, Loss i is a target detection loss value of the migration model P1 in the ith iteration of the tth training; Loss j is a target detection loss value of the migration model P1 in the jth iteration of the tth training; the ith iteration and the jth iteration are any two different iterations from the first iteration to the E1th iteration in the tth training, and i is less than j.

[0049] Preferably, in an optional implementation, the test drop rate of the tth training is:

[0050]

[0051] Wherein, Loss1 is the target detection loss value of the migration model P1 in the first iteration round of the tth training; Loss E1 is the target detection loss value of the migration model P1 in the E1th iteration round of the tth training.

[0052] In an optional implementation, the migration target detection model is a model obtained by migrating a target detection model developed on a GPU platform to an NPU through an automatic migration tool. It should be noted that any existing automatic migration tool can be used, such as a PyTorch GPU2Ascend migration tool, and the present application is not limited in this regard.

[0053] In an optional implementation, the migration target detection model is a model optimized through binary compilation. It should be noted that there are various ways of binary compilation optimization, and the present application is not limited in this regard. Preferably, in an optional implementation, when the shapes of the input and output tensors of the target detection model developed on the GPU platform are fixed, the compilation mode in the automatic migration tool is set to a fixed shape compilation mode; when the shapes of the input and output tensors of the target detection model developed on the GPU platform are not fixed, the compilation mode in the automatic migration tool is set to a dynamic shape compilation mode.

[0054] In an optional implementation, in the process of training the migration model P1 on the NPU using the target detection training set, in the data loading stage, the image sample data in the target detection training set is allocated to the lock page memory on the NPU.

[0055] In an optional implementation, in the process of training the migration model P1 on the NPU using the target detection training set, in the model training stage, the image sample data in the target detection training set is migrated from the host memory to the NPU device memory using a non-blocking data delivery method.

[0056] In an optional implementation, the migration target detection model further comprises a gradient scaler.

[0057] It should be noted that there are various target detection tasks to which the present application can be applied, such as panoramic scene segmentation, key point detection, vehicle detection, etc., and the present application is not limited in this regard. In an optional implementation, the above-mentioned target detection task is PCB defect detection, and at this time, the above-mentioned target detection training set is a labeled PCB image training sample set; the label includes whether there is a defect and the location information of the defect.

[0058] It should be noted that the above-mentioned PCB image training sample set can be any existing PCB image training sample set. Preferably, in an optional implementation, the method for obtaining the PCB image training sample set comprises:

[0059] obtain a set of original PCB images, wherein the set of original PCB images only includes defect PCB images;

[0060] cut each PCB image in the set of original PCB images according to a preset overlap rate to obtain a first image set;

[0061] select a plurality of defect PCB images and a plurality of non-defect PCB images from the first image set, so that the ratio of the two is a preset ratio, and label the area where the defect is located in the defect PCB image to obtain a second image set;

[0062] perform one or more of a scaling operation, a stretching operation, a rotation operation, a color transformation operation, a brightness and contrast enhancement operation, and a Gaussian noise addition operation on the defect PCB images in the second image set.

[0063] Embodiment 1

[0064] The present application will be described in detail below taking the training of a target detection model YOLOV9 on an Ascend NPU for a super large target detection training set as an example.

[0065] After the open source YOLOV9 PyTorch code is transplanted to the Ascend NPU platform through an automatic migration tool, it is directly trained on a certain super large target detection training set (the number of samples of which is 70k). The following problems are encountered: the training speed is only 1 / 10 of that of a GPU, the Loss curve is violently oscillated and cannot be converged, and the error of gradient being NaN (Not a Number) frequently occurs. The main reasons are as follows:

[0066] A super large data set means an extremely complex loss (Loss) graph. The model starts training from a random initialization state, and it is difficult to find an effective convergence path. The automatic differentiation mechanism on the NPU may be different from the original framework, resulting in unstable gradient calculation. In dynamic graph frameworks such as PyTorch, the CPU is responsible for sequentially sending calculation operators (Operators) to the NPU for execution. In an ideal state, the computing pipeline of the NPU should be continuously full. However, if the CPU sending speed cannot keep up with the NPU execution speed, the NPU will be idle, causing waste of computing power. Conversely, if the CPU sends too fast, the bottleneck is on the execution efficiency of the NPU. The core of performance optimization is to shorten the operator sending time (Host-side) of the CPU and the operator execution time (Device-side) of the NPU.

[0067] To solve the above problems, the embodiment provides a GPU to NPU migration model training method. Considering the difficulty of directly training a super large data set, in each training process, first, the migration YOLOV9 model is trained using the super large target detection training set; after a preset iteration E1 (in the embodiment, the preset iteration E1 is 30poch), the test drop rate of the current training is calculated, when the test drop rate of the current training is less than a preset rate (in the embodiment, the preset rate is 0.075), the calculation graph g t of the migration model P1 of the t-th training is recorded, and when the preset condition is met, the migration YOLOV9 model is trained using a standard data set COCO128 (a subset of the target detection training set) for a preset iteration E2 (in the embodiment, the preset iteration E2 is 30poch), this stage aims to let the model learn general target features and obtain a good weight initial value. Repeat the above process (at this time, since the model already has basic feature extraction capability, it can converge faster and more stably on the super large target detection training set), until the test drop rate of the current training is greater than or equal to the preset rate, at this time, the migration model P1 is trained using the super large target detection training set on the NPU for subsequent iteration, until the migration YOLOV9 model converges, thereby completing the model training. In the above process, if the preset condition is not met, it means that the training fails, and the operation is ended. The preset condition includes: t=1, or t>1 and the complexity of the calculation graph g t is less than the complexity of the calculation graph g t-1 ; the training subset is a subset of the target detection training set; the number of image samples in the training subset is less than a preset number (in the embodiment, the preset number is 128). In an optional implementation, the complexity of the calculation graph is obtained through an adjacency list: O(|V|+|E|); wherein the calculation graph is regarded as a directed acyclic graph G=(V,E); V is a node in the directed acyclic graph, representing an operator in the calculation graph; E is an edge in the directed acyclic graph, representing a variable in the calculation graph; O is a complexity factor representation form.

[0068] The migration target detection model in this embodiment further includes a gradient scaler to prevent gradient overflow or underflow and ensure the stability of mixed precision training. During the training process, the gradient scaler is used to adjust the gradient calculation of the migration target detection model optimizer to maintain gradient stability. In the training script, a gradient scaler (torch_npu.amp.GradScaler) is configured for the optimizer (such as AdamW) when mixed precision training is used. By dynamically adjusting the scale factor (scale_factor), the loss is multiplied by the factor before back propagation, which amplifies the gradient value and prevents it from underflowing to zero in FP16; before updating the weight, the gradient is divided by the factor to restore its original size, thereby effectively preventing gradient vanishing and explosion.

[0069] The training system in this embodiment is configured to optimize operator compilation and data transmission, including enabling fixed shape operator compilation mode, using locked page memory when loading data, and transmitting data to NPU in a non-blocking manner.

[0070] Specifically, considering that the input image size of the YOLOV9 model is usually fixed (for example, 640x640) during training and inference. Therefore, set SOC_VERSION = Ascend910B (example version) in the system environment variable or compilation option and configure the fixed shape mode (i.e., fixed shape compilation mode). The GE (Graph Engine) of the NPU performs more aggressive operator fusion during graph compilation, fusing multiple small operators into one large fused operator, reducing Kernel startup overhead and memory interaction, and improving performance. For the scenario where the input and output tensor shape (Shape) of the model is fixed, the "fixed Shape" compilation mode is enabled. This enables the NPU's graph compiler (Graph Engine) to perform more in-depth operator fusion and optimization, generating binary files with better execution performance.

[0071] During the data loading stage of the training process, the image sample data in the target detection training set is allocated to the locked page memory on the NPU. Specifically, when defining the DataLoader of PyTorch, set pin_memory = True. Locked page memory ensures that data resides in physical memory, avoiding frequent page swapping in and out of the operating system, thereby significantly accelerating data transmission from CPU to NPU.

[0072] In the model training phase of the training process, the image sample data in the target detection training set is migrated from the host memory to the NPU device memory in a non-blocking data delivery manner (non_blocking=True). Specifically, in the training loop, when data is transferred from the CPU to the NPU,.to('npu:0', non_blocking=True) is called. The above non-blocking data transmission manner realizes the parallel execution between CPU data preprocessing and NPU model calculation, and reduces the waiting time.

[0073] Through the above design, the pipeline of data preprocessing and loading is more smooth, and the CPU can prepare the next batch of data in parallel with the NPU calculating the current batch of data, which significantly reduces the idle time of the NPU caused by data waiting.

[0074] Through the above operation, the training performance of the YOLOV9 model on the Ascend NPU is fundamentally improved: in terms of training speed: compared with the initial migration version, the overall training speed is improved by 8-10 times, close to the training speed on a high-performance GPU; in terms of training stability: the loss curve is smooth and descending, the gradient NaN problem is completely eliminated, and the model successfully converges on a super large data set; in terms of target detection accuracy: the model finally reaches the mAP (mean Average Precision) index that meets the expected requirements. It can be seen that the multi-level model training method proposed in the present application is effective and superior, and the present application can significantly improve the training speed and stability of the model on the NPU, reduce the engineering complexity of optimization and debugging, and has important application value.

[0075] Embodiment 2

[0076] In order to further illustrate the training method of the GPU-to-NPU migration model provided by the present application, a specific target detection task will be further described below. The target detection task in this embodiment is a PCB image defect detection task.

[0077] In the manufacturing process of printed circuit boards (PCB), automatic optical inspection (AOI) systems are indispensable tools to ensure product quality and reliability. AOI systems capture PCB images using high-resolution cameras and use image processing algorithms to identify possible defects or flaws, such as poor solder joints, missing components, or positional offsets. Traditional AOI systems mainly rely on predefined rules and template matching for detection, but this approach is less adaptable to complex and varied types of defects. With the development of machine learning and deep learning technologies, image-based defect detection algorithms have been widely applied, bringing significant improvements to automated production and quality control.

[0078] The embodiment considers that most of the existing improvement measures focus on optimizing the detection model architecture itself, such as introducing a more complex network structure, using pre-trained model transfer learning, or designing a specific loss function to improve the model performance. However, these methods often require a large amount of computing resources and may not completely overcome the challenges caused by the limitations of the training data set itself. In reality, it is difficult to obtain a sufficient number and high quality of defect samples. In addition, if the non-defect background pictures are not handled properly, it may cause imbalance between positive and negative samples, affecting the learning effect and generalization ability of the model. Therefore, the embodiment starts from how to effectively expand and optimize the training data set to improve the accuracy and robustness of the detection model.

[0079] In the embodiment, data augmentation operations are first performed on the original PCB image set; wherein the original PCB image set only includes defect PCB images;

[0080] For the original PCB image set x0, any size of the original PCB picture in x0 is cut into a k*k size square sub-picture, and the sub-picture overlap rate is selected as λ1, and the new data set obtained is x1. In the embodiment, k is 512 or 640, and λ1 is 0.1.

[0081] All the sub-pictures with defects in x1 are selected to form an image set x2. The sub-pictures without defects in x1 are selected, and a part of them is randomly discarded, so that the number of the remaining part is λ2 times the number of the image set x2, and the remaining part is used as the background picture, that is, the image set x3, which is used for training. In the embodiment, the value of λ2 is preferably 1. Through the above operations, the balance between positive and negative samples is ensured, the learning deviation caused by the imbalance of samples is avoided, and the learning effect and accuracy of the model are improved.

[0082] The pictures in the image set x2 are scaled, stretched and rotated, and the specific method is as follows: the pictures are enlarged to λ3 times the area and rotated by 180 degrees to obtain the image set x 4-1 ; the pictures are reduced to 1 / λ3 times the area and rotated by 90 degrees to obtain the image set x 4-2 ; the pictures are contracted to λ4 times the area and rotated by 270 degrees to obtain the image set x 4-3 . The image sets x 4-1 , x 4-2 , and x 4-3 are combined to obtain the image set x4. In the embodiment, λ3 is 4 and λ4 is 2 / 3.

[0083] The pictures in the image set x2 are color transformed, and the specific method is as follows: a picture t is taken out, and each pixel value [r, g, b] of t is regarded as a vector v, and v is set as vw, wherein w is a three-row and three-column matrix, and a new picture t' is obtained. The above operation is performed on each picture to obtain the image set x5 from the image set x2.

[0084] Further, a preferred solution is to perform the above transformation five times and combine the results of the five times to obtain the final image set x5. The w matrix of the five times is different, respectively: 1 1 0 0 1 1 1 0 0 0 1 0 0 0 1

[0086] 0 0 1,1 0 0,0 1 1,1 0 1,1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[0088] The images in the image set x2 are subjected to brightness and contrast enhancement, the brightness of the images is increased by λ5 times, and the contrast value of the images is increased by λ6, to obtain an image set x6.

[0089] Further, the value of λ5 is preferably 1.5, and the value of λ6 is preferably 1.5.

[0090] The images in the image set x2 are subjected to Gaussian noise addition. For each image t in the image set x2, each value y in each pixel point value [r, g, b] of t is set to y = y + ηN(0, 1), where N(0, 1) is a standard normal distribution, and η is a parameter. Thus, an image set x7 is obtained. In this embodiment, η is preferably 0.25.

[0091] The images in the image set x3 are subjected to random rotation. Specifically, for each image in the image set x3, there is a 1 / 4 probability of rotating 90 degrees, a 1 / 4 probability of rotating 180 degrees, and a 1 / 4 probability of rotating 270 degrees. Thus, an image set x8 is obtained.

[0092] The images in the image set x8 are subjected to random color transformation. Specifically, for each image in the image set x8, there is a λ7 probability of random color transformation. The specific method of random color transformation is as follows: for each image, a three-row and three-column matrix w is randomly generated, and the same color transformation as the above color transformation is performed on the image. Thus, an image set x9 is obtained. In this embodiment, λ7 is preferably 1 / 5.

[0093] The image sets x2, x4, x5, x6, x7, and x9 are combined to form a new data set, which is the final PCB image training sample set.

[0094] The above data enhancement method is used to generate high-quality and diverse PCB image training sample set, and to ensure that each image input into the migrated YOLOv9 model meets the expected format and size requirements. Specifically, the pictures in the original PCB image set are cut into 512*512 square sub-pictures, and the overlap rate of each sub-picture is 0.1; the flawless pictures are randomly discarded to the same number as the flawed pictures, and then 1 / 5 of the pictures are randomly colored, all pictures are randomly rotated, and integrated into the final data set; the flawed pictures are respectively not treated, colored as red, blue, cyan, orange, purple, added with Gaussian noise with an intensity of 0.25, set to 1.5 times of brightness contrast, rotated, and enlarged by 4 times or reduced by 4 times or shrunk to 2 / 3, and integrated into the final data set. Finally, a rich and diverse enhanced data set is generated.

[0095] Then, the related configurations of the YOLOv9 model are adjusted according to the hardware characteristics of the Ascend NPU 910b, and the training is ready to start. Specifically, the configurations of the migrated YOLOv9 model are adjusted according to the hardware characteristics, including but not limited to the batch size, the learning rate scheduling strategy and other hyperparameter settings, and necessary operator migration analysis and optimization work. The loss function of the migrated YOLOv9 model is replaced with the Inner-EIoU loss function, which focuses on improving the accuracy of target frame prediction, especially in small target detection. The migrated YOLOv9 model in this embodiment is the model after the model is migrated to the Ascend NPU 910b platform through the Ascend Toollist automatic migration tool.

[0096] The training method of the GPU-to-NPU migration model provided in Embodiment 1 of the application is used for training; during the training process, the Inner-EIoU loss function is used to guide the learning process of the model to ensure the accuracy of the detection frame. At the same time, the Ascend PyTorchProfiler tool is used to closely monitor the training progress, and the performance state of the model during training is analyzed in all directions. Through upgrading the high-performance library, using the lock page memory, the NPU affinity optimizer, the non-blocking data delivery, the operator binary compilation optimization, the NPU affinity API scanning, and timely adjusting the model configuration to achieve the best performance. Through a series of optimization measures, the model can normally run on the Ascend NPU 910b. The model is exported as an onnx file, so that it can run the model inference on a 2-core 8G CPU. The related technical solutions are the same as those in Embodiment 1 of the application, and will not be repeated here.

[0097] After training, the performance of the trained model is evaluated by the test set, the detection ability of the model for various types of PCB defects is checked, and the output information such as defect type, number, position and confidence is recorded to judge the effectiveness of the model.

[0098] After sufficient training and verification, the model is successfully deployed in Huawei Cloud, realizing real-time and efficient detection of PCB board defects, greatly improving the automation level and quality control capability of production.

[0099] In summary, the embodiment aims to solve the problems of insufficient training data, lack of sample diversity and insufficient recognition ability of the model under complex conditions in the prior art. Through a series of data enhancement strategies and optimized YOLOv9 detection model configurations, the present application has the following advantages: (1) improving the diversity and representativeness of the data set, enhancing feature expression; (2) high detection efficiency and accuracy; (3) adapting to special hardware platforms, capable of running on Ascend NPU910b. The embodiment significantly improves the robustness and precision of PCB board defect detection, promotes the development of electronic manufacturing industry, and improves the quality control level of electronic manufacturing industry.

[0100] In a third aspect, the present application provides a target detection method based on NPU, comprising:

[0101] inputting the image to be classified into the pre-trained migration target detection model on the NPU for classification;

[0102] The migration target detection model is a model migrated from a target detection model developed on a GPU platform to an NPU; and the pre-trained migration target detection model is pre-trained by using the training method provided in the first aspect of the present application.

[0103] The related technical solutions are the same as the training method provided in the first aspect of the present application, which will not be repeated here.

[0104] Those skilled in the art will readily understand that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A training method of a GPU-to-NPU migration model, characterized in that, The method comprises the following steps: S1, set training times t = 1; Initialize a migration model P1 as a migration target detection model to be trained; The migration target detection model is a model after a target detection model developed on a GPU platform is migrated to an NPU; S2, start the tth training of the migration model P1 on the NPU by using a target detection training set; After a preset iteration round E1, calculate a test drop rate of the tth training; The test drop rate is a drop rate of a target detection loss value of the migration model P1 in the preset iteration round E1; S3, determining whether the test descent rate of the tth training is less than a preset rate, if yes, recording the calculation graph g of the migration model P1 of the tth training t , terminating the tth training, and turning to S4; otherwise, turning to S5; S4, when a preset condition is met, training the transfer model P1 by using a training subset on the NPU for a preset iteration number E2 to obtain a transfer model P2; updating the transfer model P1 to the transfer model P2, setting t=t+1, and turning to S2; the preset condition includes: t=1, or t>1 and the complexity of the computational graph g t is less than the complexity of the computational graph g t-1 ; the training subset is a subset of the target detection training set; the number of image samples in the training subset is less than a preset number. S5, continue the subsequent iteration round training of the migration model P1 on the NPU by using the target detection training set until the migration model P1 converges, thereby completing the model training.

2. The training method of claim 1, wherein, The test drop rate of the tth training is: Wherein, Loss1 is the target detection loss value of the migration model P1 in the first iteration round of the t-th training; Loss E1 is the target detection loss value of the migration model P1 in the E1-th iteration round of the t-th training.

3. The training method of claim 1, wherein, The migration target detection model is a model after a target detection model developed on a GPU platform is migrated to an NPU by using an automatic migration tool.

4. The training method of claim 3, wherein, The migration target detection model is a model after binary compilation optimization. When shapes of input tensors and output tensors of the target detection model developed on the GPU platform are fixed, a compilation mode in the automatic migration tool is set as a fixed shape compilation mode. When the shapes of the input tensors and the output tensors of the target detection model developed on the GPU platform are not fixed, the compilation mode in the automatic migration tool is set as a dynamic shape compilation mode.

5. The training method of claim 1, wherein, In the process of training the migration model P1 on the NPU by using the target detection training set, in a data loading stage, image sample data in the target detection training set is allocated to a lock page memory on the NPU.

6. The training method according to any one of claims 1 to 5, characterized in that, In the process of training the migration model P1 on the NPU by using the target detection training set, in a model training stage, the image sample data in the target detection training set is migrated from a host memory to an NPU device memory in a non-blocking data delivery manner.

7. The training method according to any one of claims 1 to 5, characterized in that, In the process of training the migration model P1 on the NPU by using the target detection training set, in a model training stage, the image sample data in the target detection training set is migrated from a host memory to an NPU device memory in a non-blocking data delivery manner.

8. The training method according to any one of claims 1 to 5, characterized in that, The migration target detection model further comprises a gradient scaler.

9. The training method according to any one of claims 1 to 5, characterized in that, The target detection training set is a labeled PCB image training sample set; the label comprises whether there is a defect and position information of the defect; The method for obtaining the PCB image training sample set comprises: Obtain an original PCB image set; wherein the original PCB image set only comprises defect PCB images; Cut each PCB image in the original PCB image set according to a preset overlap rate to obtain a first image set; Select a plurality of defect PCB images and a plurality of non-defect PCB images from the first image set so that the ratio of the two is a preset ratio, and label the area where the defect is located in the defect PCB image to obtain a second image set; Perform one or more of a scaling operation, a stretching operation, a rotating operation, a color transformation operation, a brightness and contrast enhancement operation and a Gaussian noise adding operation on the defect PCB images in the second image set. 10.A method for target detection based on NPU, characterized in that, The application relates to a method for training a target detection model on a GPU platform. The method comprises the following steps: inputting an image to be classified into a pre-trained migration target detection model on an NPU for classification; wherein the migration target detection model is a model obtained by migrating a target detection model developed on a GPU platform to an NPU; and the pre-trained migration target detection model is obtained by pre-training according to the training method in any one of claims 1-9.

Images