A data enhancement method for PID image category imbalance

Abstract

The application provides a data enhancement method for PID image category imbalance, and relates to the technical field of image processing data enhancement.The application can directionally select one or more categories in a PID image for data enhancement, and fit the number of categories that need to be enhanced after data enhancement to a normal distribution, so that the diversity of target and background combination is increased, the accuracy and robustness of model identification are improved, the generalization ability of the model is enhanced, and the cost and time of manual image collection and labeling are reduced to a certain extent.

Description

Technical Field

[0001] This invention relates to the field of image processing data augmentation technology, specifically a data augmentation method for PID image class imbalance. Background Technology

[0002] Digitizing Piping and Instrumentation Diagrams (PIDs) is a key focus for petrochemical companies. Currently, digitization primarily relies on manual drawing by professional engineers, which is inefficient and prone to errors. While Deep Convolutional Neural Networks (CNNs) improve the efficiency of PID digitization, severe class imbalance in PIDs leads to low accuracy and poor generalization ability in trained models. Data augmentation methods, such as increasing the number of targets and fusing targets and background, are common approaches to address class imbalance. Mixup, which proportionally merges two images, can blur and occlude PID images, affecting the model's detection capabilities. Mosaic, which randomly crops and scales four images before stitching them together, increases the number of targets and the diversity of the detected background, but it cannot specifically increase the number of targets of one or more classes in the PID image. Furthermore, the target proportions in all PID images for the same project are fixed, and Mosaic's random scaling method also reduces target recognition accuracy. Therefore, a new data augmentation method is needed to address the class imbalance problem in PID images, thereby improving the model's accuracy, robustness, and generalization ability. Summary of the Invention

[0003] The purpose of this invention is to provide a data augmentation method for class imbalance in PID images. To achieve the above objective, this invention provides the following technical solution: A data augmentation method for class imbalance in PID images, comprising the following steps:

[0004] S1. Calculate the augmentation categories and category label dictionary, store the augmentation categories in the augmentation list, generate a normal curve and perform a linear transformation to determine the number of each augmentation category after data augmentation is completed;

[0005] S2. Use the roulette wheel algorithm to obtain four enhancement categories. For each enhancement category, randomly select the corresponding annotation file from the category label dictionary. Then, randomly select the target belonging to the current enhancement category from the annotation file. Based on the positional relationship between the center point of the target annotation box and the PID image, crop the PID image containing the target of the current enhancement category as the cropped image. Use a formula based on width and height overlap to calculate the overlap between the target and the cropped image.

[0006] S3. Generate an enhanced image with the same size as the PID image. Select the center point of the enhanced image as the stitching point of the cropped image. Place the cropped image in the upper left, upper right, lower left, and lower right of the enhanced image in sequence. Generate the annotation information of the enhanced image. Update the number of enhanced categories in the category label dictionary. Remove categories in the enhancement list whose number of categories is greater than or equal to the number of enhanced categories after data enhancement.

[0007] S4. Repeat steps S2-S3. If the length of the augmentation list is equal to 0, the data augmentation for PID image class imbalance is completed.

[0008] Preferably, in step S1, the augmentation categories and category label dictionary are statistically analyzed, the augmentation categories are stored in the augmentation list, a normal curve is generated and linearly transformed, and the number of each augmentation category after data augmentation is completed is determined, including:

[0009] S11. The PID dataset includes PID images and annotation files. Iterate through all annotation files to obtain the number of each category and record the annotation file where each category is located. Manually set a threshold and use the categories with a number below the threshold as augmentation categories. Then, use the augmentation categories, the number of augmentation categories, and the annotation file where the augmentation categories are located as a category label dictionary.

[0010] S12. Use the Python normal distribution algorithm package to generate a normal curve. Perform a linear transformation on the normal curve using the following formula to obtain the number of each augmentation category after data augmentation: Where A is the hyperparameter of the difference between the highest and lowest points of the normal curve after linear transformation, and B is the hyperparameter of the lower limit of the lowest point of the normal curve after linear transformation.

[0011] Preferably, the step of using the roulette wheel algorithm to obtain four enhancement categories, randomly selecting the corresponding annotation file from the category label dictionary for each enhancement category, then randomly selecting a target belonging to the current enhancement category from the annotation file, and cropping the PID image containing the target of the current enhancement category based on the positional relationship between the center point of the target annotation box and the PID image, and using a formula based on width and height overlap to calculate the overlap between the target and the cropped image, including:

[0012] S21. Calculate the total number of enhanced categories in the category label dictionary. Then, divide the number of each enhanced category by the total number to obtain the probability of each enhanced category being selected. Sort the categories by probability from largest to smallest and calculate the cumulative probability of each enhanced category using the following formula: Where p(x) j q(x) represents the probability that the j-th enhancement category is selected. i) is the cumulative probability of the i-th enhanced category; a random number is generated in the interval [0,1]. If the random number falls within q(x) i ) and q(x i+1 If the i-th enhancement category is selected between () and () the i-th enhancement category is selected;

[0013] S22. Let the coordinates of the top-left and bottom-right corners of the target annotation box be (x1, y1) and (x2, y2) respectively. The coordinates of the center point of the target annotation box are then (x_center, y_center).

[0014] S23. Based on the path attribute in the annotation file, find the corresponding PID image in the PID dataset. Let the width and height of the PID image be img_w and img_h, respectively. The positional relationship between the center point of the target annotation box and the PID image, as well as the coordinates of the upper left and lower right corners of the corresponding cropped image, include:

[0015]

[0016] S24. The formula based on width and height overlap is as follows:

[0017]

[0018] Where w1 and h1 represent the width and height of the overlap between the target and the cropped image, respectively, and w2 and h2 represent the actual width and height of the target, respectively. If w overlap and h overlap If both w and w are equal to 1.0 and the target is an enhancement category, then retain the target; if w overlap and h overlap It is equal to 1.0, but the target does not belong to the enhancement category, or w overlap and h overlap If the value is less than 0.7, the target is filled with the background of the PID image.

[0019] Preferably, the process involves generating an enhanced image with the same size as the PID image, selecting the center point of the enhanced image as the stitching point for the cropped image, placing the cropped image sequentially at the upper left, upper right, lower left, and lower right of the enhanced image, generating annotation information for the enhanced image, updating the number of enhanced categories in the category label dictionary, and removing categories from the enhancement list whose number of categories is greater than or equal to the number of enhanced categories after data enhancement, including:

[0020] S31. The target annotation information in the cropped image is obtained by subtracting the coordinates of the target in the PID image from the coordinates of the top left corner of the cropped image.

[0021] S32. Let the width and height of the enhanced image be aug_w and aug_h, respectively. The coordinates of the upper left, upper right, lower left, and lower right of the enhanced image are:

[0022]

[0023] Based on the specific location of the cropped image stitched into the enhanced image, the coordinates of the target annotation box in the cropped image are added to the corresponding coordinates in the upper left, upper right, lower left, and lower right of the enhanced image to obtain the annotation information of the enhanced image. The target categories and quantities of the current enhanced image are counted, and the number of enhanced categories in the corresponding category label dictionary is increased. Attached Figure Description

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention can selectively select one or more categories in the PID image for data augmentation, and fit the number of categories to be augmented after data augmentation to a normal distribution, which increases the diversity of target and background combinations, improves the accuracy and robustness of model recognition, enhances the generalization ability of the model, and reduces the cost and time of manual image collection and annotation to a certain extent.

[0025] Figure 1 This is a flowchart of a data augmentation method for PID image class imbalance according to the present invention.

[0026] Figure 2 This is a bar chart showing the number of each category before data augmentation and the number of augmented categories after data augmentation, provided as an embodiment of the present invention.

[0027] Figure 3 This is a schematic diagram of the positional relationship between the center point of the target annotation box and the PID image, and the corresponding cropped image after width and height overlap processing, provided in an embodiment of the present invention.

[0028] Figure 4 This is a schematic diagram illustrating data augmentation of a PID image according to an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Please see Figures 1 to 4 The diagram illustrates a flowchart of a data augmentation method for PID image class imbalance provided by an embodiment of the present invention, the method comprising the following steps.

[0031] S1. Calculate the augmentation categories and category label dictionary, store the augmentation categories in the augmentation list, generate a normal curve and perform a linear transformation to determine the number of each augmentation category after the data augmentation is completed.

[0032] Iterate through all the annotation files in the PID dataset to obtain the number of each category and record the annotation file containing each category, such as... Figure 2 As shown in ①, the horizontal axis represents 51 categories, and the vertical axis represents the number of each category. The threshold is set to 400, and categories with fewer than 400 are designated as enhanced categories. The 23 enhanced categories with fewer than the threshold are then added to the enhancement list.

[0033] Through formula Generate a normal curve, where 0 ≤ x < 23 and x is an integer, u equals 11, and σ equals 5. Then, perform a linear transformation on the normal curve according to Y = Af(x) + B, where A equals 6000 and B equals 400. Next, sort Y and the number of augmented categories in the category label dictionary from smallest to largest, thus establishing a one-to-one correspondence between Y and the augmented categories. Finally, obtain the number of each augmented category after data augmentation, as shown below. Figure 2 As shown in ②, the horizontal axis represents the 23 augmentation categories, and the vertical axis represents the number of each augmentation category after data augmentation.

[0034] S2. Use the roulette wheel algorithm to obtain four enhancement categories. For each enhancement category, randomly select the corresponding annotation file from the category label dictionary. Then, randomly select the target belonging to the current enhancement category from the annotation file. Based on the positional relationship between the center point of the target annotation box and the PID image, crop the PID image containing the target of the current enhancement category as the cropped image. Use a formula based on width and height overlap to calculate the overlap between the target and the cropped image.

[0035] like Figure 3As shown, the PID image size is 1024×1024. Assume that targets outlined by solid rectangles belong to the enhancement category, while targets outlined by dashed rectangles do not. Furthermore, the roulette wheel algorithm selects targets outlined by solid rectangles as the enhancement category. By reading the annotation file of the PID image, the coordinates of the top-left corner (288, 70) and bottom-right corner (354, 194) of the target annotation box are obtained, leading to the center point coordinates (321, 132). Then, it is determined that the center point of the target annotation box is located in the region... The range of domain ⑥ is determined, and the coordinates of the upper left corner (65,0) and lower right corner (577,512) of the cropped image are determined. Then, the overlap of the solid-line rectangular target and the cropped image is calculated using the formula based on width and height overlap. The overlap of the target and the cropped image is equal to 1, and the target belongs to the enhancement category. The solid-line rectangular target is retained. Although the overlap of the dashed rectangular target and the cropped image is equal to 1, the target does not belong to the enhancement category. The white background of the PID image is used to fill the target box, and finally a 512×512 cropped image is obtained.

[0036] S3. Generate an enhanced image with the same size as the PID image. Select the center point of the enhanced image as the stitching point of the cropped image. Place the cropped image in the upper left, upper right, lower left, and lower right of the enhanced image in sequence. Generate the annotation information of the enhanced image. Update the number of enhanced categories in the category label dictionary. Remove categories from the enhancement list whose number of categories is greater than or equal to the number of enhanced categories after data enhancement.

[0037] like Figure 4 As shown, the middle image represents the augmented image, and the four images on either side represent the PID images. The dashed square boxes in the PID images represent the cropped images corresponding to each image. The augmented and PID images are 1024×1024 pixels in size, and the cropped images are 512×512 pixels in size. Based on the specific position of the cropped images within the augmented image, the coordinates of the target annotation boxes in the cropped images are added to the corresponding coordinates in the augmented image's upper left (0,0), upper right (512,0), lower left (0,512), and lower right (512,512) coordinates to obtain the annotation information for the augmented image. The rectangle in the augmented image is the target annotation box. The target categories and quantities in the current augmented image are then counted, and the number of augmented categories in the corresponding category label dictionary is increased.

[0038] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A data augmentation method for class imbalance in PID images, characterized in that, The method includes: S1. Calculate the augmentation categories and category label dictionary, store the augmentation categories in the augmentation list, generate a normal curve and perform a linear transformation to determine the number of each augmentation category after data augmentation is completed; S2. Use the roulette wheel algorithm to obtain four enhancement categories. For each enhancement category, randomly select the corresponding annotation file from the category label dictionary. Then, randomly select the target belonging to the current enhancement category from the annotation file. Based on the positional relationship between the center point of the target annotation box and the PID image, crop the PID image containing the target of the current enhancement category as the cropped image. Use a formula based on width and height overlap to calculate the overlap between the target and the cropped image. S3. Generate an enhanced image with the same size as the PID image. Select the center point of the enhanced image as the stitching point of the cropped image. Place the cropped image in the upper left, upper right, lower left, and lower right of the enhanced image in sequence. Generate the annotation information of the enhanced image. Update the number of enhanced categories in the category label dictionary. Remove categories in the enhancement list whose number of categories is greater than or equal to the number of enhanced categories after data enhancement. S31. The coordinates of the target bounding box in the cropped image are obtained by subtracting the coordinates of the target in the PID image from the coordinates of the top left corner of the cropped image. S32. Assume the width and height of the enhanced image are aug_w and aug_h, respectively, and the coordinates of the top left, top right, bottom left, and bottom right of the enhanced image are (0, 0), respectively. Based on the specific position of the cropped image stitched into the enhanced image, the coordinates of the target annotation box in the cropped image are added to the corresponding coordinates in the upper left, upper right, lower left, and lower right of the enhanced image to obtain the annotation information of the enhanced image. The target categories and quantities of the current enhanced image are counted, and the number of enhanced categories in the corresponding category label dictionary is increased. S4. Repeat steps S2-S3. If the length of the augmentation list is equal to 0, the data augmentation for PID image class imbalance is completed.

2. The data augmentation method for PID image class imbalance according to claim 1, wherein step S1 involves statistically analyzing the augmentation categories and category label dictionary, storing the augmentation categories in an augmentation list, generating a normal curve and performing a linear transformation, and determining the number of each augmentation category after data augmentation, characterized in that... include: S11. Iterate through all annotation files to obtain the number of each category and record the annotation file where each category is located. Select the categories with a number lower than the manually set threshold as the enhanced categories. Then, use the enhanced categories, the number of enhanced categories, and the annotation file where the enhanced categories are located as the category label dictionary. S12. Generate a normal curve and perform a linear transformation on the normal curve using the following formula to obtain the number of each augmentation category after data augmentation: Where A is the hyperparameter of the difference between the highest and lowest points of the normal curve after linear transformation, B is the hyperparameter of the lower limit of the lowest point of the normal curve after linear transformation, and parameter x is a specific value, where 0 ≤ x < length of the enhanced list and x is an integer, σ, μ, and σ 2 These are the parameters of the normal distribution, where μ is the mean, σ is the standard deviation, and σ0 is the standard deviation. 2 Let Variance be the variance.

3. The data augmentation method for class imbalance in PID images according to claim 1, wherein in step S2, a roulette wheel algorithm is used to obtain four augmentation categories, and for each obtained augmentation category, a corresponding annotation file is randomly selected from the category label dictionary. Then, a target belonging to the current augmentation category is randomly selected from the annotation file. Based on the positional relationship between the center point of the target annotation box and the PID image, the PID image containing the target of the current augmentation category is cropped as the cropped image, and the overlap between the target and the cropped image is calculated using a formula based on width and height overlap. include: The formula based on width and height overlap is as follows: Where w1 and h1 represent the width and height of the overlap between the target and the cropped image, respectively, and w2 and h2 represent the actual width and height of the target, respectively. overlap h is the width of overlap between the target image and the cropped image. overlap It is a high degree of overlap between the target and the cropped image; If w overlap and h overlap If both w and w are equal to 1.0 and the target is an enhancement category, then retain the target; if w overlap and h overlap It is equal to 1.0, but the target does not belong to the enhancement category, or w overlap and h overlap If the overlap is less than θ, then the background of the PID image is used to fill the target, where θ is a hyperparameter of overlap.

Images