Defect category identification method of hub image, medium and equipment

Abstract

According to the hub image defect category identification method, the medium and the equipment, the edge gradient map corresponding to the original hub image is obtained, the gradient direction of each pixel point is obtained, the hub circle parameter is obtained according to the coordinate and the gradient direction of each pixel point, then the blank image with the same size as the original hub image is created, and the defect category of the hub image is identified. Obtaining a hub foreground parameter according to the hub circle parameter, cutting the blank image according to the hub circle parameter to obtain a foreground mask image, obtaining a first contour target image according to the hub foreground parameter and the foreground mask image, and performing feature enhancement on the first hub target image to obtain a three-channel target image; and determining a rotation sampling period according to spokes of the hub, rotating the three-channel image to obtain a plurality of sampling images, and inputting the sampling images of the original hub images into the constructed and trained neural network model to obtain defect categories corresponding to the original hub images. The problems of high recognition complexity, low recognition rate and the like in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, medium and device for identifying defect categories of a wheel hub image. Background Art

[0002] In the field of automobile manufacturing, the installation of tires and wheels is a crucial link. It is not only related to the safety performance of the vehicle, but also a key factor affecting customer satisfaction and corporate image. However, due to the complexity of production management and the uncertainty of human factors, errors or omissions often occur during the wheel assembly process, resulting in the installation of tires and wheels that do not match customer orders. Once this happens, it will not only cause customer complaints and dissatisfaction, but also cause serious damage to the reputation of automobile manufacturers, thereby affecting their market competitiveness. The traditional wheel installation and inspection process often relies on manual inspection, which is not only inefficient, but also easily affected by human factors such as worker fatigue and negligence, resulting in inaccurate inspection results. In addition, once an error is discovered, it is usually in the later stage of the production process. At this time, correction is not only costly, but may also lead to production delays and bring economic losses to the company. In order to solve these problems, the industry has tried to use automation and intelligent technologies to improve the accuracy and efficiency of wheel assembly. However, most of the existing technologies focus on the geometric size detection of the wheel, while ignoring the image feature analysis of the wheel. As a circular target, the image features of the wheel hub have natural periodicity and symmetry in the radial direction, which are often not fully utilized in traditional image processing methods.

[0003] Therefore, how to improve it is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0004] Based on this, the purpose of this application is to provide a method, medium and device for identifying defect categories of wheel hub images to solve at least one technical problem mentioned in the above background technology.

[0005] In a first aspect, the present application provides a method for identifying defect categories of a wheel hub image, comprising:

[0006] Perform edge detection on the original wheel hub image to obtain edge points in the original wheel hub image, construct an edge gradient map, and obtain the gradient direction of each pixel point in the edge gradient map;

[0007] The hub circle parameters are obtained according to the coordinates and gradient direction of each pixel in the edge gradient map;

[0008] Creating a blank image of the same size as the original wheel hub image, obtaining wheel hub foreground parameters according to the wheel hub circle parameters, cropping the blank image according to the wheel hub circle parameters to obtain a foreground mask image, and obtaining a first contour target image according to the wheel hub foreground parameters and the foreground mask image;

[0009] Perform feature enhancement on the first hub target image to obtain a three-channel target image;

[0010] Determine the rotation sampling period according to the spokes of the hub, and obtain a number of sampling images by rotating the three-channel image;

[0011] Construct and train a neural network model that takes the sampled images of each original wheel image as input and the wheel defect category corresponding to the original wheel image as output;

[0012] The sampled images of each original wheel hub image are input into the trained neural network model to obtain the defect category corresponding to each original wheel hub image.

[0013] Further, the steps of performing edge detection on the original wheel hub image, obtaining edge points in the original wheel hub image, constructing an edge gradient map, and obtaining the gradient direction of each pixel point in the edge gradient map include:

[0014] Obtain the gradient value of each pixel in the original wheel hub image. If it is greater than the maximum value of the gradient threshold range, it is determined to be an edge point; if it is less than the minimum value of the gradient threshold range, it is removed; if it is between the minimum and maximum values ​​of the gradient threshold range, determine whether the pixel is adjacent to the edge point. If it is adjacent, it is determined to be an edge point; if it is not adjacent, it is removed to obtain an edge gradient map;

[0015] Get the gradient of each pixel in the edge gradient map in the X and Y directions to get the gradient direction of each pixel.

[0016] Further, the step of obtaining the hub circle parameters according to the coordinates and gradient direction of each pixel point in the edge gradient map includes:

[0017] Create a blank 3D histogram statistics table;

[0018] Set and traverse the radius threshold to obtain the corresponding polar coordinates according to the coordinates of each pixel point and the currently traversed radius, and construct the hub circle parameters with the polar coordinates and the corresponding radius;

[0019] Map each hub circle parameter to a three-dimensional histogram statistics, and obtain a set number of hub circle parameters with the largest number of votes, which are the initial hub circle parameters;

[0020] Obtaining original wheel hub image features of each a priori region of interest in the original wheel hub image; the image features include cosine approximation and gradient variance;

[0021] The hub image features corresponding to each hub circle parameter in each prior region of interest of the edge gradient image are obtained, and it is determined whether the deviation between the hub image features and the original hub image features is less than the deviation threshold. If not, the corresponding circle parameters are eliminated. If yes, the hub circle parameters are determined.

[0022] Furthermore, the specific steps of obtaining the image features of each priori region of interest in the original wheel hub image include:

[0023] Get the grayscale image corresponding to the original wheel hub image;

[0024] Counting the number of grayscale values ​​of each pixel in each prior region of interest in the grayscale image, and establishing a grayscale histogram of each prior region of interest;

[0025] Normalize the grayscale histogram and calculate the cosine approximation of each a priori region of interest;

[0026] According to the gradient value of each pixel point, the gradient variance of each prior region of interest is obtained.

[0027] Furthermore, the step of performing feature enhancement on the first hub target image to obtain a three-channel target image includes:

[0028] Create a blank image twice the size of the original wheel image, draw a circle in the blank image according to the wheel circle parameters and the first cropping ratio, and obtain the wheel foreground mask image;

[0029] A wheel hub foreground image is obtained according to the wheel hub foreground mask image, and a wheel hub foreground parameter is obtained according to the wheel hub circle parameter and the cropping ratio, so as to crop the wheel hub foreground image according to the wheel hub foreground parameter to obtain a second wheel hub target image;

[0030] Obtaining a hub foreground parameter according to the hub circle parameter and the second cropping ratio, and cropping the hub foreground image according to the hub foreground parameter to obtain a third hub target image;

[0031] The first hub target image, the second hub target image, and the third hub target image are normalized and concatenated into a three-channel image.

[0032] Furthermore, the specific structure of the neural network model includes:

[0033] Input layer: used to receive the input sampled image data and then pass the sampled image data to the first convolutional layer;

[0034] The first convolutional layer: connected to the input layer, performs convolution operation on the input image, uses 6 5x5 convolution kernels to extract local features of the image, and obtains 6 initial feature maps;

[0035] The first maximum pooling layer: connected to the first convolutional layer, performs the maximum pooling operation on each initial feature map, reduces the spatial size of each initial feature map, extracts the main features to reduce the amount of calculation, and obtains 6 optimized feature maps;

[0036] The second convolutional layer is connected to the first maximum pooling layer to further extract features from each optimized feature map. It uses 32 5x5 convolution kernels, takes the 6 optimized feature maps of the previous layer as input, and outputs 32 updated feature maps.

[0037] Second maximum pooling layer: connected to the second convolutional layer, performs pooling operation on each updated feature map to further reduce the size of the updated feature map to obtain the pooled feature map;

[0038] The third convolutional layer: connected to the second maximum pooling layer, uses 8 5x5 convolution kernels to further extract the high-level features of the pooling feature map and obtain 8 convolution feature maps;

[0039] The third maximum pooling layer: connected to the third convolutional layer, pools each convolutional feature map to further reduce the size of the feature map and obtain the final feature map;

[0040] The first fully connected layer: connected to the third maximum pooling layer, converts all feature maps into vectors of several nodes, and then introduces nonlinearity through the ReLU activation function;

[0041] The second fully connected layer is connected to the first fully connected layer, further compresses the vectors of several nodes output by the first fully connected layer, and divides them into N defect categories. The more vectors each defect category contains, the higher the corresponding category probability.

[0042] Output layer: connected to the second fully connected layer, the output of the second fully connected layer is converted into a probability distribution using the softmax function to represent the predicted probability of each category, and the defect category with the highest probability is output, which is the final determined wheel defect category.

[0043] Furthermore, the construction and training process of the neural network model includes:

[0044] Build an initial neural network model;

[0045] Obtain a number of original wheel hub images of equal size and their corresponding defect categories, and obtain the sampling images corresponding to each original wheel hub image according to the above steps, with each sampling image and the corresponding defect category as a subset;

[0046] The training data set is input into the initial neural network model to optimize the parameters of the initial neural network model and obtain the final neural network model.

[0047] Further, defect categories include:

[0048] Any one or more of surface defects, porosity defects, surface shrinkage, undercasting defects, cold shut defects and die casting defects.

[0049] In a second aspect, the present application further provides a computer storage medium storing an executable program code; the executable program code is used to execute the defect category identification method of the wheel hub image described in any one of the first aspects.

[0050] In a third aspect, the present application further provides a terminal device, comprising a memory and a processor; the memory stores a program code that can be executed by the processor; the program code is used to execute the defect category identification method of the wheel hub image described in any one of the first aspects.

[0051] The present invention provides a defect category recognition method for a wheel hub image. The method performs edge detection on an original wheel hub image to obtain edge points in the original wheel hub image, constructs an edge gradient map, and obtains the gradient direction of each pixel point in the edge gradient map. The wheel hub circle parameter is then obtained according to the coordinates and gradient direction of each pixel point in the edge gradient map. A blank image of the same size as the original wheel hub image is created, and a wheel hub foreground parameter is obtained according to the wheel hub circle parameter. The blank image is cropped according to the wheel hub circle parameter to obtain a foreground mask image, and a first contour target image is obtained according to the wheel hub foreground parameter and the foreground mask image. The first wheel hub target image is feature enhanced to obtain a three-channel target image. The feature correlation between pixels closer to and farther from the dot in the radial direction of the original hub image is enhanced by splicing and fusion. Then, the rotation sampling period is determined according to the spokes of the hub, and several sampling images are obtained by rotating the three-channel image. The characteristics of the periodic changes in the features brought about by the rotation change are associated by self-rotation around the center of the hub, which is conducive to extracting the hub features. Then, a neural network model is constructed and trained with the sampling images of each original hub image as input and the hub defect category corresponding to the original hub image as output. Finally, the sampling images of each original hub image are input into the trained neural network model to obtain the defect category corresponding to each original hub image. The problems of high recognition complexity and low recognition rate in the existing technology are solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flow chart of a method for identifying defect categories of a wheel hub image according to an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of the location of a priori regions of interest according to an embodiment of the present invention;

[0054] Figure 3 A schematic diagram of the number of spokes of a hub according to an embodiment of the present invention;

[0055] Figure 4Schematic diagram of a sampling image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] It should be noted that if the embodiments of the present invention involve directional indications, such as up, down, left, right, front, back, etc., then the directional indication is only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly. In addition, if the embodiments of the present invention involve descriptions of "first, second", "S1, S2", "step one, step two", etc., then such descriptions are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of indicated technical features or indicating the execution order of the method, etc. Those skilled in the art can understand that anything that does not violate the gist of the invention under the technical concept of the invention should be included in the protection scope of the present invention.

[0058] like Figure 1 As shown, the present invention provides an image feature enhancement method:

[0059] S1: Perform edge detection on the original wheel image to obtain edge points in the original wheel image, construct an edge gradient map, and obtain the gradient direction of each pixel point in the edge gradient map;

[0060] Specifically, it is optional but not limited to setting a gradient threshold range, obtaining a photo of the automobile wheel hub to be detected as the original wheel hub image, and calculating the gradient value of each pixel in the original wheel hub image, so as to extract the corresponding pixel point, i.e., the edge point, according to the magnitude relationship between the gradient threshold range and the gradient value of each pixel point, obtain the edge gradient map corresponding to the original wheel hub image, and use the gradient operator to calculate the gradient direction of each pixel point in the edge gradient map. Preferably, when obtaining the original wheel hub image, the automobile wheel hub is facing the acquisition device, and the center point of the wheel hub is the center point of the original wheel hub image.

[0061] Preferably, the step of performing edge detection on the original wheel hub image to obtain edge points in the original wheel hub image and acquiring the gradient direction of each edge point may optionally include but is not limited to:

[0062] S11: Obtain the gradient value of each pixel in the original wheel hub image. If it is greater than the maximum value of the gradient threshold range, it is determined to be an edge point; if it is less than the minimum value of the gradient threshold range, it is removed; if it is between the minimum value and the maximum value of the gradient threshold range, it is determined whether the pixel is adjacent to the edge point. If it is adjacent, it is determined to be an edge point; if it is not adjacent, it is removed to obtain an edge gradient map;

[0063] Get the gradient value of each pixel in the original wheel image and determine whether it is greater than the gradient threshold range. If it is greater, keep the corresponding pixel as an edge point; if it is less than, remove the corresponding pixel;

[0064] Specifically, it is optional but not limited to setting a gradient threshold range [min, max], then calculating the gradient of each pixel in the X and Y directions respectively, merging to obtain the gradient value of each pixel, and then judging the size relationship between each pixel and the gradient threshold range [min, max]. When the gradient value corresponding to the pixel is greater than max, it proves that the pixel is located in the edge area and is an edge point, and needs to be retained. When the gradient value corresponding to the pixel is less than min, it proves that the pixel is located in the internal area and needs to be removed. The gradient threshold range is arbitrarily set by technicians in this field. Preferably, the gradient threshold range is [30, 40].

[0065] More specifically, when the gradient value of a pixel point is within the gradient threshold range [min, max], it is not possible to directly determine whether the pixel point is an edge point. It is necessary to determine whether it is adjacent to the edge point obtained in step S11. If it is adjacent to any edge point, it is also an edge point. If it is not adjacent to any edge point, it is located in the internal area and is not an edge point. When all non-edge points are removed, the image composed of all edge points is the edge gradient map. It is worth noting that when the pixel point whose gradient value is within the gradient threshold range in step S12 is adjacent to the edge point obtained in step S11, it is also an edge point. Therefore, the pixel point adjacent to the edge point and whose gradient value is within the gradient threshold range is also an edge point.

[0066] S12: Obtain the gradient of each pixel in the edge gradient image in the X and Y directions to obtain the gradient direction of each pixel.

[0067] Specifically, it is optional but not limited to using a Sobel operator to calculate the gradient of each pixel in the edge gradient map in the X and Y directions to obtain the gradient direction of each pixel.

[0068] Preferably, it is optional but not limited to calculating the gradient map of the edge gradient map in the X and Y directions according to formulas 1-1 and 1-2, and then calculating the gradient direction of each pixel point according to formula 1-3:

[0069]

[0070] θ i = arctan(G yi / G xi )1 - 3

[0071] where Gx and Gy are the gradient maps of the edge gradient map in the X and Y directions respectively, and I grad is the edge gradient map, G xi and G yi are the i-th pixel points in the gradient maps in the X and Y directions respectively, and θ i is the gradient direction of the i-th pixel point in the edge gradient map, 0 < i ≤ N, and N is the number of pixel points in each gradient map.

[0072] S2: Obtain the hub circle parameters according to the coordinates and gradient directions of each pixel point in the edge gradient map;

[0073] Specifically, optionally but not limited to, establish a rectangular coordinate system with any corner point of the edge gradient map as the origin, and set a radius threshold [R1, R n , which includes several set radii R1, R2, R3........R n , n is the number of radii. Then, create a three-dimensional histogram statistical table Hist(a, b, R). According to the coordinates of each pixel point and each set radius R, the corresponding polar coordinates (a, b) can be calculated to obtain the hub circle parameters (a, b, R) composed of the polar coordinates and the corresponding set radius. Then, record the number of times the hub circle parameters appear at the corresponding position in the three-dimensional histogram statistical table, that is, increase the vote count by one at the corresponding position of the hub circle parameters in the three-dimensional histogram statistical table. By traversing the pixel point coordinates and set radii, the vote counts of several hub circle parameters in Hist(a, b, R) can be obtained. Finally, detect the set number of hub circle parameters with the highest vote counts to obtain the qualified hub circle parameters, which are the finally determined hub circle parameters.

[0074] Preferably, take the upper left corner point of the edge gradient map as the origin, the direction from the upper left corner to the upper right corner as the x-axis, and the direction from the upper left corner to the lower left corner as the y-axis to construct a rectangular coordinate system to obtain the coordinates of each pixel point. More specifically, the specific accuracy of the set radii included in the radius threshold can be arbitrarily set by those skilled in the art.

[0075] Further preferably, the steps of obtaining the hub circle parameters according to the coordinates and gradient directions of each pixel point in the edge gradient map optionally but not limited to include:

[0076] S21: Create a blank three-dimensional histogram statistical table;

[0077] Specifically, but not limited to, creating a blank three-dimensional histogram statistical table, with the values of each hub circle parameter (a, b, R) on the horizontal axis and the number of votes corresponding to each hub circle parameter on the vertical axis.

[0078] S22: Set and traverse the radius threshold to obtain the corresponding polar coordinates based on the coordinates of each pixel point and the currently traversed radius, and construct hub circle parameters with the polar coordinates and the corresponding radius;

[0079] Specifically, but not limited to, setting the radius threshold [R1, R n , and then traversing the radii and the coordinates of each pixel point within the radius threshold range. A corresponding polar coordinate can be calculated for each pixel point coordinate and radius, and the polar coordinate and the corresponding radius form hub circle parameters. After the traversal is completed, several hub circle parameters can be obtained.

[0080] Exemplarily, assume that the radius threshold [R1, R n includes radii r1, r2, and r3. Then, substitute the coordinates of each pixel point and the values of r1, r2, and r3 into the polar coordinate conversion formula respectively to obtain the polar coordinates (a, b) corresponding to each pixel point and each set radius R, and obtain several hub circle parameters (a, b, R i ).

[0081] Preferably, the polar coordinate conversion formula can be optionally expressed as Equation 2-1:

[0082] {a = x - R i cosθ; b = y - R i sinθ} 2-1

[0083] Where, (a, b) are the polar coordinates of the center of the hub, (x, y) are the pixel point coordinates, R i is the i-th set radius in the radius threshold, and θ i is the gradient direction of the i-th pixel point in the edge gradient map, 0 < i ≤ N, and N is the number of pixel points in each gradient map.

[0084] S23: Map each hub circle parameter to the three-dimensional histogram statistics to obtain the set number of hub circle parameters with the most votes, which are the initial hub circle parameters;

[0085] Specifically, since in the actual operation process, there may be influencing factors such as environmental noise, which may cause errors such as false circles when obtaining the hub circle parameters, it is also necessary to detect the hub circle parameters obtained in step S22 to avoid errors. Therefore, it is optional but not limited to that whenever the corresponding polar coordinates (a, b) are obtained according to the coordinates of each pixel point and the currently traversed radius R in step S22, the number of votes for the corresponding hub circle parameter (a, b, R) in the three-dimensional histogram statistical table is increased by one. If the hub circle parameter is not yet included in the three-dimensional histogram statistical table, the hub circle parameter is added to the three-dimensional histogram statistical table, and then the corresponding number of votes for the hub circle parameter is increased by one. Then, according to the number of votes of each hub circle parameter in the three-dimensional histogram statistical table, a set number of hub circle parameters with the largest number of votes are obtained, which are the initial hub circle parameters to be detected.

[0086] S24: obtaining original wheel hub image features of each a priori region of interest in the original wheel hub image; the image features include parameters such as cosine approximation and gradient variance;

[0087] Specifically, it is optional but not limited to graying the original wheel hub image to obtain a corresponding grayscale image, and counting the grayscale values ​​of each pixel in each prior region of interest in the grayscale image to obtain the number of each grayscale value, so as to establish a grayscale histogram corresponding to each prior region of interest, and calculate the cosine approximation and gradient variance and other parameters of the corresponding prior region of interest according to each grayscale histogram; the prior region of interest is arbitrarily set by those skilled in the art. Preferably, the prior region of interest is as follows Figure 2 Shown in the rectangular area.

[0088] Further preferably, the specific steps of obtaining the image features of each a priori region of interest in the original wheel hub image may optionally include but are not limited to:

[0089] S241: Obtaining a grayscale image corresponding to the original wheel hub image;

[0090] Specifically, it is optional but not limited to obtaining the red channel value, green channel value and blue channel value of each pixel in the original wheel image, adding them together, and obtaining the gray value corresponding to each pixel according to gray value calculation formula 2-2, so as to obtain the gray image corresponding to the original wheel image:

[0091] R*0.299+G*0.587+B*0.114=g 2-2

[0092] Among them, g is the grayscale value, R is the red channel value, G is the green channel value, and B is the blue channel value.

[0093] S242: Counting the number of grayscale values ​​of each pixel in each priori region of interest in the grayscale image, and establishing a grayscale histogram of each priori region of interest;

[0094] Specifically, it is optional but not limited to obtaining the grayscale value of each pixel in the a priori region of interest in the original wheel hub image, and counting the frequency or number of occurrences of each grayscale value in each a priori region of interest to establish a grayscale histogram of each a priori region of interest.

[0095] S243: normalizing the grayscale histogram and calculating the cosine approximation of each a priori region of interest;

[0096] Specifically, it is optional but not limited to normalizing the grayscale histogram according to formula 2-3 so that the sum of the probabilities of all gray levels is 1:

[0097] P(gi)=H(gi) / total number of pixels 2-3

[0098] Where P(gi) is the frequency of occurrence of the i-th grayscale value, H(gi) is the number of the i-th grayscale value, and gi is the i-th grayscale value.

[0099] Then, the cosine approximation of each prior region of interest is calculated according to formula 2-4:

[0100]

[0101] Among them, u1 is the cosine approximation, L is the number of gray levels, g is the gray value, P(gi) is the frequency of occurrence of the i-th gray value, and gi is the i-th gray value.

[0102] S244: Obtaining the gradient variance of each a priori region of interest according to the gradient value of each pixel point.

[0103] Specifically, it is optional but not limited to obtaining the average gradient value of all pixels in each priori region of interest based on the gradient value of each pixel in the original hub image obtained in step S1, and then calculating the average of the sum of squares of the differences between the gradient value of each pixel and the average gradient value to obtain the gradient variance of each priori region of interest.

[0104] For example, the gradient average of all pixels in each prior region of interest may be calculated according to formula 2-5:

[0105]

[0106] Among them, u g is the average gradient of all pixels in each prior region of interest, N is the total number of pixels in each prior region of interest, gj is the gradient of the jth pixel, 0 <j≤N。

[0107] Then, according to the gradient variance calculation formula 2-6, the gradient variance of each prior region of interest can be obtained:

[0108]

[0109] in, is the gradient variance of each prior region of interest, N is the total number of pixels in each prior region of interest, gj is the gradient of the jth pixel, 0 <j≤N,u g is the average gradient of all pixels in each prior region of interest.

[0110] S25: Obtain the hub image features corresponding to each hub circle parameter in each prior region of interest of the edge gradient image, and determine whether the deviation between the hub image features and the original hub image features is less than a deviation threshold; if not, remove the corresponding circle parameter; if yes, determine the hub circle parameter.

[0111] Specifically, the hub image features corresponding to each hub circle parameter in each priori region of interest are obtained according to steps S241-S244, and the difference between the cosine approximation and gradient variance in the hub image features corresponding to each hub circle parameter and the corresponding parameters of the original hub image is obtained to determine whether it is less than a threshold value. If not, it means that the hub circle parameter error is too large and needs to be eliminated. If so, it means that the hub circle parameter is the final hub circle parameter (a hub ,b hub ,R hub ).

[0112] It is worth noting that step S2 is only a reference example for obtaining the hub circle parameters based on the edge detection algorithm, but is not limited to this. The method for obtaining the hub circle parameters may also include common shape detection algorithms such as random sampling consistency algorithm, least squares fitting method, template matching method, and region growing algorithm.

[0113] S3: creating a blank image of the same size as the original wheel hub image, obtaining wheel hub foreground parameters according to the wheel hub circle parameters, cropping the blank image according to the wheel hub circle parameters to obtain a foreground mask image, and obtaining a first contour target image according to the wheel hub foreground parameters and the foreground mask image;

[0114] Specifically, it is optional but not limited to creating a blank image of the same size as the original hub image, then cropping the blank image according to the hub circle parameters obtained in step S2 to obtain a foreground mask image of the same size as the hub circle parameters, then calculating the hub foreground parameters based on the hub circle parameters, and cropping the blank image according to the hub foreground parameters to obtain a first hub target image.

[0115] Preferably, step S3 specifically includes:

[0116] S31: creating a blank image of the same size as the original wheel hub image, and cropping the blank image according to the wheel hub circle parameter to obtain a foreground mask image;

[0117] Specifically, it is optional but not limited to generate a blank image with the same size as the original wheel image, and set the pixel value of each pixel in the blank image to the first pixel value, and then change the pixel value of the wheel cover area to the second pixel value through the image processing algorithm according to the wheel circle parameter obtained in step S2 and cut it, so as to obtain the foreground mask image I msk The image processing algorithm may be selected from, but not limited to, common algorithms such as watershed segmentation algorithm, flood fill algorithm, region-based segmentation algorithm, edge-based segmentation algorithm, graph-based segmentation algorithm, clustering-based segmentation algorithm, etc. Preferably, the image processing algorithm uses a flood fill algorithm.

[0118] Preferably, the first pixel value is 0 and the second pixel value is 255.

[0119] S32: obtaining a wheel hub foreground image according to the foreground mask image, and obtaining wheel hub foreground parameters according to the wheel hub circle parameters, so as to crop the wheel hub foreground image according to the wheel hub foreground parameters to obtain a first wheel hub target image.

[0120] Specifically, it is optional but not limited to substituting the foreground mask image into equation 3-1 to calculate the wheel foreground image:

[0121] I dst =I src ×(I msk / 255) 3-1

[0122] Specifically, I dst is the wheel foreground image, I src is the original wheel image, I msk is the foreground mask image. Then substitute the wheel circle parameters into equations 3-2, 3-3, 3-4 and 3-5 to calculate the wheel foreground parameters:

[0123] x hub =a hub -R hub 3-2

[0124] y hub =b hub -R hub 3-3

[0125] w hub =2×R hub 3-4

[0126] h hub =2×R hub 3-5

[0127] Among them, (a hub ,b hub ,R hub ) is the hub circle parameter, (x hub,y hub ) is the center point of the wheel image, (w hub ,h hub ) is the width and height of the wheel image. The combination of the wheel image center point, width and height is the wheel foreground parameter Rect hub (x hub ,y hub ,w hub ,h hub ), and the wheel hub foreground image is cut according to the wheel hub foreground parameter to obtain the first wheel hub target image I hub .

[0128] S4: performing feature enhancement on the first hub target image to obtain a three-channel target image;

[0129] Specifically, it is optional but not limited to enlarging the first hub target image according to a set ratio to obtain a plurality of hub target images of different sizes, and cropping each image to the same size and merging them to obtain a three-channel target image.

[0130] Preferably, the step of performing feature enhancement on the first hub target image to obtain a three-channel target image may optionally include but is not limited to:

[0131] S41: creating a blank image twice the size of the original wheel hub image, and drawing a circle in the blank image according to the wheel hub circle parameter and the first cropping ratio to obtain a wheel hub foreground mask image;

[0132] Specifically, it is optional but not limited to creating a blank image twice the size of the original wheel image and setting the first cropping ratio Ratio cut , the wheel circle parameters and the first cutting ratio get the cutting radius R msksub =R×Ratio cut , thus obtaining the clipping parameters (a hub ,b hub ,R msksub ), and draw a circle on the blank image according to the cropping parameters to obtain the hub foreground mask image; the first cropping ratio is arbitrarily set by those skilled in the art. Preferably, the first cropping ratio Ratio cut =2 / 3.

[0133] S42: obtaining a wheel hub foreground image according to the wheel hub foreground mask image, and obtaining a wheel hub foreground parameter according to the wheel hub circle parameter and the cropping ratio, so as to crop the wheel hub foreground image according to the wheel hub foreground parameter to obtain a second wheel hub target image;

[0134] Specifically, it is optional but not limited to substituting the wheel foreground mask image into equation 4-1 to calculate the wheel foreground image:

[0135] I dstsub =Ihub ×(I msksub / 255) 4-1

[0136] Among them, I dstsub is the wheel foreground image, I hub is the wheel target image, I msksub A blank image that is twice the size of the original wheel image.

[0137] Then substitute the wheel circle parameters into equations 4-2, 4-3, 4-4 and 4-5 to calculate the wheel foreground parameters:

[0138] x hubsub =R hub ×(1-Ratio cut ) 4-2

[0139] y hubsub =R hub ×(1-Ratio cut ) 4-3

[0140] w hubsub =2R hub ×Ratio cut 4-4

[0141] h hubsub =2R hub ×Ratio cut 4-5

[0142] Among them, R hub is the radius of the hub circle parameter, Ratio cut is the cropping ratio, (w hubsub ,h hubsub ) is the width and height of the wheel image. The combination of the wheel image center point, width and height is the wheel foreground parameter Rect hubsub (x hubsub ,y hubsub ,w hubsub ,h hubsub ), and the wheel hub foreground image is cut according to the wheel hub foreground parameter to obtain the second wheel hub target image I 1dstsub .

[0143] S43: obtaining a wheel hub foreground parameter according to the wheel hub circle parameter and the second cropping ratio, and cropping the wheel hub foreground image according to the wheel hub foreground parameter to obtain a third wheel hub target image;

[0144] Specifically, it is optional but not limited to setting the second cropping ratio, and obtaining the third wheel hub target image I according to steps S41-S42. 2dstsub The first cutting ratio is arbitrarily set by those skilled in the art. Preferably, the first cutting ratio is 1 / 3.

[0145] S44: normalizing the first hub target image, the second hub target image, and the third hub target image, and splicing them into a three-channel image.

[0146] Specifically, it is optional but not limited to normalizing the first hub target image, the second hub target image and the third hub target image obtained in steps S3-S43 so that the size of each hub target image is the same, and then creating an empty three-channel color image whose size is the same as the size of each hub target image, and then placing the first hub target image, the second hub target image and the third hub target image on the red channel, green channel and blue channel of the color image respectively to obtain the final required three-channel image.

[0147] S5: determining a rotation sampling period according to the spokes of the hub, and obtaining a plurality of sampling images by rotating the three-channel image;

[0148] Specifically, it is optional but not limited to determine the sampling period of the three-channel image obtained in step 4 according to the spokes of the hub, so as to rotate the three-channel image several times to obtain several sampled images, that is, three-channel images at different angles. Preferably, the sampling period of the three-channel image is arbitrarily set by those skilled in the art.

[0149] For example, Figure 3 As shown in the figure, when the number of spokes of the wheel is 5, its rotation period T = 360 / 5 = 72°, then set the number of image rotations to determine the angle of each rotation of the three-channel image according to the rotation period. Optionally, the rotation period T = 72° can be divided into 4 equal parts, then the rotation angle each time is θ = 72° / 4 = 18°, and each rotation is 18°. The result is as follows Figure 4 As shown, the image names are {I hub ,I hubrot1 ,I hubrot2 ,I hubrot3}

[0150] S6: construct and train a neural network model that takes the sampled images of each original wheel image as input and the wheel defect category corresponding to the original wheel image as output;

[0151] Specifically, the neural network model can use any image recognition model in the prior art to construct an initial neural network model; then obtain a number of original wheel hub images, and then mark the wheel hub defect categories contained in the original wheel hub images, and obtain a number of sampled images corresponding to each wheel hub original image according to steps S1-S5, and construct a training data set with a number of sampled images corresponding to each wheel hub original image and the corresponding wheel hub defect categories: then input the initial neural network model according to the training data set to train the various parameters of the initial neural network model to obtain the trained neural network model. It is worth noting that this step is a preparatory work, which can be constructed and trained in advance, or it can be carried out simultaneously with steps S1-S5, as long as it is constructed and trained before the neural network model is needed for prediction in step S6.

[0152] Preferably, the specific structure of the neural network model is shown in Table 1, and may include but is not limited to:

[0153] Input layer: used to receive the input sampled image data and then pass the sampled image data to the first convolutional layer;

[0154] The first convolution layer: connected to the input layer, performs convolution operation on the input image, and uses 6 5x5 convolution kernels to extract local features of the image. Each convolution kernel will generate a feature map, and 6 initial feature maps are obtained;

[0155] The first maximum pooling layer: connected to the first convolutional layer, performs the maximum pooling operation on each initial feature map, reduces the spatial size of each initial feature map, extracts the main features to reduce the amount of calculation, and obtains 6 optimized feature maps;

[0156] The second convolutional layer is connected to the first maximum pooling layer to further extract features from each optimized feature map. It uses 32 5x5 convolution kernels, takes the 6 optimized feature maps of the previous layer as input, and outputs 32 updated feature maps.

[0157] Second maximum pooling layer: connected to the second convolutional layer, performs pooling operation on each updated feature map to further reduce the size of the updated feature map to obtain the pooled feature map;

[0158] The third convolutional layer: connected to the second maximum pooling layer, uses 8 5x5 convolution kernels to further extract the high-level features of the pooling feature map and obtain 8 convolution feature maps;

[0159] The third maximum pooling layer: connected to the third convolutional layer, pools each convolutional feature map to further reduce the size of the feature map and obtain the final feature map;

[0160] The first fully connected layer: connected to the third maximum pooling layer, converts all feature maps into vectors of several nodes, and then introduces nonlinearity through the ReLU activation function;

[0161] The second fully connected layer is connected to the first fully connected layer, further compresses the vectors of several nodes output by the first fully connected layer, and divides them into N defect categories. The more vectors each defect category contains, the higher the corresponding category probability.

[0162] Output layer: connected to the second fully connected layer, the output of the second fully connected layer is converted into a probability distribution using the softmax function to represent the predicted probability of each category, and the defect category with the highest probability is output, which is the final determined wheel defect category.

[0163] It is worth noting that the number of convolution kernels used in each convolution layer is proportional to the number of input images, and the number of vectors output by each fully connected layer is arbitrarily set by those skilled in the art.

[0164] Table 1: Schematic diagram of the specific structure of the neural network model

[0165] Input Layer First convolutional layer (6, 16, 5×5) First max pooling layer (16,16,2×2,2) The second convolutional layer (16, 32, 5×5) Second maximum pooling layer (32,32,2×2,2) The third convolutional layer (32, 8, 5×5) The third maximum pooling layer (8,8,2×2,2) First fully connected layer (1024,512) Second fully connected layer (512, N) Output layer: softmax layer

[0166] The specific structure of the neural network model provided in this embodiment uses multiple downsampling, multiple upsampling combinations, and multiple cross-fusion features to enhance the correlation between local and global features. By enhancing the correlation of radial features of the wheel hub image, the efficiency of recognizing circular targets such as wheels can be improved while reducing the complexity of the recognition network and without reducing the recognition rate.

[0167] Preferably, the neural network model may optionally but not limited to adopt a resnet deep learning network to identify the wheel hub target image and output the defect category of the wheel hub in the image.

[0168] More preferably, the process of constructing and training the neural network model may optionally include but is not limited to:

[0169] S61: construct an initial neural network model; preferably use a transformer deep learning network;

[0170] S62: Acquire a number of original wheel hub images of equal size and their corresponding defect categories, and obtain sample images corresponding to each original wheel hub image according to steps S1-S5, with each sample image and corresponding defect category as a subset;

[0171] Specifically, a technician in this field can choose to identify the automobile wheel hub defect category contained in each sampled image. The optional automobile wheel hub defect category includes surface defects, pore defects, surface shrinkage, under-casting defects, cold shut defects and die-casting defects, etc.; and construct a training data set by annotating each sampled image and the corresponding defect category.

[0172] S63: Input the training data set into the initial neural network model to optimize various parameters of the initial neural network model to obtain the final neural network model.

[0173] S7: Input the sampled images into the trained neural network model to obtain the defect category corresponding to each sampled image.

[0174] Specifically, it is optional but not limited to inputting the several sample images obtained in step S5 into the trained neural network model obtained in step S6 to identify the defect category of the automobile wheel hub in each sample image.

[0175] In this embodiment, a defect category recognition method for a wheel hub image of the present invention is provided. By performing edge detection on the original wheel hub image, edge points in the original wheel hub image are obtained, an edge gradient map is constructed, and the gradient direction of each pixel point in the edge gradient map is obtained. Then, the wheel hub circle parameter is obtained according to the coordinates and gradient direction of each pixel point in the edge gradient map. Then, a blank image equal to the original wheel hub image is created, and the wheel hub foreground parameter is obtained according to the wheel hub circle parameter. The blank image is cropped according to the wheel hub circle parameter to obtain a foreground mask image, and a first contour target image is obtained according to the wheel hub foreground parameter and the foreground mask image. Feature enhancement is performed on the first wheel hub target image to obtain a three-channel target image. The characteristic correlation between pixels near and far from the dot in the radial direction of the original hub image is enhanced by splicing and fusion. Then, the rotation sampling period is determined according to the spokes of the hub. Several sampling images are obtained by rotating the three-channel image. The characteristics of the periodic change of the characteristics brought by the rotation change are associated by rotating around the center of the hub, which is conducive to extracting the hub features. Then, a neural network model is constructed and trained with the sampling images of each original hub image as input and the hub defect category corresponding to the original hub image as output. Finally, the sampling images of each original hub image are input into the trained neural network model to obtain the defect category corresponding to each original hub image. The problem of high recognition complexity and low recognition rate in the existing technology is solved.

[0176] On the other hand, the present invention further provides a computer storage medium storing executable program code; the executable program code is used to execute any of the above-mentioned methods for identifying defect categories of wheel hub images.

[0177] On the other hand, the present invention further provides a terminal device, comprising a memory and a processor; the memory stores program codes executable by the processor; the program codes are used to execute any of the above-mentioned methods for identifying defect categories of wheel hub images.

[0178] Exemplarily, the program code may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the program code in the terminal device.

[0179] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the terminal device may also include an input / output device, a network access device, a bus, etc.

[0180] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0181] The memory may be an internal storage unit of the terminal device, such as a hard disk or a memory. The memory may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device. Further, the memory may also include both an internal storage unit of the terminal device and an external storage device. The memory is used to store the program code and other programs and data required by the terminal device. The memory may also be used to temporarily store data that has been output or is to be output.

[0182] The above-mentioned computer storage medium and terminal device are created based on the above-mentioned wheel hub image defect category identification method, and their technical functions and beneficial effects are no longer repeated here. The various technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the various technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0183] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for identifying defect categories of wheel hub images, characterized in that: include: Perform edge detection on the original wheel hub image to obtain edge points in the original wheel hub image, construct an edge gradient map, and obtain the gradient direction of each pixel point in the edge gradient map; The hub circle parameters are obtained according to the coordinates and gradient direction of each pixel in the edge gradient map; Creating a blank image of the same size as the original wheel hub image, obtaining wheel hub foreground parameters according to the wheel hub circle parameters, cropping the blank image according to the wheel hub circle parameters to obtain a foreground mask image, and obtaining a first contour target image according to the wheel hub foreground parameters and the foreground mask image; Perform feature enhancement on the first hub target image to obtain a three-channel target image; Determine the rotation sampling period according to the spokes of the hub, and obtain a number of sampling images by rotating the three-channel image; Construct and train a neural network model that takes the sampled images of each original wheel image as input and the wheel defect category corresponding to the original wheel image as output; The sampled images of each original wheel hub image are input into the trained neural network model to obtain the defect category corresponding to each original wheel hub image.

2. The method according to claim 1, characterized in that The steps of performing edge detection on the original wheel hub image, obtaining edge points in the original wheel hub image, constructing an edge gradient map, and obtaining the gradient direction of each pixel point in the edge gradient map include: Obtain the gradient value of each pixel in the original wheel hub image. If it is greater than the maximum value of the gradient threshold range, it is determined to be an edge point; if it is less than the minimum value of the gradient threshold range, it is removed; if it is between the minimum and maximum values ​​of the gradient threshold range, determine whether the pixel is adjacent to the edge point. If it is adjacent, it is determined to be an edge point; if it is not adjacent, it is removed to obtain an edge gradient map; Get the gradient of each pixel in the edge gradient map in the X and Y directions to get the gradient direction of each pixel.

3. The method according to claim 1, characterized in that The step of obtaining the hub circle parameters according to the coordinates and gradient direction of each pixel point in the edge gradient image includes: Create a blank 3D histogram statistics table; Set and traverse the radius threshold to obtain the corresponding polar coordinates according to the coordinates of each pixel point and the currently traversed radius, and construct the hub circle parameters with the polar coordinates and the corresponding radius; Map each hub circle parameter to a three-dimensional histogram statistics, and obtain a set number of hub circle parameters with the largest number of votes, which are the initial hub circle parameters; Obtaining original wheel hub image features of each a priori region of interest in the original wheel hub image; the image features include cosine approximation and gradient variance; The hub image features corresponding to each hub circle parameter in each prior region of interest of the edge gradient image are obtained, and it is determined whether the deviation between the hub image features and the original hub image features is less than the deviation threshold. If not, the corresponding circle parameters are eliminated. If yes, the hub circle parameters are determined.

4. The method according to claim 1, characterized in that: The specific steps of obtaining the image features of each prior region of interest in the original wheel hub image include: Get the grayscale image corresponding to the original wheel hub image; Counting the number of grayscale values ​​of each pixel in each prior region of interest in the grayscale image, and establishing a grayscale histogram of each prior region of interest; Normalize the grayscale histogram and calculate the cosine approximation of each a priori region of interest; According to the gradient value of each pixel point, the gradient variance of each prior region of interest is obtained.

5. The method according to claim 1, characterized in that The step of performing feature enhancement on the first hub target image to obtain a three-channel target image includes: Create a blank image twice the size of the original wheel image, draw a circle in the blank image according to the wheel circle parameters and the first cropping ratio, and obtain the wheel foreground mask image; A wheel hub foreground image is obtained according to the wheel hub foreground mask image, and a wheel hub foreground parameter is obtained according to the wheel hub circle parameter and the cropping ratio, so as to crop the wheel hub foreground image according to the wheel hub foreground parameter to obtain a second wheel hub target image; Obtaining a hub foreground parameter according to the hub circle parameter and the second cropping ratio, and cropping the hub foreground image according to the hub foreground parameter to obtain a third hub target image; The first hub target image, the second hub target image, and the third hub target image are normalized and concatenated into a three-channel image.

6. The method according to claim 1, characterized in that The specific structure of the neural network model includes: Input layer: used to receive the input sampled image data and then pass the sampled image data to the first convolutional layer; The first convolutional layer: connected to the input layer, performs convolution operation on the input image, uses 6 5x5 convolution kernels to extract local features of the image, and obtains 6 initial feature maps; The first maximum pooling layer: connected to the first convolutional layer, performs the maximum pooling operation on each initial feature map, reduces the spatial size of each initial feature map, extracts the main features to reduce the amount of calculation, and obtains 6 optimized feature maps; The second convolutional layer is connected to the first maximum pooling layer to further extract features from each optimized feature map. It uses 32 5x5 convolution kernels, takes the 6 optimized feature maps of the previous layer as input, and outputs 32 updated feature maps. Second maximum pooling layer: connected to the second convolutional layer, performs pooling operation on each updated feature map to further reduce the size of the updated feature map to obtain the pooled feature map; The third convolutional layer: connected to the second maximum pooling layer, uses 8 5x5 convolution kernels to further extract the high-level features of the pooling feature map and obtain 8 convolution feature maps; The third maximum pooling layer: connected to the third convolutional layer, pools each convolutional feature map to further reduce the size of the feature map and obtain the final feature map; The first fully connected layer: connected to the third maximum pooling layer, converts all feature maps into vectors of several nodes, and then introduces nonlinearity through the ReLU activation function; The second fully connected layer is connected to the first fully connected layer, further compresses the vectors of several nodes output by the first fully connected layer, and divides them into N defect categories. The more vectors each defect category contains, the higher the corresponding category probability. Output layer: connected to the second fully connected layer, the output of the second fully connected layer is converted into a probability distribution using the softmax function to represent the predicted probability of each category, and the defect category with the highest probability is output, which is the final determined wheel defect category.

7. The method according to claim 1, characterized in that The construction and training process of the neural network model includes: Build an initial neural network model; Obtain a number of original wheel hub images of equal size and their corresponding defect categories, and obtain the sampling images corresponding to each original wheel hub image according to the above steps, with each sampling image and the corresponding defect category as a subset; The training data set is input into the initial neural network model to optimize the parameters of the initial neural network model and obtain the final neural network model.

8. The method according to claim 7, characterized in that Defect categories include: Any one or more of surface defects, porosity defects, surface shrinkage, undercasting defects, cold shut defects and die casting defects.

9. A computer storage medium, characterized in that: An executable program code is stored; the executable program code is used to execute the defect category recognition method of the wheel hub image described in any one of claims 1-8.

10. A terminal device, characterized in that: It comprises a memory and a processor; the memory stores a program code executable by the processor; the program code is used to execute the defect category identification method of the wheel hub image as described in any one of claims 1-8.

Images