Truck wheel state identification method and system

Abstract

The present application belongs to the technical field of state recognition, and particularly relates to a truck wheel state recognition method and system, comprising the following steps: S1, obtaining a truck wheel image to be recognized, and extracting multiple resolution level feature maps through a backbone network; separating each level feature map into a low-frequency coefficient matrix and a high-frequency coefficient matrix through discrete cosine transformation, performing multi-scale low-frequency feature fusion on the low-frequency coefficient matrices of different levels to obtain a cross-layer low-frequency correlation graph, and performing operation on the high-frequency coefficient matrices of the same level to obtain an intra-layer high-frequency structure graph; S2, determining a candidate region on the level feature map, detecting corner points in the candidate region, and calculating the covariance matrix of the corner point position. The present application strengthens the feature response of the key region, suppresses the interference of the background and irrelevant region, combines frequency domain analysis with spatial geometric structure prior, improves the quality of the final fused features, and thus improves the accuracy of truck wheel state recognition.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of state recognition, and particularly relates to a truck wheel state recognition method and system. BACKGROUND

[0002] The recognition technology of the wheel state is mainly divided into a traditional image processing method and a method based on deep learning. The traditional method usually relies on a hand-designed feature extraction algorithm. However, because the truck operation environment is complex, the wheel image is often affected by various factors, resulting in that the generalization ability of the hand-designed feature is seriously insufficient. With the development of artificial intelligence technology, the convolutional neural network can learn the hierarchical features of the image from a large amount of data by constructing a deep network model.

[0003] However, in the aspect of multi-scale feature fusion, the existing model usually adopts a simple feature map addition or splicing operation, which is easy to cause the high-level abstract semantic information and the low-level texture detail information to interfere with or dilute each other. The general model lacks explicit modeling ability for the key geometric structure inside the target, and the attention mechanism is usually data-driven and cannot necessarily focus on the structure area most critical to the classification task, resulting in that when facing subtle state differences or local features that are not obvious, misjudgment or omission is easy to occur. Therefore, how to design a recognition method capable of fusing multi-scale features and utilizing the key geometric structure information of the target is the key to improving the recognition accuracy of the truck wheel state. SUMMARY

[0004] The application provides a truck wheel state recognition method and system to solve the technical problems that the existing method ignores the differences in the frequency domain characteristics of different level feature maps and is easy to misjudge or miss.

[0005] In a first aspect, the application provides a truck wheel state recognition method, including the following steps:

[0006] S1, a truck wheel image to be recognized is acquired, and a plurality of resolution level feature maps are extracted through a backbone network; each level feature map is separated into a low-frequency coefficient matrix and a high-frequency coefficient matrix through discrete cosine transformation, multi-scale low-frequency feature fusion is performed on the low-frequency coefficient matrices of different levels to obtain a cross-layer low-frequency correlation graph, and the high-frequency coefficient matrices of the same level are operated to obtain an intra-layer high-frequency structure graph;

[0007] S2, a candidate region is determined on the level feature map, corner points in the candidate region are detected, a covariance matrix of the corner point position is calculated, and the maximum eigenvalue of the covariance matrix is taken as a corner point spatial distribution dispersion parameter; the corner points constituting a local maximum convex polygon are screened out to form a main corner point set;

[0008] S3, the spatial distribution of the corner point is processed by the inverse tangent function and normalized to obtain the fusion weight coefficient; based on the fusion weight coefficient, the cross-layer low-frequency correlation graph and the intra-layer high-frequency structure graph are weighted and summed to obtain the preliminary fusion feature;

[0009] S4, the spatial modulation matrix is constructed with the centroid of the main corner point set as the Gaussian kernel center, and the spatial modulation matrix is multiplied with the preliminary fusion feature element by element to obtain the final fusion feature; according to the final fusion feature, the state of the truck wheel is determined by the classifier.

[0010] Further, the hierarchical feature maps are separated into low-frequency coefficient matrices and high-frequency coefficient matrices by discrete cosine transform, including:

[0011] For the input hierarchical feature map with a size of n*n, the hierarchical feature map is divided into multiple 8*8 sub-blocks, and a two-dimensional discrete cosine transform formula is applied to each sub-block for calculation. The coefficient value at the top left corner is taken as the direct current component to form the low-frequency coefficient matrix, and the remaining 63 coefficient values are taken as the alternating current component to form the high-frequency coefficient matrix.

[0012] Further, the low-frequency coefficient matrices of different levels are fused to obtain the cross-layer low-frequency correlation graph, including:

[0013] The low-frequency coefficient matrices L3, L4, L5 of the three levels A3, A4, A5 output by the backbone network are selected. The low-resolution low-frequency coefficient matrices L4, L5 are upsampled to make the spatial resolution of the low-frequency coefficient matrix consistent with that of the high-resolution low-frequency coefficient matrix L3, and the three low-frequency coefficient matrices with consistent size are added element by element to generate the cross-layer low-frequency correlation graph.

[0014] Further, the high-frequency coefficient matrices of the same level are operated to obtain the intra-layer high-frequency structure graph, including:

[0015] Autocorrelation operation is performed on the high-frequency coefficient matrices H3, H4, H5 of the three levels A3, A4, A5 to obtain three independent intra-layer high-frequency structure graphs. The low-resolution intra-layer high-frequency structure graphs are upsampled to make the spatial resolution consistent with that of the highest-resolution intra-layer high-frequency structure graph, and the three intra-layer high-frequency structure graphs with consistent size are added element by element to fuse into a single intra-layer high-frequency structure graph. Figure One

[0016] Further, the corner points in the candidate region are detected, and the covariance matrix of the corner point position is calculated, including:

[0017] The Harris corner detection algorithm is used to set the response threshold and identify all corner points in the candidate region. The two-dimensional coordinates of the detected N corner points are represented as a set , and the mean values of the x and y coordinates are calculated​​ and And calculate according to the following formula covariance matrix :

[0018]

[0019] in,

[0020]

[0021]

[0022]

[0023] Where, x i Let y be the x-coordinate value of the i-th corner point in the two-dimensional coordinate system. i Let be the ordinate value in the two-dimensional coordinates of the i-th corner point. This represents the mean of the x-coordinates in the two-dimensional coordinate system of the set. This represents the mean of the y-coordinates in the two-dimensional coordinate system of the set. , , , These represent the elements in the covariance matrix.

[0024] Furthermore, the corner points that form the local maximal convex polygon are selected to form the principal point set, including:

[0025] The Graham scan algorithm is applied to filter the corner points. The corner point with the smallest y-coordinate value is selected as the starting point. The polar angles of the remaining corner points relative to the starting point are calculated and sorted in ascending order. The sorted corner points are traversed sequentially using a stack structure. All corner points that form concave angles are identified and removed. The corner points that form the convex hull are retained as the set of main points.

[0026] Furthermore, the arctangent function is applied to the spatial distribution dispersion parameter of the corner points and then normalized to obtain the fusion weight coefficients, including:

[0027] Let the spatial distribution dispersion parameter of the corner points be . The fusion weighting coefficient w is calculated using the following formula:

[0028]

[0029] Where arctan() represents the arctangent function, the formula will Mapped to the interval [0, 1).

[0030] Furthermore, a spatial modulation matrix is ​​constructed with the centroid of the set of protagonist points as the center of the Gaussian kernel, including:

[0031] Calculate the average of the coordinates (x, y) of all N corner points in the main point set to obtain the centroid coordinates. Set the standard deviation of the Gaussian kernel. The distance is 0.5 times the average distance from all corner points of the protagonist's point set to the centroid; based on the centroid coordinates. and standard deviation A spatial modulation matrix with the same size as the initial fused feature is generated using a two-dimensional Gaussian function G(x,y).

[0032] .

[0033] Furthermore, based on the final fused features, the state of the truck wheels is determined by a classifier, including:

[0034] The final fused features are input into a classifier consisting of two fully connected layers and a Softmax activation function layer. The first fully connected layer reduces the feature vector dimension to 128, and the second fully connected layer outputs scores for three categories. The Softmax activation function layer converts the scores into probabilities, and the category with the highest probability is the identified wheel state, which is: normal, worn, or damaged.

[0035] Secondly, the present invention provides a truck wheel state recognition system, including a memory and a processor. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned truck wheel state recognition method is implemented.

[0036] The beneficial effects are as follows: This invention decomposes multi-level feature maps into low-frequency and high-frequency components using discrete cosine transform. By performing cross-layer tensor outer product operations on the low-frequency components and intra-layer operations on the high-frequency components, the global contextual relevance and local structural detail expressiveness of the features are enhanced, resulting in features with greater discriminative power than simple concatenation or addition. This invention utilizes the geometric set of key corner points extracted from the image to represent spatial distribution characteristics, and generates weights based on this to guide the fusion process of the aforementioned low-frequency and high-frequency features. By spatially modulating the fused features with the centroid of the key corner point set as the center, the feature response of key regions is strengthened, and interference from background and irrelevant regions is suppressed. Combining frequency domain analysis with prior spatial geometric structure improves the quality of the final fused features, thereby increasing the accuracy of truck wheel state recognition. Attached Figure Description

[0037] Figure 1 This is a flowchart of a method for recognizing the state of truck wheels. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] An embodiment of the truck wheel state recognition method provided by the present invention:

[0040] like Figure 1 As shown, the truck wheel status recognition method includes the following steps:

[0041] S1. Obtain the image of the truck wheel to be identified, and extract multi-resolution hierarchical feature maps through the backbone network; separate the feature maps of each level into low-frequency coefficient matrix and high-frequency coefficient matrix through discrete cosine transform, perform multi-scale low-frequency feature fusion on the low-frequency coefficient matrix of different levels to obtain cross-layer low-frequency correlation map, and perform operation on the high-frequency coefficient matrix of the same level to obtain intra-layer high-frequency structure map.

[0042] Specifically, a 224×224 pixel RGB image of a truck wheel is input into a convolutional neural network with ResNet-50 as the backbone. After forward propagation, feature maps output from the 2nd, 3rd, 4th, and 5th residual stages of the network are extracted as four layer feature maps of different resolutions. , , , .

[0043] For each level of feature map For each channel, a two-dimensional discrete cosine transform is performed to obtain the frequency domain coefficient matrix. The 8×8 region in the upper left corner of this frequency domain coefficient matrix is ​​taken as the low-frequency coefficient matrix. The remaining part serves as a high-frequency coefficient matrix. Select two low-frequency coefficient matrices from different levels, for example... and Multi-scale low-frequency feature fusion is performed to generate a higher-dimensional tensor as a cross-layer low-frequency correlation graph. For the same level, such as the fourth level, the high-frequency coefficient matrix... Calculate its product with its own transpose matrix, i.e. × The transpose of the result is used as the high-frequency structure diagram within the layer.

[0044] In an optional embodiment, the feature maps at each level are separated into low-frequency coefficient matrices and high-frequency coefficient matrices using discrete cosine transform, including:

[0045] For an input hierarchical feature map of size n×n, the hierarchical feature map is divided into multiple 8×8 sub-blocks. The two-dimensional discrete cosine transform formula is applied to each sub-block for calculation. The coefficient value in the upper left corner is used as the DC component to form a low-frequency coefficient matrix, and the remaining 63 coefficient values ​​are used as the AC component to form a high-frequency coefficient matrix.

[0046] Let's take a 256×256 pixel hierarchical feature map as an example. The complete hierarchical feature map is spatially divided into non-overlapping segments, forming a large grid composed of 32×32 sub-blocks, where each sub-block is an independent 8×8 pixel matrix. The global feature map is decomposed into a series of local regions, facilitating subsequent frequency domain analysis of each local region.

[0047] A two-dimensional discrete cosine transform is applied to each of the 1024 8×8 sub-blocks. This transform converts each sub-block from the spatial domain to the frequency domain, generating an 8×8 frequency coefficient matrix. In the frequency coefficient matrix, the first coefficient in the upper left corner, the DC component, represents the average pixel intensity within that 8×8 sub-block, reflecting the overall brightness and macroscopic information of that area. The remaining 63 coefficients in the frequency coefficient matrix are the AC components, which represent high-frequency information such as texture, edges, and details within the sub-block.

[0048] The 1024 DC components extracted from the 1024 sub-blocks are combined into a new 32×32 matrix according to their relative positions in the original image. This matrix is ​​the low-frequency coefficient matrix for this level. Simultaneously, the 63 AC components generated from each sub-block are aggregated to form the corresponding high-frequency coefficient matrix. This successfully decomposes the original feature map into a low-frequency component carrying macroscopic contour information and a high-frequency component carrying microscopic texture information.

[0049] In an optional embodiment, multi-scale low-frequency feature fusion is performed on low-frequency coefficient matrices at different levels to obtain a cross-level low-frequency correlation map, including:

[0050] Select the low-frequency coefficient matrices L3, L4, and L5 from the three levels A3, A4, and A5 output by the backbone network. Then, perform an upsampling operation on the low-resolution low-frequency coefficient matrices L4 and L5 to improve their spatial resolution. To achieve consistency, the three low-frequency coefficient matrices of the same size are added element by element to generate a cross-layer low-frequency correlation graph.

[0051] Specifically, assuming the backbone network outputs feature maps of three levels (A3, A4, and A5) at different stages from an input image, the corresponding low-frequency coefficient matrix is ​​obtained after discrete cosine transform. , and Due to the downsampling operation in the network, the spatial dimensions of the three matrices are different, for example... The dimensions are 64×64. Its dimensions are 32×32, and The dimensions are 16×16. Among them, It contains richer spatial details, and It contains more advanced semantic information.

[0052] To integrate three low-frequency coefficient matrices containing information at different scales, their sizes are unified, and the largest size is selected. As a baseline, 64×64. For a size of 32×32... Perform a 2x upsampling operation, for example using bilinear interpolation, to scale up to 64×64. Similarly, for a size of 16×16... A 4x upsampling operation is performed, also scaling the image to 64x64. After upsampling, three low-frequency matrices of identical size are obtained, which are the original... and after size adjustment and .

[0053] After unifying the dimensions of all low-frequency coefficient matrices, a fusion operation is performed. This operation involves element-wise addition, creating a new 64×64 blank matrix. The value of this blank matrix at any coordinate (i, j) is equal to the sum of the values ​​of the three matrices at the same coordinate (i, j). The resulting 64×64 matrix is ​​the cross-layer low-frequency correlation map, which integrates macroscopic feature information from shallow to deep layers and can represent the overall outline and structure of the target object.

[0054] In an optional embodiment, operations are performed on the high-frequency coefficient matrix of the same level to obtain an intra-layer high-frequency structure map, including:

[0055] Autocorrelation operations are performed on the high-frequency coefficient matrices H3, H4, and H5 of the three levels A3, A4, and A5, respectively, to obtain three independent intra-level high-frequency structure maps. The low-resolution intra-level high-frequency structure map is then upsampled to achieve a spatial resolution comparable to the highest-resolution intra-level high-frequency structure map. Figure One To achieve this, the three intra-layer high-frequency structure maps of the same size are added element by element and merged into a single intra-layer high-frequency structure map.

[0056] Specifically, high-frequency coefficient matrices from the three levels A3, A4, and A5 are processed separately. , and Two-dimensional autocorrelation is performed independently on each matrix. Autocorrelation can represent the similarity between the internal structure of a matrix and itself under different spatial displacements, thus enhancing periodic textures and repetitive structural features present in the matrix. For example, for... After performing autocorrelation calculations, a new matrix is ​​obtained. ,matrix This refers to the intra-layer high-frequency structure map at the A3 level, highlighting the texture patterns within the features of that level. Similarly, for... and Perform calculations to obtain the high-frequency structure diagrams within each layer. and .

[0057] Due to the original high-frequency coefficient matrix , , Different sizes result in different generated structural diagrams. , , The sizes also differ, for example It is 64×64. It is 32×32, It is 16×16. To fuse the structural information from the three levels, size alignment is required. Similar to the processing of the low-frequency coefficient matrix, it is done at the highest resolution. That is, 64×64 is the target size, and By doubling the upsampling, All were adjusted to a size of 64×64 by upsampling by 4x.

[0058] Three-layer high-frequency structure diagram after size alignment Upsampling and after upsampling Element-by-element summation is performed. This summation operation combines the high-frequency structural information detected at different levels to form a single, more informative intra-layer high-frequency structure map.

[0059] S2. Determine a candidate region on the hierarchical feature map, detect corner points within the candidate region, calculate the covariance matrix of the corner point positions, and use the largest eigenvalue of the covariance matrix as the spatial distribution dispersion parameter of the corner points; filter the corner points that form the local maximum convex polygon to form the main point set.

[0060] Specifically, in the hierarchical feature map The process involves identifying regions containing wheel bolt holes as candidate regions using either a preset threshold or a lightweight region proposal network. Within these regions, the FAST corner detection algorithm is applied with an intensity threshold of 20 to detect a series of corners. The two-dimensional coordinates of the corners are treated as a point set, and a 2×2 covariance matrix of this coordinate set is calculated. The two eigenvalues ​​of this covariance matrix are then solved, and the larger of the two is taken as the spatial distribution dispersion parameter of the corners. .

[0061] For all detected corner coordinates, the Graham scan algorithm is applied to calculate the 2D convex hull of the corners. All vertices constituting this convex hull are then selected, and the set of these vertices is the set of protagonist points.

[0062] In an optional embodiment, detecting corner points within the candidate region and calculating the covariance matrix of the corner point positions includes:

[0063] The Harris corner detection algorithm is used, and a response threshold is set to identify all corners within the candidate region; the two-dimensional coordinates of the N detected corners are represented as a set. Calculate the mean of the x and y coordinates. and And calculate according to the following formula covariance matrix :

[0064]

[0065] in,

[0066]

[0067]

[0068]

[0069] Where, x i Let y be the x-coordinate value of the i-th corner point in the two-dimensional coordinate system. i Let be the ordinate value in the two-dimensional coordinates of the i-th corner point. This represents the mean of the x-coordinates in the two-dimensional coordinate system of the set. This represents the mean of the y-coordinates in the two-dimensional coordinate system of the set. , , , These represent the elements in the covariance matrix.

[0070] Specifically, the Harris corner detection algorithm is applied to the candidate region feature map generated by fusing low-frequency and high-frequency information. This algorithm calculates a corner response value for each pixel in the map, representing the degree of gradient change around the pixel in various directions. A response threshold is set; any pixel whose corner response value is greater than the threshold is identified as a corner. By traversing the entire candidate region, all corners that meet the criteria are selected, and their two-dimensional coordinates are recorded.

[0071] Suppose five corner points are detected, with coordinates of point 1 (10, 25), point 2 (12, 30), point 3 (15, 22), point 4 (18, 28), and point 5 (20, 20). Calculate the mean of the coordinates of the five corner points. (The mean of the x-coordinate is also mentioned.) =15, mean of y-coordinate =25.

[0072] After obtaining the mean, the elements of the 2×2 covariance matrix can be calculated using the formula. It is the variance of the x-coordinate, calculated by comparing the x-coordinate of each point with the mean. The sum of the squares of the differences, divided by 4, gives the element. This is the variance of the y-coordinate, calculated similarly. (Element) and It is the covariance of the x and y coordinates, calculated by the x-coordinate of each point and the y-coordinate. The difference multiplied by the y-coordinate and The sum of the products of the differences is then divided by 4. The calculated... , , , The four values ​​are filled into the covariance matrix C, which represents the dispersion and correlation of all detected corner points in spatial distribution.

[0073] In an optional embodiment, corner points that form a local maximal convex polygon are selected to form a set of principal points, including:

[0074] The Graham scan algorithm is applied to filter the corner points. The corner point with the smallest y-coordinate value is selected as the starting point. The polar angles of the remaining corner points relative to the starting point are calculated and sorted in ascending order. The sorted corner points are traversed sequentially using a stack structure. All corner points that form concave angles are identified and removed. The corner points that form the convex hull are retained as the set of main points.

[0075] Specifically, assuming a set of corner points, the first step of the Graham scan algorithm is to find a reference point. It iterates through the coordinates of all corner points and finds the point with the smallest y-coordinate value. If multiple points have the same smallest y-coordinate value, the point with the smallest x-coordinate value is selected. This selected point will serve as the starting point and reference point for all subsequent calculations.

[0076] After selecting a reference point, calculate the polar angles of all other corner points relative to that reference point. The polar angle is the angle between the line connecting each point to the reference point and the positive horizontal direction. Based on the calculated polar angles, sort all non-reference point corner points in ascending order. If the polar angles are the same, sort them according to their distance from the reference point, with the closer ones appearing first. This results in an ordered sequence of corner points.

[0077] A stack data structure is used to construct the convex hull. The pivot point and the first sorted point are pushed onto the stack. Starting from the second sorted point, the stack is traversed sequentially. For each currently traversed point, its orientation relative to the top two points of the stack is considered. If the three points form a counter-clockwise or left-turning direction, the current point is a vertex of the convex hull, and is pushed onto the stack. If they form a clockwise or right-turning direction, the stack vertex is a concave point and does not belong to the convex hull, so it is popped from the stack. This popping operation is repeated until the left-turning condition is met, at which point the current point is pushed onto the stack. After traversing all points, the remaining points on the stack constitute the vertices of the largest convex polygon, forming the set of pivot points.

[0078] S3. The discreteness parameter of the corner point spatial distribution is processed by the arctangent function and normalized to obtain the fusion weight coefficient. Based on the fusion weight coefficient, the cross-layer low-frequency correlation graph and the intra-layer high-frequency structure graph are weighted and summed to obtain the preliminary fusion features.

[0079] Specifically, the discreteness parameter of the spatial distribution of corner points Substituting the arctangent function arctan(), we get arctan(). The arctangent value is mapped to the interval between 0 and 1 using a linear mapping function to obtain the fusion weight coefficient w.

[0080] By upsampling or pooling operations, the dimensions of the cross-layer low-frequency correlation graph and the intra-layer high-frequency structure graph are adjusted to be consistent. Using the fusion weight coefficient w, the formula is applied... Perform a weighted summation, where It is a cross-layer low-frequency correlation diagram. This is an intralayer high-frequency structure diagram, from which preliminary fusion features are obtained. .

[0081] In an optional embodiment, the spatial distribution dispersion parameter of the corner points is processed by the arctangent function and normalized to obtain the fusion weight coefficients, including:

[0082] Let the spatial distribution dispersion parameter of the corner points be . The fusion weighting coefficient w is calculated using the following formula:

[0083]

[0084] Where arctan() represents the arctangent function, the formula will Mapped to the interval [0, 1).

[0085] Specifically, the input is the discreteness parameter of the corner point spatial distribution. This parameter is derived from the previously calculated corner position covariance matrix, typically the largest eigenvalue of that matrix. It represents the degree of dispersion of the corner points along the most dominant directions of their spatial distribution. A large The value indicates that the corner points are widely and dispersed, while a small value indicates that the corner points are widely and dispersed. The value indicates that the corner points are compactly and centrally distributed. The value of this parameter is greater than or equal to zero, with no upper limit.

[0086] To transform this unlimited physical quantity into a fixed-range weighting coefficient that is easy to use in subsequent calculations, the arctangent function is applied. Assume that the result is obtained through calculation... The value is 1.732. Calculating the arctangent of this value, arctan(1.732), yields 60°. The arctangent function maps any non-negative input to a range of radians from 0 to π / 2.

[0087] After obtaining the radian value, a normalization operation is performed, which involves multiplying the radian value by a coefficient 2 / π. The purpose is to linearly map the range of radian values ​​from 0 to π / 2 to the interval between 0 and 1. Continuing with the example above, the result is approximately 0.667. The resulting value w is the fusion weight coefficient. Through this formula, regardless of the original... The value of the fusion weight system w is always constrained between 0 and 1, making it suitable for controlling the fusion ratio of two features.

[0088] S4. Construct a spatial modulation matrix with the centroid of the protagonist point set as the Gaussian kernel center, and multiply the spatial modulation matrix element-wise with the preliminary fusion features to obtain the final fusion features; based on the final fusion features, determine the state of the truck wheels through a classifier.

[0089] Specifically, the average coordinates of all corner points in the protagonist point set are calculated to obtain the centroid coordinates. A preliminary fusion feature is then constructed. A two-dimensional Gaussian kernel matrix with the same spatial dimension is used, with the center peak position set at the centroid coordinates and the standard deviation set to a predetermined value. This two-dimensional Gaussian kernel matrix is ​​the spatial modulation matrix. This spatial modulation matrix is ​​then combined with the preliminary fusion features. Element-by-element multiplication is performed to obtain the final fusion feature.

[0090] The resulting fused feature map is then subjected to global average pooling, flattening it into a one-dimensional feature vector. This one-dimensional feature vector is input into a classifier containing two fully connected layers and a ReLU activation function. The number of output nodes in the last fully connected layer is equal to the preset number of wheel state categories. The probability of each category is calculated using the Softmax function, and the category with the highest probability value is the identified truck wheel state.

[0091] In an optional embodiment, a spatial modulation matrix is ​​constructed with the centroid of the protagonist point set as the center of the Gaussian kernel, including:

[0092] Calculate the average of the coordinates (x, y) of all N corner points in the main point set to obtain the centroid coordinates. Set the standard deviation of the Gaussian kernel. The distance is 0.5 times the average distance from all corner points of the protagonist's point set to the centroid; based on the centroid coordinates. and standard deviation A spatial modulation matrix with the same size as the initial fused feature is generated using a two-dimensional Gaussian function G(x,y).

[0093] .

[0094] Specifically, assume the set of main points consists of four corner points with coordinates: point 1 (20, 20), point 2 (60, 20), point 3 (60, 60), and point 4 (20, 60). Calculate the geometric center of this set, i.e., the centroid. Add the x-coordinates of all the corner points together and divide by 4 to obtain the x-coordinate of the centroid. The value is 40. Similarly, the average value of the y-coordinates is calculated to obtain the centroid y-coordinate. The value is 40. Therefore, the center of the Gaussian kernel is determined at coordinates (40, 40).

[0095] Determine the standard deviation σ of the Gaussian kernel, which determines the breadth or sparseness of the Gaussian distribution, i.e., the size of its influence range. Calculate the Euclidean distance from each corner point in the set to the centroid (40, 40). For example, the distance from point one to the centroid is approximately 28.28. After calculating the distances from all four corner points to the centroid, calculate the average distance; in this example, the average distance is still 28.28. Multiply the average distance by 0.5 to obtain the value of the standard deviation σ, which is 14.14.

[0096] Based on the determined center coordinates =40, =40, and standard deviation σ=14.14, to generate a spatial modulation matrix. The size of this matrix is ​​exactly the same as the size of the initial fused feature map to be modulated, for example, 64×64. For each pixel (x, y) in the 64×64 matrix, its value is calculated using a two-dimensional Gaussian function formula. The result is that near the center point (40, 40), the matrix value is close to 1, while as the point (x, y) moves away from the center, the value smoothly decreases and approaches 0. The generated matrix acts like a spotlight, with the center of the spot aligned with the centroid of the set of protagonist points, to enhance the feature response of that region.

[0097] In an optional embodiment, the state of the truck wheels is determined by a classifier based on the final fused features, including:

[0098] The final fused features are input into a classifier consisting of two fully connected layers and a Softmax activation function layer. The first fully connected layer reduces the feature vector dimension to 128, and the second fully connected layer outputs scores for three categories. The Softmax activation function layer converts the scores into probabilities, and the category with the highest probability is the identified wheel state, which is: normal, worn, or damaged.

[0099] Specifically, a final fused feature representing key information about the wheel is obtained. This final fused feature is typically a high-dimensional vector, such as a one-dimensional vector of length 1024. This 1024-dimensional vector is then input into the first fully connected layer of the classifier. This layer performs a linear transformation using a weight matrix and a bias vector, mapping and compressing the 1024-dimensional vector into a 128-dimensional intermediate feature vector. This process can be seen as a further refinement and abstraction of the original fused feature.

[0100] The 128-dimensional intermediate vector is fed into the second fully connected layer. The output dimension of this layer is set to 3, corresponding to the three wheel state categories to be identified: normal, worn, and damaged. Therefore, the output of this layer is a vector containing three elements, for example, [2.5, 0.8, -1.2]. These three values ​​represent the original confidence scores, or log odds, of the model's assessment that the input feature belongs to the normal, worn, or damaged categories. Higher scores indicate a greater likelihood that the model considers the feature to belong to that category.

[0101] To transform the raw scores into an intuitive probability distribution, a vector containing three scores is passed to a Softmax activation function layer. The Softmax activation function layer exponentially and normalizes the three values, resulting in a sum of 1 for the three new output values, with each value between 0 and 1. For example, scores [2.5, 0.8, -1.2] might become [0.84, 0.14, 0.02] after processing by the Softmax activation function layer. This indicates that the model predicts the wheel has an 84% probability of being in a normal state, a 14% probability of being in a worn state, and a 2% probability of being in a damaged state. The category with the highest probability is selected as the recognition result, thus classifying the wheel as being in a normal state.

[0102] An embodiment of the truck wheel status recognition system provided by the present invention:

[0103] The truck wheel status recognition system includes a processor and a memory. The memory stores computer program instructions, which are executed by the processor to implement the truck wheel status recognition method described above.

[0104] The truck wheel status recognition system also includes other components well known to those skilled in the art, such as communication interfaces. Their settings and functions are known in the art and will not be described in detail here.

[0105] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented using computer-readable / executable instructions stored or otherwise maintained by such a computer-readable medium.

[0106] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A truck wheel condition recognition method, characterized by, The method comprises the following steps: S1, obtaining a truck wheel image to be recognized, extracting multiple resolution level feature maps through a backbone network; separating each level feature map into a low-frequency coefficient matrix and a high-frequency coefficient matrix through discrete cosine transformation, performing multi-scale low-frequency feature fusion on the low-frequency coefficient matrices of different levels to obtain a cross-layer low-frequency correlation map, and performing operation on the high-frequency coefficient matrices of the same level to obtain an intra-layer high-frequency structure map; S2, determining a candidate region on the level feature map, detecting corner points in the candidate region, calculating a covariance matrix of the corner point position, and taking the maximum eigenvalue of the covariance matrix as a corner point spatial distribution dispersion parameter; screening the corner points that form a local maximum convex polygon to form a main corner point set; S3, performing arctangent function processing and normalization on the corner point spatial distribution dispersion parameter to obtain a fusion weight coefficient; based on the fusion weight coefficient, performing weighted summation on the cross-layer low-frequency correlation map and the intra-layer high-frequency structure map to obtain preliminary fusion features; S4, constructing a spatial modulation matrix with the centroid of the main corner point set as the Gaussian kernel center, and element-wise multiplying the spatial modulation matrix with the preliminary fusion features to obtain final fusion features; determining the state of the truck wheel through a classifier according to the final fusion features.

2. The truck wheel state recognition method according to claim 1, characterized in that, The method of separating each level feature map into a low-frequency coefficient matrix and a high-frequency coefficient matrix comprises: For an input level feature map with a size of n*n, the level feature map is divided into multiple 8*8 sub-blocks, a two-dimensional discrete cosine transformation formula is applied to each sub-block for calculation, the coefficient value at the top left corner is taken as a direct current component to form a low-frequency coefficient matrix, and the remaining 63 coefficient values are taken as alternating current components to form a high-frequency coefficient matrix.

3. The truck wheel state recognition method according to claim 1, characterized in that, The method of performing multi-scale low-frequency feature fusion on the low-frequency coefficient matrices of different levels to obtain a cross-layer low-frequency correlation map comprises: The low-frequency coefficient matrix L3, L4, L5 of three levels A3, A4, A5 output by the backbone network is selected, the low-resolution low-frequency coefficient matrix L4, L5 is subjected to an upsampling operation to make the spatial resolution of the low-frequency coefficient matrix consistent with The three low-frequency coefficient matrices of the same size are added element by element to generate a cross-layer low-frequency correlation graph.

4. The truck wheel state recognition method according to claim 1, characterized in that, The method of performing operation on the high-frequency coefficient matrices of the same level to obtain an intra-layer high-frequency structure map comprises: Autocorrelation operation is performed on the high-frequency coefficient matrices H3, H4, and H5 of the three levels A3, A4, and A5 respectively to obtain three independent intra-layer high-frequency structure maps; the spatial resolution of the low-resolution intra-layer high-frequency structure map is made consistent with that of the intra-layer high-frequency structure map with the highest resolution through upsampling operation; and the three intra-layer high-frequency structure maps with consistent sizes are added element by element to be fused into a single intra-layer high-frequency structure map.

5. The truck wheel condition identification method according to claim 1, characterized by, The method of detecting corner points in the candidate region and calculating a covariance matrix of the corner point position comprises: Using Harris corner detection algorithm, set the response threshold, identify all the corner points in the candidate region; the two-dimensional coordinates of the detected N corner points are expressed as a set , the mean values of x coordinate and y coordinate and are calculated, and the covariance matrix of is calculated according to the following formula : wherein, where x i is the horizontal coordinate value in the two-dimensional coordinate of the i-th corner point, y i is the vertical coordinate value in the two-dimensional coordinate of the i-th corner point, denotes the mean value of the x coordinate in the two-dimensional coordinate in the set, denotes the mean value of the y coordinate in the two-dimensional coordinate in the set, , , , denote the elements in the covariance matrix, respectively.

6. The truck wheel condition identification method according to claim 1, characterized by, The method of screening the corner points that form a local maximum convex polygon to form a main corner point set comprises: Graham scan algorithm is applied for screening, the corner point with the smallest y-coordinate value is found as a starting base point among all the corner points, the polar angles of the remaining corner points relative to the starting base point are calculated and sorted in ascending order, the sorted corner points are traversed in sequence through a stack structure, all the corner points forming concave angles are removed, and the corner points forming a convex polygon are retained as the main corner point set.

7. The truck wheel condition identification method according to claim 1, characterized by, The method of performing arctangent function processing and normalization on the corner point spatial distribution dispersion parameter to obtain a fusion weight coefficient comprises: The discrete degree parameter of the spatial distribution of the angular point is The fusion weight coefficient w is calculated by the following formula: where arctan() represents the inverse tangent function, and the formula maps to the interval [0, 1). mapping to the interval [0, 1).

8. The truck wheel condition identification method according to claim 1, characterized by, The method of constructing a spatial modulation matrix with the centroid of the main corner point set as the Gaussian kernel center comprises: Calculate the average of all N corner point coordinates (x, y) in the main corner point set to obtain the centroid coordinates ; Set the standard deviation of the Gaussian kernel 0.5 times the average distance of all corner points in the main corner point set to the centroid; based on the centroid coordinates and the standard deviation , generate a spatial modulation matrix with the same size as the preliminary fusion feature using the two-dimensional Gaussian function G(x, y): 。 9. The truck wheel condition identification method according to claim 1, characterized by, The method of determining the state of the truck wheel through a classifier according to the final fusion features comprises: The final fusion features are input into a classifier composed of two fully connected layers and a Softmax activation function layer, wherein the first fully connected layer reduces the feature vector dimension to 128, the second fully connected layer outputs scores of 3 categories, and the scores are converted into probabilities through the Softmax activation function layer, and the category with the highest probability is the recognized state of the wheel, and the categories are: normal, wear, and damage.

10. A truck wheel condition identification system characterized by, The application also provides a computer readable storage medium, comprising a computer program instruction, wherein the computer program instruction is executed by a processor to implement the truck wheel state recognition method in any one of claims 1-9.

Images