Cascade industrial vision quality inspection method and system based on rotating target detection

Abstract

The application is suitable for the technical field of industrial vision detection, and provides a cascaded industrial vision quality detection method and system based on rotating target detection. The method comprises the following steps: inputting an original image containing a workpiece to be detected into a rotating target detection model, and outputting rotating bounding box parameters of the workpiece to be detected; determining four vertex coordinates of a target region corresponding to the workpiece to be detected in the original image based on the rotating bounding box parameters, and performing rotating mapping on the target region based on a rotating angle to obtain a rotating alignment region consistent with the main direction of the workpiece to be detected; performing pretreatment operations such as reverse rotating correction, cutting and size normalization on the rotating alignment region based on the rotating bounding box parameters to obtain a standardized target image block; and performing an industrial vision quality detection task based on the standardized target image block, and outputting an industrial vision quality detection result, thereby improving the detection accuracy of the rotating target, and further improving the processing efficiency and accuracy of the downstream task.

Description

Technical Field

[0001] This application belongs to the field of industrial vision inspection technology, and in particular relates to a cascaded industrial vision quality inspection method and system based on rotating target detection. Background Technology

[0002] As the automotive industry continues to evolve towards intelligent manufacturing and flexible production lines, automated quality inspection in the vehicle manufacturing process has become a crucial link in ensuring product consistency and production efficiency. In automotive welding, painting, and final assembly workshops, there are numerous structural components and body panels with specific spatial orientations. These inspection objects typically exhibit geometric characteristics such as tilted placement and highly unbalanced aspect ratios, such as crash beams, chassis longitudinal beams, and identification codes. Achieving accurate identification and stable positioning of these targets is fundamental for subsequent applications such as dimensional measurement, text recognition, classification, and assembly guidance.

[0003] Existing industrial vision inspection solutions mainly include target detection methods based on horizontal bounding boxes and pixel-level recognition methods based on instance segmentation. While the former offers good real-time performance, when handling tilted or slender targets, the detection bounding box often contains a large amount of background, leading to insufficient localization accuracy and significant background interference. This negatively impacts downstream classification, optical character recognition (OCR), or defect detection tasks. The latter, while capable of accurately depicting target contours, suffers from high pixel-level annotation costs, large training and inference computational loads, and its output typically cannot be directly used as input for downstream tasks, requiring additional image processing, making engineering implementation complex. Furthermore, in existing solutions, the target detection module and downstream task modules are usually independent, lacking a collaborative optimization design from "target detection result" to "downstream task input." This results in the detection output being directly fed into subsequent stages without targeted correction and standardization, making it difficult to simultaneously achieve detection accuracy, downstream task accuracy, annotation efficiency, real-time performance, and system integration.

[0004] Therefore, how to reduce background interference while ensuring detection accuracy for tilted objects with extremely uneven aspect ratios, and provide target images with uniform posture and regular size for downstream tasks such as classification, text recognition, defect detection, or size measurement, so as to improve the processing efficiency and accuracy of downstream tasks, has become an urgent technical problem to be solved. Summary of the Invention

[0005] To address at least one of the aforementioned technical problems, and for inspection objects that are tilted and have extremely uneven aspect ratios, this application proposes a cascaded industrial vision quality inspection method and system based on rotating target detection. This method aims to reduce background interference while ensuring detection accuracy, and to provide target images with uniform posture and regular size for subsequent downstream tasks such as classification, text recognition, defect detection, or size measurement, thereby improving the processing efficiency and accuracy of downstream tasks.

[0006] In a first aspect, this application provides a cascaded industrial vision quality inspection method based on rotating target detection, the method comprising: S1, Acquire the original image containing the workpiece to be inspected; S2, input the original image into the rotating target detection model, and output the rotation bounding box parameters of the workpiece to be detected. The rotation bounding box parameters include at least the center coordinates, width, height and rotation angle. S3, based on the rotation bounding box parameters, determine the coordinates of the four vertices of the target region corresponding to the workpiece to be detected in the original image, and perform rotation mapping on the target region based on the rotation angle to obtain a rotation alignment region consistent with the main direction of the workpiece to be detected; S4. Based on the rotated bounding box parameters, the rotated aligned region is preprocessed to obtain a standardized target image patch; S5. Based on the standardized target image block, perform industrial visual quality inspection task and output industrial visual quality inspection results.

[0007] In one possible implementation, the rotating target detection model includes a rotating feature extraction module and a rotating bounding box joint regression module; The rotation feature extraction module is used to extract rotation features and perform multi-scale fusion on the original image, and output multi-scale rotation features. The multi-scale rotation features include at least a high-resolution feature map, a medium-resolution feature map and a low-resolution feature map. The high-resolution feature map is used to represent edge details and positional information, the medium-resolution feature map is used to represent local structure and semantic information, and the low-resolution feature map is used to represent overall semantic and geometric information. The rotating bounding box joint regression module is used to perform rotating bounding box joint regression prediction based on the multi-scale rotation features and output the rotating bounding box parameters of the workpiece to be detected.

[0008] In one possible implementation, the rotation feature extraction module includes a feature extraction unit, a geometric prior guidance unit, a multi-scale feature fusion unit, and a path enhancement unit connected in sequence. The feature extraction unit is used to perform convolutional downsampling, feature enhancement and spatial pyramid pooling processing on the original image, and output a first layer feature map, a second layer feature map and a third layer feature map, with the downsampling factor corresponding to the first layer feature map, the second layer feature map and the third layer feature map increasing sequentially. The geometric prior guidance unit is used to generate geometric prior weights based on the aspect ratio and main direction statistical results of the workpiece to be detected in the training samples. The multi-scale feature fusion unit is used to upsample the third-layer feature map based on the geometric prior weights, and then perform weighted fusion with the first-layer feature map and the second-layer feature map respectively to output preliminary fused features; The path enhancement unit is used to downsample the preliminary fusion features step by step, and then fuse the downsampled features with the features of the corresponding scale again to output the multi-scale rotation features of the workpiece to be detected.

[0009] In one possible implementation, the geometric prior guidance unit is specifically used for: The aspect ratio and size distribution of each rotated bounding box in the training samples are statistically analyzed to obtain geometric distribution information; Based on the geometric distribution information, K-means clustering based on intersection-union distance is used to obtain multiple scale priors; The target K value is determined based on the trend of the average crossover ratio (CRO) corresponding to different K values, and a scale prior set matching the geometric features of the workpiece to be inspected is obtained. The scale prior set is matched with feature maps of different scales to generate the geometric prior weights.

[0010] In one possible implementation, the rotated bounding box joint regression module includes a multi-scale rotation feature input unit, a bounding box parameter regression unit, a rotation angle prediction unit, and a parameter output unit connected in sequence. The multi-scale rotation feature input unit is used to perform channel mapping on the high-resolution feature map, the medium-resolution feature map, and the low-resolution feature map to obtain a feature representation suitable for rotating target prediction. The bounding box parameter regression unit is used to regress the center coordinates, width, and height of the workpiece to be detected through a convolutional prediction head based on the feature representation suitable for predicting rotating targets; wherein the width and height are initialized based on a scale prior set that matches the geometric features of the workpiece to be detected and then corrected by regression. The rotation angle prediction unit is used to output the sine and cosine values ​​of the angle based on the feature representation suitable for predicting the rotation target, and to recover the rotation angle of the workpiece to be detected based on the sine and cosine values ​​of the angle. The parameter output unit is used to generate and output the rotation bounding box parameters based on the center coordinates, width, height and rotation angle.

[0011] In one possible implementation, the rotating target detection model is optimized using a joint loss function during the training phase; The joint loss function includes the rotated bounding box regression loss, the classification loss, and the rotation angle loss; Wherein, the rotation bounding box regression loss is a regression loss based on Gaussian Wasserstein distance, the classification loss is a focus loss, and the rotation angle loss is a loss function based on angle periodic constraints.

[0012] In one possible implementation, step S3 includes: S31, using the center coordinates of the rotated bounding box as the origin, determine the coordinates of the four vertices of the target region based on the width and height of the rotated bounding box; S32, construct a rotation matrix based on the rotation angle, and map the coordinates of the four vertices to the original image coordinate system; S33, perform interpolation alignment processing on the target region based on the coordinates of the four mapped vertices to obtain the rotated aligned region.

[0013] In one possible implementation, step S4 includes: S41, calculate the reverse rotation matrix based on the rotation angle, and perform an affine transformation on the rotation alignment region to adjust the rotation alignment region to a horizontal orientation; S42, based on the width and height in the rotated bounding box parameters, the image region after posture correction is cropped to obtain a rectangular image containing only the main body of the workpiece to be detected; S43, the rectangular image is scaled to a preset size to obtain the standardized target image block.

[0014] In one possible implementation, the industrial vision quality inspection task includes at least one of classification, optical character recognition, defect detection, and dimensional measurement.

[0015] On the other hand, this application also provides a computer system including a memory and a processor; the memory stores program code executable by the processor; the program code is used to perform the method as described in the first aspect or any of the implementations thereof.

[0016] Compared with existing technologies, the advantages of this application are as follows: By outputting the rotation bounding box parameters of the workpiece to be detected through the rotation target detection model, the center position, size information, and rotation angle information of workpieces that are tilted or have extremely uneven aspect ratios can be obtained more accurately, thus improving the target positioning accuracy. Furthermore, based on the rotation bounding box parameters, preprocessing operations such as rotation mapping, reverse rotation correction, cropping, and size normalization are performed on the corresponding regions in the original image. This effectively reduces the interference of background regions and pose differences on subsequent processing, resulting in standardized target image blocks with uniform pose and regular size. These standardized target image blocks can be directly used for downstream industrial visual quality inspection tasks such as classification, optical character recognition, defect detection, or size measurement, which helps to improve the processing efficiency and recognition accuracy of downstream tasks and reduce false detections, missed detections, and additional post-processing costs caused by pose inconsistencies and background interference.

[0017] It is understood that the computer system provided in this application has the same beneficial effects as the cascaded industrial vision quality inspection method based on rotating target detection described above, and will not be repeated here. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a cascaded industrial vision quality inspection method based on rotating target detection, provided in an embodiment of this application; Figure 2 A schematic diagram of the architecture of a cascaded industrial vision quality inspection device based on rotating target detection provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the detection results of a traditional horizontal bounding box detection scheme in actual production. Figure 4 This is a schematic diagram of the detection results of a cascaded industrial vision quality inspection method based on rotating target detection provided in an embodiment of this application during actual production. Detailed Implementation

[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0021] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0022] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0024] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0026] This application is preferably applicable to industrial visual quality inspection in automotive welding, painting, and final assembly scenarios. For ease of understanding, the technical solution of this application will be described in detail below with reference to the accompanying drawings.

[0027] Figure 1 This is a flowchart illustrating a cascaded industrial vision quality inspection method based on rotating target detection, provided as an embodiment of this application. Figure 1 As shown, for ease of explanation, only the parts relevant to this embodiment are shown. The method provided in this embodiment includes the following steps: S1, Acquire the original image containing the workpiece to be inspected.

[0028] Optionally, an image acquisition subsystem acquires an original image containing the workpiece to be inspected. The image acquisition subsystem includes at least an industrial camera, a lens, and a light source. The industrial camera can be a GigE Vision or USB 3.0 interface industrial area array camera, such as a 5-megapixel global exposure CMOS camera, and is mounted directly above or to the side of the production line inspection station. The lens is preferably a fixed-focus or zoom industrial lens to ensure the field of view covers the entire inspection area of ​​the workpiece. For the geometric characteristics of slender workpieces, a low-angle ring light source or backlight is preferred to highlight the edge features of the workpiece and suppress ambient light interference. In actual operation, a programmable logic controller (PLC) can send a trigger signal to the industrial camera to complete the original image acquisition at a preset station. In this embodiment, the workpiece to be inspected is preferably a workpiece placed at an angle and / or with a highly uneven aspect ratio.

[0029] Furthermore, the acquired raw images undergo preprocessing, such as using Contrast Limited Adaptive Histogram Equalization (CLAHE) for adaptive histogram equalization and Gaussian filtering to suppress noise. During the training phase, samples can also be enhanced by rotation and affine transformation, with the rotation angle preferably limited to the actual tilt range of the workpiece to avoid generating unreasonable postures. To maintain the aspect ratio without distortion, the affine transformation preferably preserves the original geometric proportions of the workpiece.

[0030] S2, input the original image into the rotating target detection model, and output the rotation bounding box parameters of the workpiece to be detected. The rotation bounding box parameters include at least the center coordinates, width, height and rotation angle.

[0031] In one possible implementation, the rotating target detection model includes a rotating feature extraction module and a rotating bounding box joint regression module; The rotation feature extraction module is used to extract rotation features and perform multi-scale fusion on the original image, and output multi-scale rotation features. The multi-scale rotation features include at least a high-resolution feature map, a medium-resolution feature map and a low-resolution feature map. The high-resolution feature map is used to represent edge details and positional information, the medium-resolution feature map is used to represent local structure and semantic information, and the low-resolution feature map is used to represent overall semantic and geometric information. The Rotated Bounding Box Joint Regression Module is used to perform joint regression prediction of rotated bounding boxes based on multi-scale rotation features, and outputs the rotated bounding box parameters of the workpiece to be inspected.

[0032] As an example, the rotation feature extraction module includes a feature extraction unit, a geometric prior guidance unit, a multi-scale feature fusion unit, and a path enhancement unit connected in sequence. The feature extraction unit is used to perform convolutional downsampling, feature enhancement and spatial pyramid pooling processing on the original image, and outputs the first layer feature map, the second layer feature map and the third layer feature map, and the downsampling factor corresponding to the first layer feature map, the second layer feature map and the third layer feature map increases sequentially. The geometric prior guidance unit is used to generate geometric prior weights based on the aspect ratio and main direction statistics of the workpiece to be detected in the training samples. The multi-scale feature fusion unit is used to upsample the third-layer feature map based on geometric prior weights, and then perform weighted fusion with the first-layer and second-layer feature maps respectively to output preliminary fused features; The path enhancement unit is used to downsample the preliminary fused features step by step, and then fuse the downsampled features with the features of the corresponding scale again to output the multi-scale rotation features of the workpiece to be detected.

[0033] For example, the feature extraction unit preferably employs a Backbone feature extraction network, which consists of multiple cascaded convolutional layers, a C2f feature enhancement module, and a terminal Spatial Pyramid Pooling-Fast (SPPF) module. The input original image first enters the convolutional layer for downsampling, then undergoes feature enhancement through the C2f module, and finally passes through the SPPF module to integrate multi-scale contextual information, progressively extracting edge, texture, and structural semantic information from the original image, and outputting feature maps at three different scales.

[0034] For example, a geometric prior guidance unit, preferably embedded in the feature fusion node, receives three feature maps at different scales output by the feature extraction unit. It generates geometric prior weights by calculating the geometric distribution priors of the workpiece to be detected, such as aspect ratio and principal direction. These geometric prior weights guide subsequent convolution and feature fusion operations, enabling the convolution operation in the subsequent feature fusion process to automatically adjust the receptive field direction according to the elongated shape of the workpiece, thereby enhancing the feature response to rotated and slender workpieces. The principal direction is the overall geometric extension direction of the workpiece to be detected, preferably the long side direction of the rotated bounding box.

[0035] For example, the multi-scale feature fusion unit preferably adopts a Feature Pyramid Network (FPN). It adopts a top-down path, upsamples the third-layer feature map output by the feature extraction unit, and then concatenates or adds it with the first-layer and second-layer feature maps respectively. In this process, the geometric prior weights generated by the geometric prior guidance unit are introduced to transfer the semantic information of the third-layer feature map to the first-layer and second-layer feature maps, so as to achieve the initial fusion of semantic information and location information.

[0036] For example, the path enhancement unit preferably employs a Path Aggregation Network (PAN), using a bottom-up path. The preliminary fused features output by the multi-scale feature fusion unit, which include the positional information of the first and second layer feature maps and the semantic information of the third layer feature map, are downsampled again at each level and fused with the first, second, and third layer feature maps of the corresponding scales. This enhances the transmission of positional information between features at different scales, achieving bidirectional deep fusion of positional and semantic information, and finally outputting a multi-scale fused feature map.

[0037] In a preferred embodiment, to address the challenges posed by elongated rotating workpieces in industrial settings, which have large width-to-height ratios, significant dimensional variations, and difficulties in matching traditional target priors, the geometric prior guidance unit may specifically execute the following steps: The aspect ratio and size distribution of each rotated bounding box in the training samples are statistically analyzed to obtain geometric distribution information; Based on geometric distribution information, K-means clustering based on intersection-union distance is used to obtain priors at multiple scales; The target K value is determined based on the trend of the average crossover ratio (CRO) corresponding to different K values, and a scale prior set matching the geometric features of the workpiece to be inspected is obtained. The scale prior set is matched with feature maps of different scales to generate geometric prior weights.

[0038] In practice, we first statistically analyze each rotated bounding box in the training samples and calculate its aspect ratio. For the i-th rotated bounding box, its aspect ratio can be expressed as: ; in, Let i be the aspect ratio of the i-th rotated bounding box. Let be the width of the i-th rotated bounding box. Let be the height of the i-th rotated bounding box.

[0039] This yields the target aspect ratio distribution: ; in, Let n be the set of aspect ratio distributions corresponding to the training samples, and n be the total number of rotated bounding boxes in the training samples.

[0040] In the industrial scenarios addressed in this application, the aspect ratio of elongated workpieces is mainly concentrated between 5:1 and 15:1. Since traditionally manually set aspect ratios are mostly 1:1, 1:2, or 2:1, they are difficult to effectively cover slender workpieces with aspect ratios such as 10:1. Therefore, K-means clustering is used to generate target priors, transforming each workpiece into clustering inputs. ,get: ; in, This represents the set of samples used for clustering. This represents the width and height of the i-th rotated bounding box.

[0041] To measure the degree of matching between the prior bounding box and the ground truth bounding box, we use the standard Euclidean distance, which can be expressed as: ; in, For Euclidean distance, and These are the width and height of the first rotated bounding box, respectively. and These are the width and height of the second rotated bounding box, respectively.

[0042] Since Euclidean distance is easily affected by the absolute value of the scale, the Intersection over Union (IoU) distance is preferred. ; in, Represents the target box With prior boxes IoU distance between them For the target box With prior boxes The intersection-union ratio can be expressed as: ; in, For the target box With prior boxes The area of ​​the intersecting region, For the target box With prior boxes The area of ​​the union region. The degree of shape matching can be directly reflected by the IoU distance.

[0043] Because the choice of K value directly affects the clustering effect, it can be iterated through. We calculate all possible values ​​for K and plot the average IoU curve. We then select the inflection point where the curve flattens out as the optimal K value to avoid overfitting and computational redundancy. The average IoU can be expressed as: ; in, is the average intersection-union ratio, used to measure the overall matching degree between cluster centers and all ground truth bounding boxes; i is the index of the ground truth bounding box; j is the index of the cluster center; For the i-th ground truth bounding box, take the maximum value of its intersection-union ratio with all cluster centers; This represents the i-th ground truth bounding box; Let N be the j-th cluster center, and N be the total number of true target boxes.

[0044] Assuming calculations show that the improvement in average IoU plateaus after K=5, then K is chosen as 5. For example, the following scale prior set can be obtained after clustering: ; Then, the average scale of the scale prior in each cluster is calculated. The average scale of the k-th cluster center can be expressed as: ; in, This represents the average scale of the prior at the k-th scale. and Let represent the width and height of the prior at the k-th scale, respectively.

[0045] Furthermore, the scale matching relationship is established as follows: ; in, This represents the scale matching relationship between the k-th scale prior and the ith feature layer. This represents the average scale of the prior at the k-th scale. This represents the downsampling factor of the i-th feature layer in the multi-scale feature fusion unit relative to the input image.

[0046] By selection The feature layer that is closest to the preset effective feature scale range can automatically match the target scale prior with the feature layers of different scales of FPN, thereby improving the detection effect of long strip targets.

[0047] As an example, the rotated bounding box joint regression module includes a multi-scale rotation feature input unit, a bounding box parameter regression unit, a rotation angle prediction unit, and a parameter output unit connected in sequence. A multi-scale rotation feature input unit is used to perform channel mapping on high-resolution feature maps, medium-resolution feature maps and low-resolution feature maps to obtain feature representations suitable for rotating target prediction. The bounding box parameter regression unit is used to regress the center coordinates, width, and height of the workpiece to be detected through a convolutional prediction head based on a feature representation suitable for predicting rotating targets. The width and height are initialized based on a scale prior set that matches the geometric features of the workpiece to be detected and then corrected by regression. The rotation angle prediction unit is used to output the sine and cosine values ​​of the angle based on the feature representation suitable for predicting the rotation target, and to recover the rotation angle of the workpiece to be detected based on the sine and cosine values ​​of the angle. The parameter output unit is used to generate and output the parameters of the rotated bounding box based on the center coordinates, width, height and rotation angle.

[0048] For example, the multi-scale rotation features input to the rotated bounding box joint regression module can be represented as: ; in, For high-resolution feature maps, It is a medium-resolution feature map. This is a low-resolution feature map. Multi-scale rotation features must include at least the target's spatial location information, edge texture information, aspect ratio information, and rotation direction information.

[0049] For example, a multi-scale rotation feature input unit adjusts the channels of features at different scales through a convolutional mapping layer, which can be represented as: ; in, This represents the i-th scale feature map of the input. This represents the feature representation after channel mapping. This represents a convolutional mapping operation, yielding a feature representation suitable for predicting rotated targets. This method allows for feature input of workpieces of different sizes, enabling smaller targets to utilize higher-resolution features more effectively, while larger targets combine deep semantic information from lower-resolution feature maps, thus ensuring that rotating targets of different scales can participate in prediction.

[0050] For example, the bounding box parameter regression unit receives the feature representation output by the multi-scale rotated feature input unit. Then, the center coordinates, width, and height of the workpiece to be inspected are regressed using a convolutional prediction head. The offset can be predicted first. ; in, For the center position offset, This represents the regression correction for width and height.

[0051] During the decoding stage, the final geometric parameters can be obtained: ; ; ; ; in, and These are the center coordinates of the predicted rotating box. and These are the predicted width and height of the rotating frame, respectively. and To initialize the center position, and The initial width and height are provided by the scale prior set.

[0052] Unlike traditional methods that directly regress dimensions, this application uses target geometric priors as dimension initialization, enabling the network to learn the target dimension offset, thereby improving the stability of dimension prediction for long and narrow workpieces.

[0053] For example, since industrial workpieces can rotate in any direction, directly predicting the angle would result in periodic discontinuities. Therefore, an angle encoding method is used, and the rotation angle prediction unit does not directly output the angle. The output is: ; ; The rotation angle can then be restored using the following method: ; in, For the cosine component in angle coding, For the sine component in angle coding, This represents the two-parameter arctangent function. This method can solve the boundary problems of 0° / 180° and the angular discontinuity problems caused by changes in the direction of long, narrow targets.

[0054] In one possible implementation, the rotating target detection model is optimized using a joint loss function during the training phase. The joint loss function includes a rotation bounding box regression loss, a classification loss, and a rotation angle loss. The rotation bounding box regression loss is a regression loss based on Gaussian Wasserstein distance, the classification loss is a focus loss, and the rotation angle loss is a loss function based on angular periodic constraints.

[0055] Optionally, due to the arbitrary rotation, large differences in width-to-height ratios, and wide range of dimensional variations of workpieces in industrial production environments, traditional bounding box loss methods struggle to simultaneously account for changes in position, size, and orientation. Therefore, this application constructs a joint regression loss function based on rotational geometric constraints as follows: ; Where L is the joint loss function, The rotation bounding box regression loss is used to optimize the target position, size, and rotation space distribution; This is the classification loss, used to improve the ability to identify target categories; The rotation angle loss is used to optimize target orientation prediction. These are the weighting coefficients for the corresponding loss terms.

[0056] The loss function for the rotated bounding box uses a regression loss based on Gaussian Wasserstein distance. Compared to traditional IoU and Complete Intersection over Union (CIoU), traditional methods may confuse width and height when handling a 90° rotation of the target. The regression loss based on Gaussian Wasserstein distance transforms the rotated bounding box into a two-dimensional probability distribution, thus avoiding the ambiguity of width and height swapping. Its distance form can be expressed as: ; in, The Gaussian Wasserstein distance between the predicted bounding box and the ground truth bounding box; Center of the prediction box; Centered at the true bounding box; This is the covariance matrix corresponding to the predicted bounding box; The covariance matrix corresponding to the true bounding box; This is for matrix trace operations.

[0057] The rotation bounding box regression loss can be further defined as: ; This loss avoids the discontinuity of bounding boxes caused by rotation angles, solves the problem of exchanging the width and height of long targets, and can provide effective gradients even if the predicted box and the ground truth box do not overlap, thus improving the accuracy of position and size regression of rotated targets.

[0058] The classification loss function uses focal loss to dynamically adjust sample weights. When a sample is easy to classify, its weight is reduced; when a sample is difficult to classify, a larger gradient is retained to improve the model's ability to identify small and edge targets. Its definition is: ; ; in, This represents the probability of a positive prediction, where the positive class is a predefined target category, preferably the target category corresponding to the workpiece to be detected. The true category label is defined as follows: when y=1, it indicates that the sample belongs to the positive class; when y=0, it indicates that the sample belongs to the negative class. The negative class is preferably the background or non-target category. The predicted probability corresponding to the true category; For class balancing parameters; Adjust parameters for difficult samples. When samples are easy to classify... ,but Conversely, when the sample is difficult to classify, Larger values ​​preserve a larger gradient.

[0059] Long industrial workpieces typically have directional characteristics, but most exhibit 180° rotational symmetry. For example, a metal rod parallel to the x-axis can be considered to have an angle of 0°, but after reversing its orientation, it can still be considered to have the same configuration, with an angle of 180°. Therefore, its effective angular period is... , instead of 2 Traditional angle loss is usually achieved through... Calculations are performed, in which, For the prediction angle, This is the actual angle; the calculation period for this formula is 2. Therefore, it cannot effectively handle 180° situations.

[0060] This application employs the following rotation angle loss: ; Since the period of cos(2θ) is Therefore, this rotation angle loss can eliminate angle boundary jumps, avoid 180° directional ambiguity, and improve the prediction stability of the long strip rotation target angle.

[0061] The final joint loss function can be written as: .

[0062] S3. Based on the rotation bounding box parameters, determine the coordinates of the four vertices of the target region corresponding to the workpiece to be detected in the original image, and perform rotation mapping on the target region based on the rotation angle to obtain a rotationally aligned region consistent with the main direction of the workpiece to be detected.

[0063] Specifically, since target workpieces in industrial settings are often placed in arbitrary orientations, directly using traditional horizontal rectangular region extraction methods would introduce a large number of invalid background areas and cause loss of target edge information. Therefore, this step uses the bounding box parameters to accurately locate the target region in the original image based on the spatial position and orientation information of the rotated box, and completes the rotation region mapping.

[0064] In one possible implementation, step S3 includes: S31, using the center coordinates of the rotated bounding box as the origin, determine the coordinates of the four vertices of the target region based on the width and height of the rotated bounding box.

[0065] As an example, the parameters of a rotated bounding box can be expressed as: ; Where x and y represent the center position; These represent the width and height of the bounding box of the rotated frame, respectively. The angle is the rotation angle.

[0066] Establish a local coordinate system based on the center coordinates, width and height dimensions, and rotation angle of the rotated frame. The local coordinates of the rotated frame can be represented as follows: ; in, Let be the coordinates of the i-th vertex in the local coordinate system. In the local coordinate system, the first The x-coordinates of the vertices, In the local coordinate system, the first The y-coordinates of the four vertices. The coordinates of the four vertices are respectively... , , , .

[0067] S32 constructs a rotation matrix based on the rotation angle and maps the coordinates of the four vertices to the original image coordinate system.

[0068] As an example, based on the rotation angle The constructed rotation matrix is: ; Mapping the coordinates of the four vertices of the rotated bounding box to the original image coordinates can be represented as: , ; in, Let i be the coordinates of the i-th actual vertex in the original image coordinate system. These are the center coordinates of the rotated bounding box. This method ensures that the target area is always positioned according to the direction of the rotated bounding box.

[0069] S33, based on the coordinates of the four mapped vertices, performs interpolation and alignment processing on the target region to obtain the rotated aligned region.

[0070] As an example, region mapping is performed on the original image based on the calculated coordinates of the four vertices. Image interpolation is used. ; in, For the original image, Represents the coordinates in the mapped target region. Pixel value at that location, Let i be the coordinates of the i-th neighboring pixel in the original image that is involved in the interpolation. This represents the corresponding interpolation weight. By rotating the region for alignment, a workpiece in any orientation can be transformed into a regular rectangular coordinate space.

[0071] S4, based on the rotation bounding box parameters, preprocesses the rotation-aligned region to obtain a standardized target image patch.

[0072] Preferably, the preprocessing operations include at least reverse rotation correction, cutting, and size normalization.

[0073] In one possible implementation, step S4 includes: S41, calculate the reverse rotation matrix based on the rotation angle, and perform an affine transformation on the rotation alignment region to adjust the rotation alignment region to a horizontal orientation.

[0074] As an example, based on the detected rotation angle Calculate the reverse rotation matrix: ; Then, through affine transformation: ; in, This represents the image after the rotated regions have been aligned. This is the image after posture correction. This is the translation compensation amount in the x-direction. This represents the translation compensation amount in the y-direction.

[0075] S42, based on the width and height in the rotation bounding box parameters, crop the image region after attitude correction to obtain a rectangular image containing only the main body of the workpiece to be detected.

[0076] As an example, after completing the posture correction, based on the rotation frame size... The image after pose correction is cropped to obtain: ; in, Indicates the region of interest. Let x be the x-coordinate of the top-left corner of the cropping area. This represents the ordinate of the top-left corner of the cropped area. To cut the width of the area, The height of the cropping region is defined. This generates a rectangular image containing only the main body of the workpiece to be inspected. Since the rotation box has already undergone orientation matching during the inspection phase, the resulting image has the advantages of high height matching between the cropping region and the target boundary, and a reduced background proportion.

[0077] S43, scale the rectangular image to a preset size to obtain a standardized target image block.

[0078] As an example, due to differences in target size, the ROI size is normalized to ensure consistent input for downstream tasks. The resulting image will be: ; in, To standardize target image patches, This indicates an image scaling operation. and These represent the normalized target width and target height, respectively. The normalization size is specified by the downstream task, and the final output normalized target image patch serves as the input for the subsequent S5 task.

[0079] This step generates high-quality target image patches with clean backgrounds, normalized poses, and standardized sizes.

[0080] S5 performs industrial visual quality inspection tasks based on standardized target image blocks and outputs industrial visual quality inspection results.

[0081] Optionally, industrial vision inspection tasks include at least one of sorting, optical character recognition, defect detection, and dimensional measurement.

[0082] As an example, classification tasks can use classification networks to determine the category of workpieces, such as "qualified / unqualified" or specific models; OCR text recognition tasks can use OCR models to extract character information from the surface of workpieces, such as batch numbers, traceability codes, production dates, etc.; defect detection tasks can use object detection or image segmentation models to identify defects such as scratches, pits, and burrs on the surface of workpieces; and dimensional measurement tasks can use edge detection or key point localization models to measure the key dimensional parameters of workpieces.

[0083] Furthermore, the final quality inspection results, inspection frame parameters, and defect category information can be visualized on the host computer software running in the industrial control computer and recorded in the local database or cloud management system to form a quality traceability file.

[0084] The final quality inspection results, inspection frame parameters, and defect category information are visualized on the host computer, and corresponding audible and visual alarms or control signals are issued. These results can also be fed back to the PLC in real time, forming a cascaded closed loop of "inspection-judgment-feedback" to guide subsequent sorting or assembly operations.

[0085] In one specific embodiment, this application also provides a cascaded industrial vision quality inspection system based on rotating target detection, which may include: The image acquisition subsystem, consisting of an industrial camera, lens, and light source, is used to acquire raw images containing the workpiece to be inspected. The image processing and inference subsystem is deployed on the computing device of the industrial control computer and runs the rotation detection, image correction and standardization and downstream task processing algorithms provided in this application. The host computer software subsystem provides a user interface for parameter configuration, result visualization, data storage, and communication management. The production line control subsystem consists of a PLC controller and actuators (robotic arms, alarm lights, etc.) used to receive judgment results and perform sorting, rejection, or alarm operations.

[0086] To illustrate the feasibility and scalability of the technical solution of this application, the rotating target detection model in step S2 and the deployment carrier of the method of this application can be equivalently replaced or adaptively adjusted without departing from the core inventive concept of this application. However, it should be noted that the above replacements or adjustments only involve changes in the model implementation or deployment platform, and do not change the cascaded processing concept of this application, which is based on the rotating target detection results, sequentially performs rotational region alignment, image correction, and region standardization processing on the corresponding regions of the workpiece to be detected in the original image, and inputs the resulting standardized target image blocks into downstream quality inspection tasks. Specific alternatives include: 1. Alternatives to rotating target detection models In an optional embodiment, the rotating target detection model in step S2 can be implemented using different deep learning network structures. Optionally, the backbone feature extraction part can be implemented using a convolutional neural network, a Transformer network, or a hybrid network combining convolution and attention mechanisms; the multi-scale feature fusion part can use other network structures capable of multi-scale feature interaction and rotation bounding box prediction; the rotation bounding box joint regression part can be implemented using different forms of detection heads, as long as it can still output the rotation bounding box parameters of the workpiece to be detected based on the original image, and provide them for subsequent steps S3 and S4 to perform rotation region alignment, image correction, and region standardization processing.

[0087] For example, the backbone feature extraction network can be replaced with Swing Transformer, Vision Transformer (ViT), or other network structures with global feature modeling capabilities; alternatively, other network frameworks or feature fusion modules for rotating target detection can be used to perform rotating bounding box detection on tilted workpieces.

[0088] Compared with the above alternatives, the preferred embodiment of this application adopts a processing method that combines rotation feature extraction, multi-scale fusion and joint regression of rotation bounding boxes. This method is more conducive to achieving stable detection of workpieces that are placed at an angle and have extremely uneven aspect ratios in the automotive production line. It is also easier to connect with subsequent rotation area alignment, image correction and area standardization processing, thereby achieving a better balance between detection accuracy, processing efficiency and engineering deployment convenience.

[0089] 2. Alternative deployment methods In one optional embodiment, the method of this application can be deployed on different computing platforms. Optionally, the trained rotating target detection model and downstream quality inspection model can be deployed in industrial control computers, embedded edge computing devices, industrial smart cameras, or cloud servers. Embedded edge computing devices can achieve lightweight deployment through model pruning, quantization, distillation, or operator optimization; cloud servers can communicate with production line image acquisition terminals and control systems via networks to achieve image uploading, model inference, and result feedback.

[0090] Compared with embedded edge deployment or cloud deployment, the preferred embodiment of this application, which uses local industrial control computer deployment, typically has the advantages of short data links, low system latency, less dependence on network environment, and easy real-time linkage with PLC, making it more conducive to realizing cascaded closed-loop control of "detection-judgment-feedback".

[0091] Of course, without changing the cascaded processing flow and core inventive concept of this application, adopting other deployment platform methods described above should still be regarded as optional implementation methods of the technical concept of this application.

[0092] In a specific embodiment, such as Figure 2 The diagram shows a schematic of the architecture of a cascaded industrial vision quality inspection device based on rotating target detection. The device includes the following modules cascaded together: Industrial camera image acquisition module: used to acquire images of workpieces on the vehicle production line and input the images to subsequent modules; Rotational target detection module: Receives the acquired image and outputs the rotational bounding box parameters (x,y,w,h,θ) of the workpiece to be detected through the rotational target detection model. Image correction and cropping module: Receives the rotation bounding box parameters, performs affine transformation and cropping operations on the detection region in the original image, and outputs a high-quality target image patch after correction and cropping; Downstream task module: Receives the corrected and cropped image block, performs one or more tasks such as classification, OCR text recognition, defect detection, or size measurement, and outputs the execution results; The host computer result presentation module receives the execution results from the downstream task modules, and presents them visually and provides audible and visual alarms.

[0093] The system forms a complete cascaded architecture of "image acquisition → rotation detection → image correction and cropping → downstream task processing → result presentation".

[0094] In one specific embodiment Figure 3 and Figure 4 These are schematic diagrams illustrating the detection results of a traditional horizontal bounding box detection scheme and the rotation detection scheme of this application during actual production. Figure 3 As shown, in traditional horizontal bounding box detection schemes, the detection box cannot rotate with the target, resulting in a significant deviation between the detection box and the true contour of the target; for example... Figure 4 As shown, in the rotation detection scheme provided in this application, the detection box can closely fit the actual contour of the target, thereby obtaining a cleaner and more complete target image block, and providing higher quality input for subsequent downstream tasks.

[0095] On the other hand, this application also provides a computer system, including a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute any of the above-mentioned cascaded industrial vision quality inspection methods based on rotating target detection.

[0096] For example, program code can be divided into one or more modules / units, which are stored in memory and executed by a processor to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the program code in a computer system.

[0097] The computer system can be a desktop computer, laptop, handheld computer, or cloud server, and may optionally include, but is not limited to, processors and memory. Those skilled in the art will understand that it may also include input / output devices, network access devices, buses, etc.

[0098] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0099] Memory can be an internal storage unit of a computer system, such as a hard drive or RAM. Memory can also be an external storage device of a computer system, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal and external storage units. Memory is used to store program code and other programs and data required by the computer system. Memory can also be used to temporarily store data that has been output or will be output.

[0100] The computer system described above is based on the cascaded industrial vision quality inspection method based on rotating target detection. Its technical function and beneficial effects will not be elaborated here. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0101] The above embodiments merely illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the protection scope of this invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A cascaded industrial vision quality inspection method based on rotating target detection, characterized in that, The method includes: S1, Acquire the original image containing the workpiece to be inspected; S2, input the original image into the rotating target detection model, and output the rotation bounding box parameters of the workpiece to be detected. The rotation bounding box parameters include at least the center coordinates, width, height and rotation angle. S3, based on the rotation bounding box parameters, determine the coordinates of the four vertices of the target region corresponding to the workpiece to be detected in the original image, and perform rotation mapping on the target region based on the rotation angle to obtain a rotation alignment region consistent with the main direction of the workpiece to be detected; S4. Based on the rotated bounding box parameters, the rotated aligned region is preprocessed to obtain a standardized target image patch; S5. Based on the standardized target image block, perform industrial visual quality inspection task and output industrial visual quality inspection results.

2. The method according to claim 1, characterized in that, The rotating target detection model includes a rotating feature extraction module and a rotating bounding box joint regression module; The rotation feature extraction module is used to extract rotation features and perform multi-scale fusion on the original image, and output multi-scale rotation features. The multi-scale rotation features include at least a high-resolution feature map, a medium-resolution feature map and a low-resolution feature map. The high-resolution feature map is used to represent edge details and positional information, the medium-resolution feature map is used to represent local structure and semantic information, and the low-resolution feature map is used to represent overall semantic and geometric information. The rotating bounding box joint regression module is used to perform rotating bounding box joint regression prediction based on the multi-scale rotation features and output the rotating bounding box parameters of the workpiece to be detected.

3. The method according to claim 2, characterized in that, The rotation feature extraction module includes a feature extraction unit, a geometric prior guidance unit, a multi-scale feature fusion unit, and a path enhancement unit connected in sequence. The feature extraction unit is used to perform convolutional downsampling, feature enhancement and spatial pyramid pooling processing on the original image, and output a first layer feature map, a second layer feature map and a third layer feature map, with the downsampling factor corresponding to the first layer feature map, the second layer feature map and the third layer feature map increasing sequentially. The geometric prior guidance unit is used to generate geometric prior weights based on the aspect ratio and main direction statistical results of the workpiece to be detected in the training samples. The multi-scale feature fusion unit is used to upsample the third-layer feature map based on the geometric prior weights, and then perform weighted fusion with the first-layer feature map and the second-layer feature map respectively to output preliminary fused features; The path enhancement unit is used to downsample the preliminary fusion features step by step, and then fuse the downsampled features with the features of the corresponding scale again to output the multi-scale rotation features of the workpiece to be detected.

4. The method according to claim 3, characterized in that, The geometric prior guidance unit is specifically used for: The aspect ratio and size distribution of each rotated bounding box in the training samples are statistically analyzed to obtain geometric distribution information; Based on the geometric distribution information, K-means clustering based on intersection-union distance is used to obtain multiple scale priors; The target K value is determined based on the trend of the average crossover ratio (CRO) corresponding to different K values, and a scale prior set matching the geometric features of the workpiece to be inspected is obtained. The scale prior set is matched with feature maps of different scales to generate the geometric prior weights.

5. The method according to claim 4, characterized in that, The rotating bounding box joint regression module includes a multi-scale rotation feature input unit, a bounding box parameter regression unit, a rotation angle prediction unit, and a parameter output unit connected in sequence. The multi-scale rotation feature input unit is used to perform channel mapping on the high-resolution feature map, the medium-resolution feature map and the low-resolution feature map to obtain a feature representation suitable for rotating target prediction. The bounding box parameter regression unit is used to regress the center coordinates, width, and height of the workpiece to be detected through a convolutional prediction head based on the feature representation suitable for predicting rotating targets; wherein the width and height are initialized based on a scale prior set that matches the geometric features of the workpiece to be detected and then corrected by regression. The rotation angle prediction unit is used to output the sine and cosine values ​​of the angle based on the feature representation suitable for predicting the rotation target, and to recover the rotation angle of the workpiece to be detected based on the sine and cosine values ​​of the angle. The parameter output unit is used to generate and output the rotation bounding box parameters based on the center coordinates, width, height and rotation angle.

6. The method according to claim 2, characterized in that, The rotating target detection model is optimized using a joint loss function during the training phase; The joint loss function includes the rotated bounding box regression loss, the classification loss, and the rotation angle loss; Wherein, the rotation bounding box regression loss is a regression loss based on Gaussian Wasserstein distance, the classification loss is a focus loss, and the rotation angle loss is a loss function based on angle periodic constraints.

7. The method according to claim 1, characterized in that, Step S3 includes: S31, using the center coordinates of the rotated bounding box as the origin, determine the coordinates of the four vertices of the target region based on the width and height of the rotated bounding box; S32, construct a rotation matrix based on the rotation angle, and map the coordinates of the four vertices to the original image coordinate system; S33, perform interpolation alignment processing on the target region based on the coordinates of the four mapped vertices to obtain the rotated aligned region.

8. The method according to claim 1, characterized in that, Step S4 includes: S41, calculate the reverse rotation matrix based on the rotation angle, and perform an affine transformation on the rotation alignment region to adjust the rotation alignment region to a horizontal orientation; S42, based on the width and height in the rotated bounding box parameters, the image region after posture correction is cropped to obtain a rectangular image containing only the main body of the workpiece to be detected; S43, the rectangular image is scaled to a preset size to obtain the standardized target image block.

9. The method according to any one of claims 1 to 8, characterized in that, The industrial vision quality inspection tasks include at least one of classification, optical character recognition, defect detection, and dimensional measurement.

10. A computer system, characterized in that, It includes a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute the cascaded industrial visual quality inspection method based on rotating target detection as described in any one of claims 1 to 9.

Images