Suture scalpel packaging quality detection method based on visual nerves

Abstract

The present application relates to the technical field of defect detection, in particular to a suture scalpel packaging quality detection method based on visual nerve, comprising the following steps: collecting suture scalpel packaging images for corner point detection and gradient screening, obtaining alignment parameters under a unified coordinate system, calculating optical flow displacement initial value and residual error, triggering a depth tracking model to correct displacement when exceeding the limit, cutting ROI sub-image to extract blade edge contour, calculating orthogonal projection length to generate scalpel form markers, extracting texture and shape features to input support vector machine for defect recognition, calling random forest regression model to calculate compensation parameters, and outputting quality detection results. In the present application, coordinate mapping is established through corner point detection and gradient screening, displacement is corrected by combining optical flow residual error and depth tracking, ROI cutting extracts blade edge contour, orthogonal projection quantifies form deviation, texture fusion constructs classification basis, support vector machine recognizes defects, random forest regression compensates parameters, and closed-loop feedback optimizes performance.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a method for detecting the quality of suture scalpel packaging based on visual nerves. Background Technology

[0002] The field of defect detection technology encompasses research on quality control and defect identification technologies in the packaging process of medical devices. The core focus of this field is on how to achieve timely and accurate detection of anomalies during the packaging of surgical scalpels through intelligent vision and image recognition. This technology systematically covers the configuration of image acquisition equipment, visual image preprocessing methods, feature extraction techniques, and discrimination mechanisms for different defect types. The aim is to replace traditional manual inspection methods with automated visual inspection to improve inspection efficiency and consistency. Visual neural mechanisms are gradually becoming a core component of research in this field, simulating human visual perception to enhance the ability to identify defects in complex backgrounds, and is an important direction driving the development of this field.

[0003] The suture scalpel packaging quality inspection method utilizes a biomimetic visual neural mechanism to construct an image recognition strategy specifically designed to identify quality issues arising during the packaging process. This approach addresses technical aspects such as scalpel body position offset detection, scalpel tip protection defect identification, and seal edge damage identification. Specifically, it involves acquiring multi-angle image information, combining it with a visual receptive field model to perform layered response analysis of image regions, and then using pre-defined local contrast features and edge response information to classify and identify abnormal patterns, thereby automatically detecting the packaging status of the suture scalpel. This method primarily relies on biomimetic image response mechanisms, local contrast function calculations, and edge extraction operations to perform the task of identifying multiple types of defects.

[0004] Existing technologies rely on the hierarchical response mechanism of visual receptive field models to process image regions. However, their methods for extracting local contrast features and edge response information are insufficiently adaptable to complex backgrounds. When packaging materials have texture interference or reflective areas, the hierarchical response mechanism is susceptible to high-frequency noise, leading to missed detections of blade tip protection defects. For example, a reflective area on a metal blade might be misidentified as a normal sealing edge. Feature extraction methods are limited to preset local contrast functions, making it difficult to dynamically adapt to the spatial distribution characteristics of different defect types. For instance, blade position offset detection relies on fixed thresholds and cannot automatically adjust the judgment criteria for different packaging specifications, resulting in decreased detection consistency. Image preprocessing methods do not introduce multi-angle coordinate mapping mechanisms, requiring manual calibration of alignment parameters for images from different viewpoints, increasing operational complexity and introducing human error. The discrimination mechanism lacks multi-source data fusion and compensation verification steps, and defect classification relies on a single feature response. For example, sealing edge damage detection is based solely on edge response information, ignoring texture continuity analysis, which can easily lead to misjudgments in scenarios with minor cracks in the sealing line. Furthermore, existing technologies do not integrate dynamic tracking models to correct displacement parameters. Image blurring or displacement deviation caused by robotic arm movement requires repeated detection processes, which reduces real-time performance and increases hardware load. Summary of the Invention

[0005] To address the shortcomings of existing technologies that rely on hierarchical response mechanisms based on visual receptive field models to process image regions, their methods for extracting local contrast features and edge response information are insufficiently adaptable to complex backgrounds. When packaging materials contain texture interference or reflective areas, the hierarchical response mechanism is susceptible to high-frequency noise, leading to missed detections of blade tip protection defects, such as a reflective area of ​​a metal blade being misidentified as a normal sealing edge. Feature extraction methods are limited to preset local contrast functions, making it difficult to dynamically adapt to the spatial distribution characteristics of different defect types. For example, blade position offset detection relies on fixed thresholds and cannot automatically adjust the judgment criteria for different packaging specifications, resulting in decreased detection consistency. Image preprocessing methods do not introduce multi-angle coordinate mapping mechanisms, requiring manual calibration of alignment parameters for images from different viewpoints, increasing operational complexity and introducing human error. The discrimination mechanism lacks multi-source data fusion and compensation verification, relying on single feature responses for defect classification. For example, sealing edge damage detection is based solely on edge response information, ignoring texture continuity analysis, which can easily lead to misjudgments in scenarios with minor cracks in the sealing line. Furthermore, existing technologies do not integrate dynamic tracking models to correct displacement parameters. Image blurring or displacement deviations caused by robotic arm movement require repeated detection processes, reducing real-time performance and increasing hardware load. This invention provides a method for detecting the packaging quality of suture scalpels based on visual neural networks. The technical solution is as follows:

[0006] On the one hand, a method for quality inspection of suture scalpel packaging based on visual nerves is provided, the method including:

[0007] S1: Acquire the initial image of the suture scalpel packaging, perform corner detection and gradient direction filtering, extract corner coordinates to complete pixel physical coordinate mapping, convert to a unified coordinate system, calculate feature point Euclidean offset to obtain packaging alignment parameters;

[0008] S2: Calculate the initial value of optical flow displacement based on the packaging alignment parameters, extract the predicted value to calculate the residual, determine whether the residual exceeds the limit, if it exceeds the limit, trigger the depth tracking model and correct the displacement, compare the correction result with the preset offset threshold, and output the packaging displacement anomaly information.

[0009] S3: Based on the abnormal packaging displacement information, crop the ROI sub-image, call Canny to detect and extract the cutting edge contour, calculate the offset between the orthogonal projection X-axis length and the standard cutting edge length, and generate the tool shape mark;

[0010] S4: Based on the tool shape mark, locate the cutting edge area, extract texture and shape features, calculate the contrast, energy and entropy of the gray-level co-occurrence matrix, analyze the contour parameters, and input them into the trained support vector machine classifier for defect identification, and output defect labels.

[0011] S5: Based on the defect labels, displacement anomaly information, and morphological markers, call the random forest model to calculate compensation parameters, perform confidence assessment, and encapsulate and output quality inspection results.

[0012] As a further aspect of the present invention, the corner detection and gradient direction filtering extract structural feature points with directional stability and spatial positioning accuracy, which are used as initial feature inputs into the depth tracking model and support vector machine classifier, and are fused with the extracted spatiotemporal texture features.

[0013] The depth tracking model is a three-layer convolutional neural network that extracts spatiotemporal features and generates displacement correction values.

[0014] The Canny edge detection method performs Gaussian filtering on the image to reduce noise, calculates pixel gradients to identify edge strength, filters edge pixels based on gradient magnitude, and connects edges using a double threshold method.

[0015] The gray-level co-occurrence matrix employs a multi-directional feature extraction method and undergoes gray-level quantization processing.

[0016] The support vector machine classifier uses the high-confidence texture features output by the depth tracking model as input to complete the classification and identification of tool defect types;

[0017] The alignment parameters specifically include translation amount, rotation angle, and scaling factor; the displacement anomaly information includes horizontal displacement deviation, vertical displacement deviation, and correction residual value; the tool shape markings include cutting edge curvature, cutting edge length offset, and contour symmetry; the defect labels specifically refer to crack type, notch size, and corrosion level; and the quality inspection results include compensation angle, confidence score, and packaging level.

[0018] As a further aspect of the present invention, the specific steps of S1 include:

[0019] S101: Acquire the initial image of the suture scalpel packaging, use Harris corner detection to traverse the pixel region, calculate the gray-level change rate, construct an autocorrelation matrix to calculate the gray-level response value of the corner, filter candidate corners according to the set threshold, call the Sobel operator to extract the gradient direction, calculate the cosine similarity with the preset angle interval, filter corners with similarity higher than the tolerance threshold, and generate a set of corner coordinates.

[0020] The tolerance threshold is a preset allowable deviation range for the gradient direction, which is determined by statistically analyzing the gradient direction dispersion of effective corner points in multiple sets of samples.

[0021] S102: Based on the set of corner coordinates, call the camera intrinsic parameters and distortion coefficients to perform distortion correction, transform to the robot arm base coordinate system through homogeneous coordinate transformation, and superimpose the end pose transformation matrix to generate a physical coordinate set.

[0022] S103: Call the continuous frame coordinates of the same feature point in the physical coordinate set, calculate the three-axis Euclidean offset to construct the least squares residual matrix, filter the offset samples within the trajectory error tolerance, fit the translation vector and rotation matrix, map the feature point Euclidean offset to a unified coordinate system, dynamically correct the initial offset estimate, and generate a packaging alignment parameter set.

[0023] The trajectory error tolerance is set based on the repeatability accuracy index of the robotic arm, constraining the maximum allowable value of the offset deviating from the theoretical trajectory.

[0024] As a further aspect of the present invention, the calculation of the corner grayscale response value adopts the formula:

[0025]

[0026] Among them, R i This represents the grayscale response value of the i-th pixel. This represents the Sobel gradient value in the X direction of the i-th pixel during the j-th window sampling. This represents the Sobel gradient value in the Y direction of the i-th pixel during the j-th window sampling. Represents all pixels within the window that are of the i-th pixel. The average value, Represents all pixels within the window that are of the i-th pixel. The average value of ε is a constant to avoid the smallest positive number with a denominator of zero, and n represents the number of pixels in a 3×3 window.

[0027] As a further aspect of the present invention, the specific steps of S2 include:

[0028] S201: Based on the packaging alignment parameters, select matching feature points in adjacent frames, construct a gray-level gradient spatiotemporal matrix, obtain the motion vector by least squares solution, extract the horizontal and vertical components as the predicted displacement, subtract each item from the model reference displacement value, calculate the absolute value of the residual, and generate a residual vector set.

[0029] The model reference displacement value is compared with the actual calculated motion vector components to evaluate the accuracy of the motion estimation;

[0030] S202: Call the residual vector set, calculate the residual mean and standard deviation, set the residual screening threshold to a multiple of the standard deviation, screen the image coordinates corresponding to the out-of-limit residuals, extract the spatiotemporal features of the image blocks, fuse optical flow information and model reference displacement values, and generate the corrected displacement.

[0031] S203: Based on the corrected displacement, calculate the difference between the length of each displacement vector and the preset offset threshold, count the number of vectors with an absolute value greater than zero, compare the proportion with the preset anomaly judgment threshold, determine whether the proportion exceeds the threshold, and generate displacement anomaly information.

[0032] The anomaly detection threshold is determined based on the proportional relationship with the offset threshold.

[0033] As a further aspect of the present invention, the specific steps of S3 include:

[0034] S301: Based on the abnormal coordinates recorded in the displacement anomaly information, traverse the X-axis and Y-axis sets, extract the extreme values ​​to calculate the side length of the rectangle surrounding the anomaly point, call the bilinear interpolation algorithm to smooth the boundary coordinates, and generate the ROI sub-image;

[0035] S302: Call the ROI sub-image, calculate the gray-level gradient magnitude of the pixels in the horizontal and vertical directions and merge them into an overall gradient map, calculate the gradient magnitude, count the frequency by binning, form an amplitude histogram, set high and low thresholds, filter pixels within the threshold range, connect adjacent pixels to form a closed curve, remove short segments, and obtain the coordinate set of the blade edge.

[0036] S303: Based on the X-axis projection direction of the edge coordinates, the frequency density of the coordinate points is statistically analyzed, the endpoints of the interval of the threshold number of consecutive effective coordinate points in the frequency density distribution are extracted, the endpoint spacing is calculated as the actual blade length, the difference is calculated with the standard blade length parameter, the absolute value is taken as the blade length offset, the offset is compared with the preset blade length tolerance threshold, if it exceeds the threshold, it is marked as abnormal shape, and a tool shape mark is generated.

[0037] The effective coordinate point threshold is calculated based on the minimum number of consecutive effective points according to the tolerance range of the tool design dimensions;

[0038] The blade length tolerance threshold is set according to the tool wear limit value in the equipment maintenance manual.

[0039] As a further aspect of the present invention, the specific steps of S4 include:

[0040] S401: Based on the coordinates of the abnormal area marked in the tool shape mark, traverse all X / Y axis coordinate points in the image coordinate system, filter the coordinate set marked as the specified abnormal category, calculate the range of the coordinate set as the boundary of the rectangular area, construct the texture analysis area with abnormality, replace the noise pixels with the mean of the neighboring pixels, and generate the cutting edge positioning mask.

[0041] S402: Call the pixel data within the blade positioning mask, statistically analyze the gray-level co-occurrence matrix of pixels at a set interval threshold along the set angle threshold direction, calculate the horizontal contrast as the weighted sum of the squares of the gray-level differences between adjacent pixels, simultaneously extract the aspect ratio of the minimum bounding rectangle of the contour point set, and generate a feature vector set.

[0042] S403: Arrange the feature vector set in the order of samples, input it into the pre-trained support vector machine classifier, call the radial basis kernel function to calculate the feature space distance, determine the sample category according to the sign of the decision hyperplane, and output the defect label.

[0043] As a further aspect of the present invention, the radial basis kernel function is used to calculate the feature space distance, employing the formula:

[0044]

[0045] Where x represents a sample point in the input space, x′ represents another sample point in the input space, K(x, x′) represents the kernel function value between points x and x′, σ represents the bandwidth parameter of the RBF kernel function, and the unit of σ is mm. σ is adaptively optimized according to the spatial distribution law of the tool edge defect texture features. By statistically analyzing the edge gradient amplitude in the defect texture image, the kernel function bandwidth that matches the actual edge shape is determined. The unit conversion is based on the image resolution of the tool inspection equipment to ensure that the kernel function bandwidth is consistent with the size of the tool entity features.

[0046] As a further aspect of the present invention, the specific steps of S5 include:

[0047] S501: Based on the defect label, displacement anomaly information and tool shape mark, the defect type is encoded as a numerical vector, the weighted statistical features of the offset error are extracted, and the shape mark level information is fused to construct a three-dimensional feature combination. The random forest model is called to perform parallel prediction and generate the predicted value of the compensation parameter.

[0048] The predicted values ​​of the compensation parameters are the corrected tool parameter estimates output by the random forest model;

[0049] S502: Based on the predicted values ​​of the compensation parameters, the independent prediction results of the decision trees in the random forest model are statistically analyzed, the standard deviation of the fluctuation of the prediction results is calculated, and the out-of-bag error rate in the random forest training process is extracted and weighted linearly combined to generate the confidence coefficient.

[0050] The confidence coefficient is an indicator that measures the reliability of the prediction results of the compensation parameters;

[0051] The out-of-bag error rate is the average error rate generated by the random forest model during the training phase on data samples that did not participate in tree building.

[0052] S503: The confidence coefficient and the predicted value of the compensation parameter are paired and encapsulated. The allowable limit of the compensation deviation is used as the judgment threshold. The confidence coefficient and the limit are compared. If the difference is less than zero, a verification mark field is added to generate the quality inspection result.

[0053] The allowable limit for compensation deviation is set based on the tolerance range of the tool package and its operating conditions, and is determined by evaluating the tool manufacturing precision requirements and the error tolerance in the actual machining environment.

[0054] As a further aspect of the present invention, the confidence coefficient is expressed by the formula:

[0055]

[0056] Where C represents the confidence coefficient, q represents the number of independent decision trees in the random forest model, and δ s This represents the difference between the predicted value of the s-th decision tree and the predicted value of the compensation parameter, in pixels. represents the mean of all decision tree discrepancies, in pixels, and ∈ represents the out-of-bag error rate in the random forest training records.

[0057] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0058] Pixel physical coordinate mapping is established through corner detection and gradient direction filtering, unifying the coordinate system of multi-angle images, eliminating the impact of viewpoint differences on feature extraction, and improving alignment accuracy and data consistency. A residual judgment mechanism based on initial optical flow displacement values, combined with a depth tracking model, corrects offsets and dynamically adjusts tracking parameters to adapt to complex motion scenarios, enhancing the robustness of displacement anomaly detection and reducing the probability of misjudgments caused by mechanical vibration or packaging material deformation. ROI sub-image cropping focuses on the cutting edge region, combined with Canny detection and orthogonal projection length calculation, quantifying tool shape deviations, reducing background noise interference, and improving the efficiency of cutting edge contour analysis. Texture feature extraction integrates gray-level co-occurrence matrix and contour parameters, fusing multi-dimensional features to construct classification criteria, optimizing defect characterization capabilities. A support vector machine classifier utilizes a high-dimensional feature space to delineate decision boundaries, improving the accuracy of identifying damaged sealing edges and missing tool tips. A random forest regression model calculates compensation parameters based on multi-source data, combined with a confidence evaluation mechanism to balance the reliability of detection results, achieving adaptive optimization of quality judgment. The above process forms a closed-loop feedback system, introducing collaborative optimization strategies in the image registration, dynamic tracking, feature fusion and result verification stages to comprehensively improve the generalization ability and fault tolerance of detection, effectively cope with interference factors such as light fluctuations, material reflection and mechanical vibration during packaging, and ensure that the defect identification coverage and false detection suppression rate are balanced. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the workflow of the present invention. Detailed Implementation

[0060] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0061] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0062] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0063] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0064] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0065] Please see Figure 1 This invention provides a method for detecting the packaging quality of suture scalpels based on visual nerves. The processing flow of this method may include the following steps:

[0066] S1: Acquire the initial image of the suture scalpel packaging, perform corner detection and gradient direction filtering, extract corner coordinates to complete pixel physical coordinate mapping, convert to a unified coordinate system, calculate feature point Euclidean offset to obtain packaging alignment parameters;

[0067] S2: Calculate the initial value of optical flow displacement based on packaging alignment parameters, extract the predicted value to calculate the residual, determine whether the residual exceeds the limit, if it exceeds the limit, trigger the depth tracking model and correct the displacement, compare the correction result with the preset offset threshold, and output packaging displacement anomaly information.

[0068] S3: Based on the abnormal packaging displacement information, crop the ROI sub-image, call Canny to detect and extract the cutting edge contour, calculate the offset between the orthogonal projection X-axis length and the standard cutting edge length, and generate tool shape markings;

[0069] S4: Based on the tool shape marker, locate the cutting edge area, extract texture and shape features, calculate the contrast, energy and entropy of the gray-level co-occurrence matrix, analyze the contour parameters, and input them into the trained support vector machine classifier for defect identification, and output defect labels;

[0070] S5: Based on the defect label, displacement anomaly information, and morphological markers, call the random forest model to calculate compensation parameters, perform confidence assessment, and encapsulate and output quality inspection results.

[0071] The alignment parameters are specifically translation, rotation angle, and scaling factor. Displacement anomaly information includes horizontal displacement deviation, vertical displacement deviation, and correction residual value. Tool shape markings include cutting edge curvature, cutting edge length offset, and contour symmetry. Defect labels specifically refer to crack type, notch size, and corrosion level. Quality inspection results include compensation angle, confidence score, and packaging level.

[0072] Specifically, such as Figure 1 As shown, the specific steps of S1 are as follows:

[0073] S101: Acquire the initial image of the suture scalpel packaging, use Harris corner detection to traverse the pixel region, calculate the gray-level change rate, construct an autocorrelation matrix to calculate the gray-level response value of the corner, filter candidate corners according to the set threshold, call the Sobel operator to extract the gradient direction, calculate the cosine similarity with the preset angle interval, filter corners with similarity higher than the tolerance threshold, and generate a set of corner coordinates.

[0074] The tolerance threshold is a preset allowable deviation range for the gradient direction, which is determined by statistically analyzing the gradient direction dispersion of effective corner points in multiple sets of samples.

[0075] Images of the suture scalpel packaging were captured using a CMOS industrial camera, performing single-frame image capture. After grayscale processing, a pixel-by-pixel traversal operation was performed, with a 3×3 basic sliding window area set. Centered on the current pixel, the horizontal and vertical gradient values ​​G of all pixels in the window under Sobel convolution were calculated. xij G yij The average gradient of the i-th point of the center pixel in the X and Y directions within the window is denoted as u. xi u yi The grayscale response value of the corner points is calculated using the following formula:

[0076]

[0077] Among them, R i This represents the grayscale response value of the i-th pixel. This represents the Sobel gradient value in the X direction of the i-th pixel during the j-th window sampling. This represents the Sobel gradient value in the Y direction of the i-th pixel during the j-th window sampling. Represents all pixels within the window that are of the i-th pixel. The average value, Represents all pixels within the window that are of the i-th pixel. The average value, ε is a constant to avoid the smallest positive number with a denominator of zero, n represents the number of pixels in a 3×3 window, ε = 10 -5 This is used to avoid the problem of the denominator being zero. The sample example uses:

[0078] The sum of the products is:

[0079]

[0080] Denominator calculation:

[0081] ∑(G xij -u xi ) 2 =44.67,∑(G yij -u yi )2 =36.67,

[0082]

[0083] The final response value is:

[0084]

[0085] To determine the screening threshold R for corner response values th R i The criteria for distinguishing between valid and invalid data were established. The study used 500 images of suture scalpel packaging as the statistical basis. For each image, the corner response value R of candidate pixels selected through Harris corner detection was calculated. i The calculations are performed, and the numerical values ​​are aggregated into a response value distribution sequence {R1, R2, ..., R...}. m}, where m represents 100–300 corner points in each image, and the total number of samples is greater than 50,000.

[0086] Next, the distribution of all corner response values ​​is statistically analyzed, yielding:

[0087] Mean: μ R =2.8;

[0088] Standard deviation: σ R =0.58;

[0089] The distribution range of response values ​​is concentrated in the interval [1.2, 4.0]. To ensure that the selection of effective corner points has a certain degree of intensity recognition, and to avoid introducing background pseudo-corner points due to retaining too many weak responses, the threshold is set in the following way:

[0090] R th =μ R -0.5·σ R =2.8 - 0.5 * 0.58 = 2.8 - 0.29 = 2.51;

[0091] Rounding the result and truncating it down to a conservative value, we get:

[0092] R th =2.5;

[0093] This setting ensures that approximately 70–75% of the effective strong corner points can be retained, while 25–30% of the weak corner points in the low-response range at the tail are removed. This range exhibits high background error and directional instability in experimental statistics, and removing it can improve the overall screening accuracy.

[0094] Table 1. Statistical Sample Table of Corner Response Value Distribution

[0095]

[0096]

[0097] As shown in Table 1, this threshold setting method combines the statistical mean and standard deviation, which not only conforms to the discrete characteristics of the response value in the real image scene, but also effectively controls the corner retention range.

[0098] Based on the set filtering threshold R th =2.5, if R i >R th If the value is not found, the point is retained as a candidate corner point. Then, cosine similarity is used to determine if the direction matches.

[0099] cos(Δθ)=cos(θ i -θ0),

[0100] Where θ i For the corner direction, θ0 = 45°, the angle tolerance is 5°, and the tolerance threshold is cos(5°)≈0.996. If the cosine of the corner direction angle is higher than this threshold, it will be further retained, and finally the corner coordinate set will be formed.

[0101] S102: Based on the set of corner coordinates, the camera intrinsic parameters and distortion coefficients are called to perform distortion correction. The coordinates are then transformed to the robot arm base coordinate system through homogeneous coordinate transformation. Finally, the end pose transformation matrix is ​​superimposed to generate a set of physical coordinates.

[0102] Based on the preserved corner point set, by calling the camera intrinsic parameter matrix:

[0103]

[0104] Distortion coefficients {k1 = -0.223, k2 = 0.150, p1 = 0.0009, p2 = 0.0012, k3 = -0.08} are used to perform distortion correction for each corner point. The pixel point (u, v) = (128, 314) is normalized to the camera coordinate system.

[0105]

[0106] Introducing a depth parameter λ = 500mm, we obtain the three-dimensional points:

[0107]

[0108] Using extrinsic transformation matrix:

[0109]

[0110] Convert the camera coordinates to the robot arm's base coordinates, and combine them with the end effector's attitude matrix T. end The overlay process is performed, and the final calculated physical coordinates of the corner points are obtained, for example:

[0111] P world =[-80.12mm, 156.24mm, 100.5mm],

[0112] After transforming all corner points in sequence, a complete set of physical space coordinates is generated.

[0113] S103: Call the continuous frame coordinates of the same feature point in the physical coordinate set, calculate the three-axis Euclidean offset to construct the least squares residual matrix, filter the offset samples within the trajectory error tolerance, fit the translation vector and rotation matrix, map the feature point Euclidean offset to a unified coordinate system, dynamically correct the initial offset estimate, and generate the packaging alignment parameter set.

[0114] The trajectory error tolerance is set based on the repeatability accuracy index of the robotic arm, constraining the maximum allowable value of the offset from the theoretical trajectory.

[0115] Retrieve the spatial coordinates of the same feature point in a series of frames from the physical coordinate set. Let the sampled values ​​of the consecutive frames be:

[0116] P1=[100.12, 150.32, 200.45], P2=[100.10, 150.34, 200.42],

[0117] P3=[100.11, 150.33, 200.43];

[0118] Calculate its three-axis Euclidean offset:

[0119] Δx=x t+1 -x t Δy=y t+1 -y t Δz=z t+1 -z t ;

[0120] Corresponding inter-frame residuals:

[0121] ||ΔP1||=0.036mm, ||ΔP2||=0.017mm, ||ΔP3||=0.012mm;

[0122] The trajectory error tolerance is set to δ = 0.02 mm, and samples satisfying ||ΔP are selected. t Sample points with ||<δ are used to fit the rigid body transformation and construct the least squares residual function:

[0123]

[0124] Finally, the rotation matrix R is obtained. align With translation vector t align This constitutes the package alignment parameter set:

[0125] {R align ,t align};

[0126] Specifically, such as Figure 1 As shown, the specific steps of S2 are as follows:

[0127] S201: Based on the packaging alignment parameters, select matching feature points in adjacent frames, construct the gray-level gradient spatiotemporal matrix, obtain the motion vector by least squares solution, extract the horizontal and vertical components as the predicted displacement, calculate the difference with the model reference displacement value for each item, calculate the absolute value of the residual, and generate a residual vector set.

[0128] The model reference displacement value is compared with the actual calculated motion vector components to evaluate the accuracy of the motion estimation;

[0129] Based on the packing alignment parameters, a multi-channel fusion tensor containing image grayscale, edge features, and color contrast is first constructed in each pair of image frames. During feature extraction, in addition to the response corner points directly obtained based on grayscale gradients, a high-dimensional texture description vector is supplemented for each feature point. This description vector is extracted by inputting an image patch into an encoding network consisting of multiple convolutional layers and feature enhancement modules. This network takes a local window of the image as input and outputs the embedded features around the feature point. The embedded features of all feature points in the current frame are paired with those in the previous frame to calculate feature similarity. Specifically, the L2 distance of each pair of feature points is calculated. And set the matching threshold ∈ = 0.15, if d ij If the feature points in the two frames are considered to have matched successfully, a feature matching set after high-dimensional filtering is constructed. Then, the image gradient matrix A and gray-level difference matrix b are constructed using the matching point pairs, and the linear system is solved. Obtain the predicted motion vector Then, the six-degree-of-freedom motion data of the robotic arm's end effector is retrieved, and based on the feedback of the six-axis angle values ​​[32°, 47°, 59°, 13°, 11°, 4°], it is input into the inverse kinematics transformation function f. -1 The theoretical displacement (Δx, Δy) = (2.0, 1.0) within the current period is calculated, and the difference between this displacement and the predicted vector is used to obtain the residual vector. The absolute values ​​of each vector are then used to construct a set of residual vectors for subsequent depth vector re-discrimination.

[0130] S202: Call the residual vector set, calculate the residual mean and standard deviation, set the residual screening threshold to a multiple of the standard deviation, screen the image coordinates corresponding to the out-of-limit residuals, extract the spatiotemporal features of the image blocks, fuse optical flow information and model reference displacement values, and generate the corrected displacement.

[0131] Based on the existing residual vector set, the displacement history of all feature points in consecutive image frames is first... Combined into a three-dimensional time series, the input structure is constructed. Each sequence corresponds to the predicted displacement record of a feature point across three consecutive frames; to further determine whether the residual is caused by interference such as temporary occlusion or reflection from packaging materials, The input is fed into a lightweight recursive structure model, which internally employs a dual-channel time update mechanism to calculate the output sequence residual trend vector through state units. Used to replace the original residual values; based on the model's output vector, the mean and standard deviation μ are then calculated in the horizontal and vertical directions, respectively. x ′,σ x ′, to set the filtering threshold τ x ′=2·σ x ′;

[0132] The residuals at each point are sequentially checked for exceeding the limit. If the residual at any point exceeds the limit in any direction, it is considered abnormal. The feature point image block is then unfolded into a grayscale temporal block along the time dimension. Optical flow analysis is performed on this block to obtain the optical flow vector and the reference displacement value (r). x ,r y The mixed generation corrected displacement is given by the following formula:

[0133]

[0134] Taking point P3 as an example, let... r x =2.0, r y =1.0, after correction we get (1.18, 0.16), and integrate all the corrected displacements to form a set V. corr This is used for the next step of anomaly statistics.

[0135] S203: Based on the corrected displacement, calculate the difference between the length of each displacement vector and the preset offset threshold, count the number of vectors with an absolute value of the difference greater than zero, compare the proportion with the preset anomaly judgment threshold, determine whether the proportion exceeds the threshold, and generate displacement anomaly information.

[0136] The anomaly detection threshold is determined based on the proportional relationship with the offset threshold.

[0137] For set V corr For each correction vector in the equation, first calculate the magnitude:

[0138]

[0139] Compared with the preset offset threshold δ=1.0, the preset offset threshold of δ=1.0 is optimized based on actual data: the experiment used a dataset containing 12,000 frames of images (annotated with 3,850 outlier points and 85,200 normal points). Analysis revealed that the spatial offset of normal points was concentrated in the range of 0.2 to 0.8, while the offset of outlier points generally exceeded 1.2. To balance detection sensitivity and false alarm rate, the threshold was dynamically optimized using gradient descent: starting from an initial value of 0.5, after 150 iterations, the final threshold stabilized at 1. The optimized test results showed that the system's capture rate of real anomalies increased to 92.3%, while reducing the false alarm rate to 5.8%, a significant improvement in overall performance compared to the threshold (0.8) set by human experience. In the face of interference scenarios such as sudden changes in lighting and mechanical vibration, the false alarm rate fluctuated within a stable range of 5.2% to 6.5%, a reduction of about 40% compared to before optimization. The difference ΔL was obtained. i =L i -δ, if the difference is positive, the point is judged as abnormal; to eliminate structural disturbances, such as multi-point misjudgments caused by periodic slippage of tools or packaging edges, the correction vector (v) is further adjusted. corr,x v corr,y The vectors from the previous three frames are input together into the anomaly detection network structure. This network uses a spatial attention mechanism to extract whether there is a high-density anomaly clustering pattern and outputs an anomaly confidence score s for each point. i ∈[0,1], set the confidence threshold θ=0.7, if s i >θ and ΔL i If the value is greater than 0, it is finally confirmed as an anomaly. The total number of anomalies m and the total number of points n are counted to obtain the ratio p = m / n. This ratio is compared with the threshold γ = 0.3 to determine frame-level anomalies. If p > 0.3, the frame is marked as having abnormal motion. The following table shows the core data of the entire process:

[0140] Table 2. Anomaly Detection Table for Fusion Deep Features and Correction Vectors

[0141]

[0142]

[0143] As shown in Table 2, by combining the residual correction modulus and confidence level, three points were finally identified as anomalies, accounting for 50%, which exceeded the set threshold of 30%, thus determining that the current image frame had a displacement anomaly. All depth path processing in the model takes the image feature sequence as input and participates in the three stages of residual screening, optical flow vector fusion, and anomaly classification, ensuring that the depth tracking structure is naturally embedded in the original scheme.

[0144] Specifically, such as Figure 1 As shown, the specific steps of S3 are as follows:

[0145] S301: Based on the abnormal coordinates recorded in the displacement anomaly information, traverse the X-axis and Y-axis sets, extract the extreme values ​​to calculate the side length of the rectangle surrounding the anomaly point, call the bilinear interpolation algorithm to smooth the boundary coordinates, and generate the ROI sub-image;

[0146] Based on the anomaly coordinates recorded in the displacement anomaly information, we first extract the coordinate sets of all anomaly points in the X and Y axes, denoted as {x1, x2, ..., x...} n} and {y1, y2, ..., y n The algorithm iterates through the two sets of coordinate values, calling the maximum and minimum value functions to calculate the extreme values ​​in the set. For example, given the set of outlier X coordinates as [123, 180, 240, 305, 360, 420] and the set of Y coordinates as [88, 92, 95, 99, 104, 110], the starting coordinates of the rectangle boundary are the top left corner (x, y). min y min ) = (123, 88), with the bottom right corner being (x max y max ) = (420, 110), based on this, the side length of the rectangle is calculated to be l. x =x max -x min =297 pixels, l y =y max -y min =22 pixels; To improve the continuity of the boundary and the accuracy of subsequent image block segmentation, the image region edge point set is called on this basis, and boundary coordinate points are inserted between the four corners of the rectangular boundary. Bilinear interpolation is performed on the value between two adjacent pixels on each side, the gray value of each interpolation point is calculated and the boundary is extended to the direction of adjacent pixels, and finally a smooth and closed ROI boundary contour is formed. The corresponding region image is cropped from the original image according to the coordinate range to form the ROI sub-image.

[0147] S302: Call the ROI sub-image, calculate the gray-level gradient magnitude of the pixels in the horizontal and vertical directions and merge them into an overall gradient map, calculate the gradient magnitude, count the frequency of binning, form an amplitude histogram, set high and low thresholds, filter pixels within the threshold range, connect adjacent pixels to form a closed curve, remove short segments, and obtain the coordinate set of the blade edge.

[0148] After calling the generated ROI sub-image, scan each pixel in the sub-image matrix pixel by pixel, and calculate the gray-level gradient value ΔG in the horizontal (x) and vertical (y) directions based on the gray-level difference between the current pixel and its adjacent pixels above, below, left, and right. x ΔG y Then, the gradient magnitudes in these two directions are summed by squares and the square root is taken to obtain the overall gradient magnitude of the pixel. A grayscale histogram is formed by statistically analyzing the gradient magnitudes of all pixels in the entire image. High and low thresholds are then set for this histogram based on pixel distribution; for example, the 15th percentile is chosen as the low threshold, denoted as T. l =20, the 85th percentile is used as the high threshold, denoted as T. h =28, sequentially determine whether the gradient value of each pixel is located within [T l T h Within the interval, if the condition is met, it is determined to be a significant edge point; then, adjacent pixels that meet the condition are grouped by a connection operation, and the length of each group of pixel chains is detected. If the chain length is lower than the minimum effective edge length (e.g., set to 8 pixels), it is discarded, and only pixel chains with the length meeting the condition are retained. The resulting closed curve contour is constructed, and the pixel positions on all chain edges are extracted in the current image coordinate system to form the edge coordinate set.

[0149] S303: Based on the X-axis projection direction of the cutting edge coordinates, the frequency density of the coordinate points is statistically analyzed, the endpoints of the interval of the threshold number of consecutive effective coordinate points in the frequency density distribution are extracted, the endpoint spacing is calculated as the actual cutting length, the difference is calculated with the standard cutting length parameter, the absolute value is taken as the cutting length offset, the offset is compared with the preset cutting length tolerance threshold, and if it exceeds the threshold, it is marked as an abnormal shape and a tool shape mark is generated.

[0150] The effective coordinate point threshold is calculated based on the allowable tolerance range of the tool design dimensions, specifying the minimum number of consecutive effective points.

[0151] The cutting edge length tolerance threshold is set according to the tool wear limit value in the equipment maintenance manual.

[0152] Based on the extracted edge coordinate set, the coordinates of all edge points are first projected along the X-axis, i.e., the number of Y coordinates corresponding to each set of X coordinate values ​​is counted to form a frequency density array {f(x)}. This frequency array is then traversed, and for any consecutive position, f(x) ≥ N. min The left and right endpoints of the segment are extracted as the effective blade segment range, where N min This is the threshold for the number of effective coordinate points. This value is calculated backward from the tool tolerance range. For example, if a standard tool has a designed cutting edge length of 31.7 pixels, the corresponding resolution is 8 pixels per mm. Allowing for ±1 mm of wear, the minimum continuous length is 8 × 1 = 8 pixels. To improve stability, the threshold is set to 10 points, i.e., N. min =10; For each frequency density curve segment, determine the continuous segments that meet the conditions. If the conditions are met within a certain X interval, extract its left and right endpoints x. l x r Calculate the distance as L = x r -x lThis is the actual length of the current cutting edge of the tool; for example, the start and end points of the continuous frequency density segment in the region where P3 is located are x l =208.6 and x r =241.0, then the blade length is L = 32.4 pixels. Call the system-defined standard blade length L. s =31.7 pixels, perform interpolation calculation to obtain the blade length offset d L =|LL s |=|32.4-31.7|=0.7 pixels, then compare this value with the blade length tolerance threshold δ L =0.5 pixels for comparison and judgment, if d L >δ L If the blade segment is not found to be abnormal, it is marked as a morphologically abnormal point. This judgment logic is applied to all valid feature points. According to statistics, only the blade length deviation of P3 exceeds the tolerance value, and the rest of the feature points are qualified points.

[0153] Table 3 Calculation Table for Cutting Edge Morphology Detection

[0154]

[0155] As shown in Table 3, P3 is the only one whose blade length offset is greater than the tolerance (0.7>0.5), and is therefore judged as abnormal.

[0156] Specifically, such as Figure 1 As shown, the specific steps of S4 are as follows:

[0157] S401: Based on the coordinates of the abnormal area marked in the tool shape mark, traverse all X / Y axis coordinate points in the image coordinate system, filter the coordinate set marked as the specified abnormal category, calculate the range of the coordinate set as the boundary of the rectangular area, construct the texture analysis area with abnormality, replace the noise pixels with the mean of the neighboring pixels, and generate the cutting edge positioning mask.

[0158] Based on the abnormal region coordinate information stored in the tool shape markings, the set of coordinate points of image blocks marked as defects is extracted one by one, and the corresponding abnormal number and category label are recorded. A double nested traversal operation is performed on the X-axis and Y-axis positions of all pixels in the image coordinate system, sequentially determining whether each pixel falls within the coordinate range corresponding to the current specified abnormal number. If its X-coordinate satisfies x min ≤x i ≤x max The Y coordinate satisfies y min ≤y i ≤y maxIf the category label corresponds to the current processing anomaly number, then the pixel is added to the current region processing set. After recording all coordinate points, the difference between the maximum and minimum values ​​in the X and Y directions is calculated as the pixel range in the horizontal and vertical directions, respectively. For example, in region E1, the coordinate range is x∈[210,240] and y∈[90,115]. The calculated pixel range is 30 for X and 25 for Y. Based on this, the 3×3 neighborhood grayscale matrix of all pixels in the region is called sequentially. After removing the center pixel value, the arithmetic mean of the remaining 8 pixels is taken. For example, if the neighborhood grayscale value of a certain point is [112, 115, 116, 117, 110, 114, 113, 118], then the mean is 114.375≈114. This mean is used to replace the original center value. This operation is performed sequentially on each pixel in the anomaly region to form a continuous, smooth, and noise-free binary mask image as the blade location region for further texture analysis.

[0159] S402: Call the pixel data within the blade positioning mask, statistically analyze the gray-level co-occurrence matrix of pixels at a set interval threshold along the set angle threshold direction, calculate the horizontal contrast as the weighted sum of the squares of the gray-level differences between adjacent pixels, simultaneously extract the aspect ratio of the minimum bounding rectangle of the contour point set, and generate a feature vector set.

[0160] In the generated mask image, the pixel block corresponding to each anomaly number is selected, and a gray-level co-occurrence matrix statistical operation is performed. The gray-level direction is set to 0°, 45°, and 90° to cover the horizontal, diagonal, and vertical directions of the image. The pixel interval for each direction is set to 2 pixels. This value is determined jointly based on the image spatial resolution and the target area size. In the actual image, the spatial resolution is 0.125 mm / pixel, and the target area is approximately 25–35 pixels wide (approximately 3–4 mm). Taking an interval d = 2 pixels gives a spatial perception range of 0.25 mm. Within this range, the local texture structure can be stably reflected while avoiding single-pixel-level texture interference. In actual system testing, when d > 3, the texture response is significantly weakened, and the contrast decreases by more than 40%. Subsequently, for each pair of reference pixels and target pixels in each direction, a gray-level pair (g) is formed. i g j The frequencies of all gray-level combinations are cumulatively counted, and a 16-level gray-level co-occurrence matrix P(i,j) is constructed. The horizontal contrast value is then calculated using the following formula:

[0161]

[0162] Where C represents the contrast value of the gray-level co-occurrence matrix at a specified angle, which is a statistical measure of the drastic change in gray level in an image region; i represents the gray level of the reference pixel corresponding to the row index in the gray-level co-occurrence matrix, with a value ranging from 0 to 15 (when the gray-level co-occurrence matrix has 16 levels); j represents the gray level of the target pixel corresponding to the column index in the gray-level co-occurrence matrix, also with a value ranging from 0 to 15; (ij) 2 P(i,j) represents the square of the difference between the gray value of the reference pixel and the gray value of the corresponding target pixel, which is used to reflect the magnitude of the gray value difference between pixels. P(i,j) represents the normalized probability frequency of the pixel pair with gray value i and gray value j in the whole image or the specified mask area, that is, the proportion of the element in the i-th row and j-th column of the co-occurrence matrix in the total number of co-occurrence pairs.

[0163] For example, the gray-level combination frequency in region E1 is mainly concentrated near the diagonal. If P(8, 9) = 0.10, P(9, 10) = 0.15, and P(10, 11) = 0.12, its contrast contribution is 0.10 + 0.15 + 0.12 = 0.37. Adding a more distant pair such as (4, 12), P = 0.08, the overall contrast value is calculated to be 18.2. Then, the contour edge point set is extracted from the mask image, and the minimum bounding rectangle contour fitting method is called to calculate the length and width of its envelope rectangle. For example, the fitting width of region E1 is 24, the height is 12, and the aspect ratio is r = 24 / 12 = 2.0. Combined with the contrast value, they form the feature vector of the current abnormal region, such as [18.2, 2.0]. This process is performed synchronously on all abnormal numbers, and the final feature vector set is shown in Table 4.

[0164] Table 4. Anomaly Region Masking and Texture Feature Extraction Table

[0165]

[0166] As shown in Table 4, the E2 region has both high contrast and high aspect ratio, indicating that its edge contour fluctuates drastically.

[0167] S403: Arrange the feature vector set in sample order, input it into the pre-trained support vector machine classifier, call the radial basis kernel function to calculate the feature space distance, determine the sample category based on the sign of the decision hyperplane, and output the defect label.

[0168] Arrange the feature vectors corresponding to the anomaly numbers in the table in numerical order to construct the input sample matrix. Each feature vector consists of two dimensions: the contrast value of the gray-level co-occurrence matrix and the aspect ratio of the minimum bounding rectangle of the region's contour. For example, the feature vector for E1 is [18.2, 2.0], for E2 it is [21.7, 2.8], and for E3 it is [17.5, 2.0]. Combine all feature vectors to form a two-dimensional sample matrix X∈R. 3×2This sample set serves as the input to the classification model. It is fed into a pre-trained support vector machine classifier for defect type identification. This classifier uses a radial basis function kernel, the form of which is:

[0169]

[0170] Where x represents a sample point in the input space, x′ represents another sample point in the input space, K(x, x′) represents the kernel function value between points x and x′, σ represents the bandwidth parameter of the RBF kernel function, and the unit of σ is mm. σ is adaptively optimized according to the spatial distribution law of the tool edge defect texture features. By statistically analyzing the edge gradient magnitude in the defect texture image, the kernel function bandwidth that matches the actual edge shape is determined. The unit conversion is based on the image resolution of the tool detection equipment to ensure that the kernel function bandwidth is consistent with the size of the tool entity features. Taking the E2 input vector [21.7, 2.8] and a certain support vector [20.5, 2.5] as an example, the calculation is as follows:

[0171] ||xx′|| 2 = (21.7 - 20.5) 2 +(2.8-2.5) 2 =1.44 + 0.09 = 1.53,

[0172] If the kernel function parameter σ = 1, then the kernel value is:

[0173]

[0174] The kernel value and the weight coefficients a of the support vectors i Its category label y i After multiplication, the kernel response values ​​of all support vectors are summed and the bias term b is added to obtain the classification decision function value:

[0175]

[0176] Among them, a i The weight coefficients of the support vectors, y i 'b' represents the class label of the support vector, and 'b' is the bias term.

[0177] If the value is positive, the input sample is identified as a defective region; if the value is negative, it is identified as a normal region. This identification process does not require adjustment of the original features; it only determines the region based on its relative position within the feature boundaries constructed from the support vectors. Where a i = 0.6 (assuming the support vector is a typical value for non-boundary samples), y i= +1 (defect class support vector), b = 0.43 (set based on training results). The training data for the support vector machine classifier comes from 210 manually annotated tool image samples. Each image is annotated with 3 regions, resulting in a total of 630 training samples, including 420 normal samples and 210 defect samples. The defect samples are distributed as follows: 90 edge cracks, 60 indented chipped corners, and 60 slight wear. The input features are processed by max-min normalization and then input into the model for training. The radial basis function kernel function is used and the optimal parameters σ = 1 are selected through 10-fold cross-validation. The penalty factor C = 1.0. Finally, the classification accuracy is 94.1%, the defect recognition precision is 91.3%, the defect recall is 88.6%, and the F1 score is 89.9 on the test set, proving that the model has good classification stability and generalization ability. With the support of this model, E2 is judged as an edge crack anomaly, and E1 and E3 are judged as normal regions. The corresponding defect annotations will be combined with the image coordinates to output the final label.

[0178] Specifically, such as Figure 1 As shown, the specific steps of S5 are as follows:

[0179] S501: Based on defect labels, displacement anomaly information and tool shape markings, the defect type is encoded as a numerical vector, the weighted statistical features of the offset error are extracted, and the shape marking level information is fused to construct a three-dimensional feature combination. The random forest model is called for parallel prediction to generate the predicted value of the compensation parameter.

[0180] The predicted values ​​of the compensation parameters are the corrected tool parameter estimates output by the random forest model;

[0181] Based on defect labels, displacement anomaly information, and tool morphology markings, the defect labels are first discretized, converting each defect category into an independent vector. Defects A, B, and C correspond to the three-dimensional vectors [1, 0, 0], [0, 1, 0], and [0, 0, 1], respectively. This method effectively maps non-numerical information to the computational space. Then, the corresponding tool morphology marking level is extracted for each sample, and the level is quantified into integer values: 1 for slight marking, 2 for moderate marking, and 3 for severe marking. Furthermore, for each defect sample, the displacement anomaly amount presented in the image space is obtained and multiplied by the morphology level to obtain a weighted displacement deviation value. For example, if the morphology level of defect A is 1... If the displacement is 0.8mm, the weighted bias is 0.8×1=0.8mm. For defect sample C, with a morphological level of 3 and a displacement of 0.5mm, the weighted bias is 0.5×3=1.5mm. Finally, the information from the above three dimensions is combined to form a feature vector of length 5, which consists of a label vector (3-dimensional), a morphological level (1-dimensional), and a weighted offset (1-dimensional). This feature combination is processed in parallel by the random forest model after structural organization. Each decision subtree independently predicts the compensation parameters for the input sample. For example, for defect B, the input feature [0,1,0,2,2.4] corresponds to a predicted value of 1.68mm. The predicted values ​​of each tree are recorded and summarized to form the compensation parameter prediction results.

[0182] S502: Based on the predicted values ​​of the compensation parameters, the independent prediction results of the decision trees in the random forest model are statistically analyzed, the standard deviation of the fluctuation of the prediction results is calculated, and the out-of-bag error rate in the random forest training process is extracted and weighted linearly combined to generate the confidence coefficient.

[0183] The confidence coefficient is an indicator that measures the reliability of the prediction results of the compensation parameters;

[0184] The out-of-bag error rate is the average error rate generated by the random forest model during the training phase on data samples that did not participate in tree building.

[0185] Based on the predicted compensation parameters generated above, the independent prediction outputs of each decision tree are extracted from within the random forest. Let δ be the difference between the predicted value of each tree and the mean of the final compensation parameters. s This is used to measure the consistency and bias of predictions. To quantify the confidence level, the confidence coefficient is calculated using the formula:

[0186]

[0187] Where C represents the confidence coefficient, q represents the number of independent decision trees in the random forest model, and δ s This represents the difference between the predicted value of the s-th decision tree and the predicted value of the compensation parameter, in pixels. represents the mean of all decision tree discrepancies, in pixels, and ∈ represents the out-of-bag error rate in the random forest training records.

[0188] The number of decision trees is fixed at 5 during the random forest modeling stage. This value is determined by setting the model parameters before training. Selecting a smaller number of trees ensures that the computational load is controllable and the prediction results have structural dispersion. 5 trees is a standard configuration for small and medium-sized detection models and is suitable for real-time prediction tasks.

[0189] The out-of-bag error rate is 0.08. In the process of building the random forest model, each tree is constructed by sampling from the training set using the bootstrap method. The unsampled part is called the out-of-bag sample. The error rate is obtained by predicting the out-of-bag sample by the model and comparing it with the label. After calculating the mean of 200 sets of samples, the out-of-bag error rate is stable at 0.08. In multiple independent training batches, its range remains between 0.07 and 0.10. This value is selected from a representative training experiment record.

[0190] The mean difference value is 1.4. The mean difference value is obtained by averaging the prediction outputs of 5 trees for a certain sample, which are 1.2, 1.5, 1.3, 1.6 and 1.4 (in pixels).

[0191] After substituting into the calculation, the normalized term in the denominator is:

[0192]

[0193] Next, the confidence sub-item was calculated for each tree, and the absolute value of the deviation was {0.2, 0.1, 0.1, 0.2, 0.0}.

[0194] The weighted terms are {0.0153, 0.0076, 0.0076, 0.0153, 0.0000};

[0195] The squared terms are {1.44, 2.25, 1.69, 2.56, 1.96};

[0196] The combined terms are {1.4553, 2.2576, 1.6976, 2.5753, 1.9600}, and the final mean is:

[0197]

[0198] Table 5. Detailed Calculation Table of Confidence Coefficient

[0199]

[0200] As shown in Table 5, the prediction differences of all decision trees were normalized and combined with the squared terms to form confidence sub-terms. Finally, the average value was calculated to obtain the confidence coefficient C = 1.9892.

[0201] S503: Combine the confidence coefficient with the predicted value of the compensation parameter, use the allowable limit of the compensation deviation as the judgment threshold, compare the confidence coefficient with the limit, and if the difference is less than zero, add a field indicating that it needs to be reviewed, and generate the quality inspection result;

[0202] The allowable limit for compensation deviation is set based on the tolerance range of the tool package and its operating conditions, and is determined by evaluating the tool manufacturing precision requirements and the error tolerance in the actual machining environment.

[0203] The generated compensation parameter prediction values ​​and confidence coefficients C = 1.9892 are correspondingly encapsulated into structured output units. Based on the dual verification of tool packaging tolerance (±0.8mm) and process capability index (Cpk = 1.45), the allowable limit for compensation deviation is finally set to 2.0. Comparing the current confidence level with this limit, the difference is calculated as 1.9892 - 2.0 = -0.0108. According to the judgment criteria, if the difference is less than zero, it indicates insufficient confidence, and a review mark addition operation needs to be performed. The corresponding field is named "Needs Review", the data type is Boolean, and the field value is set to True. The final generated structured data fields include three items: "Compensation Prediction Value", "Confidence Coefficient", and "Needs Review", as shown in the example below: [1.68, 1.9892, True]. This constitutes a data information unit for subsequent quality inspection and manual review.

[0204] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for quality inspection of suture scalpel packaging based on visual nerves, characterized in that, Includes the following steps: S1: Acquire the initial image of the suture scalpel packaging, perform corner detection and gradient direction filtering, extract corner coordinates to complete pixel physical coordinate mapping, convert to a unified coordinate system, calculate feature point Euclidean offset to obtain packaging alignment parameters; S2: Calculate the initial value of optical flow displacement based on the packaging alignment parameters, extract the predicted value and calculate the difference between it and the reference displacement value to obtain the residual, determine whether the residual exceeds the limit, if it exceeds the limit, trigger the depth tracking model and correct the displacement, compare the correction result with the preset offset threshold, and output the packaging displacement abnormal information. The specific steps of S2 include: S201: Based on the packaging alignment parameters, select matching feature points in adjacent frames, construct a gray-level gradient spatiotemporal matrix, obtain the motion vector by least squares solution, extract the horizontal and vertical components as the predicted displacement, subtract each item from the model reference displacement value, calculate the absolute value of the residual, and generate a residual vector set. The model reference displacement value is compared with the actual calculated motion vector components to evaluate the accuracy of the motion estimation; S202: Call the residual vector set, calculate the residual mean and standard deviation, set the residual screening threshold to a multiple of the standard deviation, screen the image coordinates corresponding to the out-of-limit residuals, extract the spatiotemporal features of the image blocks, fuse optical flow information and model reference displacement values, and generate the corrected displacement. S203: Based on the corrected displacement, calculate the difference between the length of each displacement vector and the preset offset threshold, count the number of vectors with an absolute value greater than zero, compare the proportion with the preset anomaly judgment threshold, determine whether the proportion exceeds the threshold, and generate displacement anomaly information. The anomaly detection threshold is determined based on the proportional relationship of the offset threshold; S3: Based on the abnormal packaging displacement information, crop the ROI sub-image, call Canny to detect and extract the cutting edge contour, calculate the offset between the orthogonal projection X-axis length and the standard cutting edge length, and generate the tool shape mark; S4: Based on the tool shape mark, locate the cutting edge area, extract texture and shape features, calculate the contrast, energy and entropy of the gray-level co-occurrence matrix, analyze the contour parameters, and input them into the trained support vector machine classifier for defect identification, and output defect labels. S5: Based on the defect labels, displacement anomaly information, and morphological markers, call the random forest model to calculate compensation parameters, perform confidence assessment, and encapsulate and output quality inspection results.

2. The method for detecting the packaging quality of suture scalpels based on visual nerves according to claim 1, characterized in that, The corner detection and gradient direction filtering extract structural feature points with directional stability and spatial positioning accuracy, which are used as initial feature inputs into the depth tracking model and support vector machine classifier, and are fused with the extracted spatiotemporal texture features. The depth tracking model is a three-layer convolutional neural network that extracts spatiotemporal features and generates displacement correction values. The Canny edge detection method performs Gaussian filtering on the image to reduce noise, calculates pixel gradients to identify edge strength, filters edge pixels based on gradient magnitude, and connects edges using a double threshold method. The gray-level co-occurrence matrix employs a multi-directional feature extraction method and undergoes gray-level quantization processing. The support vector machine classifier uses the high-confidence texture features output by the depth tracking model as input to complete the classification and identification of tool defect types; The alignment parameters specifically include translation amount, rotation angle, and scaling factor; the displacement anomaly information includes horizontal displacement deviation, vertical displacement deviation, and correction residual value; the tool shape markings include cutting edge curvature, cutting edge length offset, and contour symmetry; the defect labels specifically refer to crack type, notch size, and corrosion level; and the quality inspection results include compensation angle, confidence score, and packaging level.

3. The method for detecting the packaging quality of suture scalpels based on visual nerves according to claim 1, characterized in that, The specific steps of S1 include: S101: Acquire the initial image of the suture scalpel packaging, use corner detection to traverse the pixel region, calculate the gray-level change rate, construct the gradient autocorrelation matrix to calculate the gray-level response value of the corner, filter candidate corners according to the set threshold, call the Sobel operator to extract the gradient direction, calculate the cosine similarity with the preset angle interval, filter corners with similarity higher than the tolerance threshold, and generate a set of corner coordinates. The tolerance threshold is a preset allowable deviation range for the gradient direction, which is determined by statistically analyzing the gradient direction dispersion of effective corner points in multiple sets of samples. S102: Based on the set of corner coordinates, call the camera intrinsic parameters and distortion coefficients to perform distortion correction, transform to the robot arm base coordinate system through homogeneous coordinate transformation, and superimpose the end pose transformation matrix to generate a physical coordinate set. S103: Call the continuous frame coordinates of the same feature point in the physical coordinate set, calculate the three-axis Euclidean offset to construct the least squares residual matrix, filter the offset samples within the trajectory error tolerance, fit the translation vector and rotation matrix, map the feature point Euclidean offset to a unified coordinate system, dynamically correct the initial offset estimate, and generate a packaging alignment parameter set. The trajectory error tolerance is set based on the repeatability accuracy index of the robotic arm, constraining the maximum allowable value of the offset deviating from the theoretical trajectory.

4. The method for detecting the packaging quality of suture scalpels based on visual nerves according to claim 3, characterized in that, The formula for calculating the grayscale response value of the corner points is as follows: ； in, Representing the The grayscale response value of each pixel. Representing the The pixel in the first Sobel gradient values ​​in the X direction during secondary window sampling Representing the The pixel in the first Sobel gradient values ​​in the Y direction during secondary window sampling Representing the All pixels within the window The average value, Representing the All pixels within the window The average value, To avoid extremely small positive numbers with zero denominators, n represents the number of pixels in a 3×3 window.

5. The method for detecting the packaging quality of suture scalpels based on visual nerves according to claim 1, characterized in that, The specific steps of S3 include: S301: Based on the abnormal coordinates recorded in the displacement anomaly information, traverse the X-axis and Y-axis sets, extract the extreme values ​​to calculate the side length of the rectangle surrounding the anomaly point, call the bilinear interpolation algorithm to smooth the boundary coordinates, and generate the ROI sub-image; S302: Call the ROI sub-image, calculate the gray-level gradient magnitude of the pixels in the horizontal and vertical directions and merge them into an overall gradient map, calculate the gradient magnitude, count the frequency by binning, form an amplitude histogram, set high and low thresholds, filter pixels within the threshold range, connect adjacent pixels to form a closed curve, remove short segments, and obtain the coordinate set of the blade edge. S303: Based on the X-axis projection direction of the cutting edge coordinate set, the frequency density of the coordinate points is statistically analyzed, the endpoints of the interval of the threshold number of consecutive effective coordinate points in the frequency density distribution are extracted, the endpoint spacing is calculated as the actual cutting length, the difference is calculated with the standard cutting length parameter, the absolute value is taken as the cutting length offset, the offset is compared with the preset cutting length tolerance threshold, and if it exceeds the threshold, it is marked as an abnormal shape, and a tool shape mark is generated. The effective coordinate point threshold is calculated based on the minimum number of consecutive effective points according to the tolerance range of the tool design dimensions; The blade length tolerance threshold is set according to the tool wear limit value in the equipment maintenance manual.

6. The method for detecting the packaging quality of suture scalpels based on visual nerves according to claim 5, characterized in that, The specific steps of S4 include: S401: Based on the coordinates of the abnormal area marked in the tool shape mark, traverse all X / Y axis coordinate points in the image coordinate system, filter the coordinate set marked as the specified abnormal category, calculate the range of the coordinate set as the boundary of the rectangular area, construct the texture analysis area with abnormality, replace the noise pixels with the mean of the neighboring pixels, and generate the cutting edge positioning mask. S402: Call the pixel data within the blade positioning mask, statistically analyze the gray-level co-occurrence matrix of pixels at a set interval threshold along the set angle threshold direction, calculate the horizontal contrast as the weighted sum of the squares of the gray-level differences between adjacent pixels, simultaneously extract the aspect ratio of the minimum bounding rectangle of the contour point set, and generate a feature vector set. S403: Arrange the feature vector set in the order of samples, input it into the pre-trained support vector machine classifier, call the radial basis kernel function to calculate the feature space distance, determine the sample category according to the sign of the decision hyperplane, and output the defect label.

7. The method for detecting the packaging quality of suture scalpels based on visual nerves according to claim 6, characterized in that, The radial basis function is called to calculate the feature space distance using the following formula: ； in, This represents a sample point in the input space. Represents another sample point in the input space. Representative point With point The kernel function values ​​between The bandwidth parameter representing the RBF kernel function. The unit is mm. σ is adaptively optimized based on the spatial distribution law of the texture features of the tool edge defects. By statistically analyzing the edge gradient amplitude in the defect texture image, the kernel function bandwidth that matches the actual edge shape is determined. The unit conversion is based on the image resolution of the tool inspection equipment.

8. The method for detecting the packaging quality of a suture scalpel based on visual nerves according to claim 6, characterized in that, The specific steps of S5 include: S501: Based on the defect label, displacement anomaly information and tool shape mark, the defect type is encoded as a numerical vector, the weighted statistical features of the offset error are extracted, and the shape mark level information is fused to construct a three-dimensional feature combination. The random forest model is called to perform parallel prediction and generate the predicted value of the compensation parameter. The predicted values ​​of the compensation parameters are the corrected tool parameter estimates output by the random forest model; S502: Based on the predicted values ​​of the compensation parameters, the independent prediction results of the decision trees in the random forest model are statistically analyzed, the standard deviation of the fluctuation of the prediction results is calculated, and the out-of-bag error rate in the random forest training process is extracted and weighted linearly combined to generate the confidence coefficient. The confidence coefficient is an indicator that measures the reliability of the prediction results of the compensation parameters; The out-of-bag error rate is the average error rate generated by the random forest model during the training phase on data samples that did not participate in tree building. S503: The confidence coefficient and the predicted value of the compensation parameter are paired and encapsulated. The allowable limit of the compensation deviation is used as the judgment threshold. The confidence coefficient and the limit are compared. If the difference is less than zero, a verification mark field is added to generate the quality inspection result. The allowable limit for compensation deviation is set based on the tolerance range of the tool package and its operating conditions, and is determined by evaluating the tool manufacturing precision requirements and the error tolerance in the actual machining environment.

9. The method for detecting the packaging quality of suture scalpels based on visual nerves according to claim 8, characterized in that, The confidence coefficient is calculated using the following formula: ； in, Represents the confidence level coefficient. This represents the number of independent decision trees in the random forest model. Representing the The difference between the predicted values ​​of each decision tree and the predicted values ​​of the compensation parameters, in pixels. This represents the mean of all decision tree variances, in pixels. This represents the out-of-bag error rate in the training records of the random forest.

Images