Suturing scalpel packaging quality detection method based on optic nerves

Abstract

The invention relates to the technical field of defect detection, in particular to a suturing scalpel packaging quality detection method based on optic nerves, which comprises the following steps: collecting suturing scalpel packaging images to perform angular point detection and gradient screening, obtaining alignment parameters under a unified coordinate system, calculating an optical flow displacement initial value and a residual error, and calculating an optical flow displacement initial value; an ROI sub-image is cut to extract a cutting edge contour; an orthogonal projection length is calculated to generate a cutter form mark; texture and shape features are extracted and input into a support vector machine for defect identification; a random forest regression model is called to calculate compensation parameters; according to the method, coordinate mapping is established through angular point detection and gradient screening, displacement is corrected through combination of optical flow residual and depth tracking, a cutting edge contour is extracted through ROI cutting, shape deviation is quantized through orthogonal projection, a classification basis is constructed through texture fusion, defects are identified through a support vector machine, parameters are compensated through random forest regression, and performance is optimized through closed-loop feedback.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, in particular to a method for detecting the packaging quality of suture scalpels based on visual nerves. Background Art

[0002] The field of defect detection technology includes research on quality control and defect identification technologies in the medical device packaging process. The core content of this field mainly focuses on how to use intelligent vision and image recognition to achieve timely and accurate detection of abnormal conditions in the scalpel packaging process. This technical field systematically covers the configuration of image acquisition equipment, visual image preprocessing methods, feature extraction methods, and discrimination mechanisms for different defect types. It aims to replace traditional manual inspection methods with automated visual inspection methods to improve inspection efficiency and consistency. The visual neural mechanism has gradually become one of the core contents of research in this field. Its simulation of human visual perception ability to improve detection recognition capabilities in complex backgrounds is an important direction to promote the development of this field.

[0003] The suture scalpel packaging quality inspection method utilizes an image recognition strategy built using a bionic visual neural mechanism. It is specifically designed to identify quality issues that arise during the packaging process of suture scalpels. The technical aspects addressed in this topic include detecting blade position offset, missing blade tip protection, and damaged sealing edges. Specifically, this method automatically detects the packaging status of suture scalpels by collecting multi-angle image information, combining it with a visual receptive field model to perform layered responses on image regions, and then using preset local contrast features and edge response information to classify and identify abnormal patterns. This method primarily relies on bionic image response mechanisms, local contrast function calculations, and edge extraction operations to identify multiple types of defects.

[0004] Existing technologies rely on a layered response mechanism based on a visual receptive field model to process image regions. Their local contrast features and edge response information extraction methods are not adaptable to complex backgrounds. When packaging materials contain texture interference or reflective areas, the layered response mechanism is susceptible to high-frequency noise, leading to missed detections of missing blade tip protection. For example, a reflective area on a metal blade can be misidentified as a normal seal edge. Feature extraction methods are limited to a preset local contrast function, making it difficult to dynamically adapt to the spatial distribution characteristics of different defect types. For example, blade position offset detection relies on a fixed threshold, failing to automatically adjust the judgment criteria for different packaging specifications, resulting in reduced detection consistency. Image preprocessing methods fail to incorporate a multi-angle coordinate mapping mechanism, requiring manual calibration and alignment of images from different viewpoints, increasing operational complexity and susceptibility to human error. The discrimination mechanism lacks multi-source data fusion and compensation verification. Defect classification relies on a single feature response. For example, seal edge damage detection relies solely on edge response information, ignoring texture continuity analysis, which can easily lead to misjudgments in scenarios with minor cracks in the seal line. In addition, the existing technology does not integrate a dynamic tracking model to correct displacement parameters. Image blur or displacement deviation caused by the movement of the robotic arm requires repeated detection processes, which reduces real-time performance and increases hardware load. Summary of the Invention

[0005] To address this issue, existing technologies rely on a layered response mechanism based on a visual receptive field model to process image regions. However, their local contrast feature and edge response extraction methods lack adaptability to complex backgrounds. When packaging materials contain texture interference or reflective areas, the layered response mechanism is susceptible to high-frequency noise, leading to missed detections of missing blade protection. For example, a reflective area on a metal blade can be misclassified as a normal seal edge. Feature extraction methods are limited to a preset local contrast function, making it difficult to dynamically adapt to the spatial distribution characteristics of different defect types. For example, blade position offset detection relies on a fixed threshold, unable to automatically adjust the judgment criteria for different packaging specifications, resulting in reduced detection consistency. Image preprocessing methods fail to incorporate a multi-angle coordinate mapping mechanism, requiring manual calibration and alignment of images from different viewpoints, increasing operational complexity and susceptibility to human error. The discrimination mechanism lacks multi-source data fusion and compensation verification, and defect classification relies on a single feature response. For example, seal edge damage detection relies solely on edge response information, ignoring texture continuity analysis, which can easily lead to misclassification of even minor cracks in the seal line. In addition, the existing technology does not integrate a dynamic tracking model to correct displacement parameters. Image blur or displacement deviation caused by the movement of the robotic arm requires repeated detection processes, which reduces real-time performance and increases hardware load. The embodiment of the present invention provides a suture scalpel packaging quality inspection method based on visual nerves. The technical solution is as follows:

[0006] In one aspect, a method for detecting the quality of suture scalpel packaging based on visual nerves is provided, the method comprising:

[0007] S1: Collect the initial image of the suture scalpel packaging, perform corner detection and gradient direction screening, extract the corner coordinates to complete the pixel physical coordinate mapping, convert it to a unified coordinate system, calculate the Euclidean offset of the feature points to obtain the packaging alignment parameters;

[0008] S2: Calculate the initial value of the optical flow displacement based on the package alignment parameters, extract the predicted value to calculate the residual, determine whether the residual exceeds the limit, and if it exceeds the limit, trigger the depth tracking model and correct the displacement, compare the correction result with the preset offset threshold, and output package displacement abnormality information;

[0009] S3: Crop the ROI sub-image according to the packaging displacement abnormality information, call Canny detection to extract the edge contour, calculate the offset between the orthogonal projection X-axis length and the standard blade length, and generate the tool shape mark;

[0010] S4: Locate the cutting edge area based on the tool morphological mark, extract texture and shape features, calculate the contrast, energy and entropy of the gray-level co-occurrence matrix, analyze the contour parameters, and input them into the trained support vector machine classifier for defect recognition, and output the defect label;

[0011] S5: Based on the defect label, displacement anomaly information and morphological mark, the random forest model is called to calculate the compensation parameters, and the confidence assessment and packaging output quality inspection results are performed.

[0012] As a further solution of the present invention, the corner detection and gradient direction screening extract structural feature points with directional stability and spatial positioning accuracy, which are input into the depth tracking model and support vector machine classifier as initial features and are fused with the extracted spatiotemporal texture features;

[0013] The depth tracking model is a neural network with a three-layer convolutional structure, which extracts spatiotemporal features and generates displacement correction values;

[0014] The Canny edge detection method performs Gaussian filtering on the image to reduce noise, calculates pixel gradients to identify edge strength, filters edge pixels based on gradient amplitude, and uses a double threshold method to connect edges;

[0015] The gray-level co-occurrence matrix adopts a multi-directional feature extraction method and performs gray-level quantization processing;

[0016] The support vector machine classifier uses the high-confidence texture features output by the depth tracking model as input to complete the classification and identification of tool defect types;

[0017] The alignment parameters specifically include translation, rotation angle, and scaling factor; the displacement anomaly information includes horizontal displacement deviation, vertical displacement deviation, and correction residual value; the tool shape mark includes cutting edge curvature, blade length offset, and contour symmetry; the defect label specifically refers to crack type, notch size, and corrosion level; the quality inspection results include compensation angle, confidence score, and packaging level.

[0018] As a further solution of the present invention, the specific steps of S1 include:

[0019] S101: Capture the initial image of the suture scalpel packaging, use Harris corner detection to traverse the pixel area, calculate the grayscale change rate, construct an autocorrelation matrix to calculate the grayscale response value of the corner point, screen candidate corner points based on the set threshold, call the Sobel operator to extract the gradient direction, calculate the cosine similarity with the preset angle interval, screen corner points with similarity higher than the tolerance threshold, and generate a set of corner point coordinates;

[0020] The tolerance threshold is a preset allowable deviation range of the gradient direction, which is determined by experimentally analyzing the gradient direction dispersion of valid corner points in multiple groups of samples.

[0021] S102: Based on the corner point coordinate set, call the camera intrinsic parameters and distortion coefficients to perform distortion correction, convert to the robot arm base coordinate system through homogeneous coordinate transformation, superimpose the end position transformation matrix, and generate a physical coordinate set;

[0022] S103: Call the continuous frame coordinates of the same feature point in the physical coordinate set, calculate the three-axis Euclidean offset to construct a least squares residual matrix, filter the offset samples within the trajectory error tolerance, fit the translation vector and rotation matrix, map the Euclidean offset of the feature point to a unified coordinate system, dynamically correct the initial offset estimate, and generate a packaging alignment parameter set.

[0023] The trajectory error tolerance is set according to the repeatability index of the robot arm, and constrains the offset to deviate from the maximum allowable value of the theoretical trajectory.

[0024] As a further solution of the present invention, the grayscale response value of the corner point is calculated using the formula:

[0025]

[0026] Among them, R i Represents the grayscale response value of the i-th pixel, Represents the Sobel gradient value in the X direction of the i-th pixel in the j-th window sampling, Represents the Sobel gradient value in the Y direction of the i-th pixel in the j-th window sampling, Represents the number of pixels in the window The average value of Represents the number of pixels in the window , ε is a small positive number that avoids the denominator being zero, and n represents the number of pixels in the 3×3 window.

[0027] As a further solution of the present invention, the specific steps of S2 include:

[0028] S201: Based on the packaging alignment parameters, matching feature points in adjacent frames are selected, a grayscale gradient spatiotemporal matrix is constructed, a least squares solution is obtained to obtain a motion vector, horizontal and vertical components are extracted as predicted displacements, and the predicted displacements are subtracted item by item with the model reference displacement value to calculate the absolute value of the residual and generate a residual vector set;

[0029] The model reference displacement value is compared with the actual calculated motion vector component to evaluate the accuracy of the motion estimation;

[0030] S202: calling the residual vector set, calculating the residual mean and standard deviation, setting the residual screening threshold to a multiple range of the standard deviation, screening the image coordinates corresponding to the excess residuals, extracting the spatiotemporal features of the image blocks, fusing the optical flow information with the model reference displacement value, and generating a corrected displacement;

[0031] S203: Based on the corrected displacement, the difference between the length of each displacement vector and a preset offset threshold is calculated, the number of vectors whose absolute value of the difference is greater than zero is counted, the proportion is compared with a preset abnormality determination threshold, and whether the proportion exceeds the threshold is determined, and displacement abnormality information is generated;

[0032] The abnormality determination threshold is determined according to the proportional relationship of the offset threshold.

[0033] As a further solution of the present invention, the specific steps of S3 include:

[0034] S301: Based on the abnormal coordinates recorded in the displacement abnormality information, traverse the X-axis and Y-axis sets, extract the extreme values, calculate the side length of the rectangle surrounding the abnormal point, call the bilinear interpolation algorithm to smooth the boundary coordinates, and generate the ROI sub-image;

[0035] S302: Calling the ROI sub-image, calculating the horizontal and vertical grayscale gradient amplitudes of the pixels and merging them into an overall gradient map, calculating the gradient amplitude, binning and counting the frequencies, forming an amplitude histogram, setting high and low thresholds, filtering pixels within the threshold range, connecting adjacent pixels to form a closed curve, eliminating short segments, and obtaining a set of blade edge coordinates;

[0036] S303: Based on the X-axis projection direction of the cutting edge coordinates, the frequency density of the coordinate points is counted, and the endpoints of the interval of the threshold number of consecutive valid coordinate points in the frequency density distribution are extracted. The distance between the endpoints is calculated as the actual blade length, and the difference between the actual blade length and the standard blade length parameter is calculated. The absolute value is taken as the blade length offset, and the offset is compared with the preset blade length tolerance threshold. If the offset exceeds the threshold, it is marked as a morphological abnormality, and a tool morphology mark is generated;

[0037] The threshold value of the number of valid coordinate points is calculated based on the tolerance range of the tool design size to calculate the minimum number of consecutive valid points;

[0038] The edge length tolerance threshold is set according to the tool wear limit value in the equipment maintenance manual.

[0039] As a further solution of the present invention, the specific steps of S4 include:

[0040] S401: Based on the coordinates of the abnormal area marked in the tool shape mark, traverse all X / Y axis coordinate points in the image coordinate system, filter the coordinate set marked as the specified abnormal category, calculate the range of the coordinate set as the rectangular area boundary, construct the abnormal texture analysis area, replace the noise pixels with the neighborhood pixel mean, and generate the cutting edge positioning mask;

[0041] S402: Calling the pixel data in the edge positioning mask, counting the grayscale co-occurrence matrix of the pixels with a set interval threshold along the direction of the set angle threshold, calculating the horizontal contrast as the weighted sum of the squares of the grayscale differences of adjacent pixels, and synchronously extracting the minimum circumscribed rectangle aspect ratio of the contour point set to generate a feature vector set;

[0042] S403: Arrange the feature vector set in sample order, input it into the pre-trained support vector machine classifier, call the radial basis kernel function to calculate the feature space distance, determine the sample category according to the decision hyperplane sign, and output the defect label.

[0043] As a further solution of the present invention, the radial basis kernel function is called to calculate the feature space distance using the formula:

[0044]

[0045] Where x represents a sample point in the input space, x′ represents another sample point in the input space, K(x, x′) represents the kernel function value between point x and point x′, σ represents the bandwidth parameter of the RBF kernel function, and the unit of σ is mm. σ is adaptively optimized according to the spatial distribution law of the defect texture characteristics of the tool edge. The kernel function bandwidth that matches the actual edge morphology is determined by statistics of the edge gradient amplitude in the defect texture image. The unit conversion is performed according to the image resolution of the tool detection equipment to ensure that the kernel function bandwidth is consistent with the characteristic size of the tool entity.

[0046] As a further solution of the present invention, the specific steps of S5 include:

[0047] S501: Based on the defect label, displacement anomaly information and tool morphology mark, the defect type is encoded as a numerical vector, the weighted statistical features of the offset error are extracted, and the morphology mark level information is integrated to construct a three-dimensional feature combination, and a random forest model is called for parallel prediction to generate a compensation parameter prediction value;

[0048] The compensation parameter prediction value is a modified tool parameter estimation value output by the random forest model;

[0049] S502: Based on the predicted value of the compensation parameter, the independent prediction results of the decision trees in the random forest model are counted, the fluctuation standard deviation of the prediction results is calculated, and the out-of-bag error rate during the random forest training process is extracted and weighted linear combination is performed to generate a confidence coefficient;

[0050] The confidence coefficient is an indicator for measuring the credibility of the compensation parameter prediction results;

[0051] The out-of-bag error rate is the average error rate generated by the random forest model during the training phase for data samples that do not participate in tree building;

[0052] S503: Pair and encapsulate the confidence coefficient and the predicted value of the compensation parameter, reference the compensation deviation limit as a judgment threshold, compare the confidence coefficient with the limit, and if the difference is less than zero, add a review-required flag field to generate a quality inspection result;

[0053] The allowable limit of the compensation deviation is set according to the tolerance range of the tool package and its operating conditions, and is determined by evaluating the tool manufacturing accuracy requirements and the error tolerance in the actual processing environment.

[0054] As a further solution of the present invention, the confidence coefficient is calculated using the formula:

[0055]

[0056] Among them, C represents the confidence coefficient, q represents the number of independent decision trees in the random forest model, and δ s Represents the difference between the prediction value of the sth decision tree and the prediction value of the compensation parameter, in pixels. represents the mean of all decision tree difference values in pixels, and ∈ represents the out-of-bag error rate in the random forest training records.

[0057] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0058] Through corner detection and gradient direction screening, a pixel-to-physical coordinate mapping is established, unifying the coordinate system of multi-angle images, eliminating the impact of viewpoint differences on feature extraction, and improving alignment accuracy and data consistency. A residual error judgment mechanism based on the initial optical flow displacement, combined with a depth tracking model, corrects offsets and dynamically adjusts tracking parameters to accommodate complex motion scenarios. This enhances the robustness of displacement anomaly detection and reduces the probability of false positives caused by mechanical vibration or packaging material deformation. ROI sub-image cropping focuses on the cutting edge, combining Canny detection with orthogonal projection length calculation to quantify tool shape deviations, reduce background noise interference, and improve the efficiency of cutting edge profile analysis. Texture feature extraction integrates the gray-level co-occurrence matrix and contour parameters, fusing multidimensional features to construct a classification basis and optimize defect characterization. The support vector machine classifier utilizes a high-dimensional feature space to delineate the decision boundary, improving the accuracy of identifying damaged sealing edges and missing tool tips. A random forest regression model calculates compensation parameters based on multi-source data and, combined with a confidence assessment mechanism, balances the reliability of detection results, achieving adaptive optimization of quality judgment. The above process forms a closed-loop feedback system, introducing collaborative optimization strategies in image registration, dynamic tracking, feature fusion and result verification, comprehensively improving the generalization ability and fault tolerance of detection, and effectively dealing with interference factors such as light fluctuations, material reflections and mechanical jitter during the packaging process, ensuring a balance between defect recognition coverage and false detection suppression rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 Schematic diagram of the workflow of the present invention. DETAILED DESCRIPTION

[0060] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0061] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0062] In the embodiments of the present invention, the terms "image" and "picture" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same. The terms "of," "corresponding," and "corresponding" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same.

[0063] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0064] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0065] See also Figure 1 The embodiment of the present invention provides a method for detecting the packaging quality of suture scalpels based on the visual nerve. The processing flow of the method may include the following steps:

[0066] S1: Collect the initial image of the suture scalpel packaging, perform corner detection and gradient direction screening, extract the corner coordinates to complete the pixel physical coordinate mapping, convert it to a unified coordinate system, calculate the Euclidean offset of the feature points to obtain the packaging alignment parameters;

[0067] S2: Calculate the initial value of the optical flow displacement based on the package alignment parameters, extract the predicted value to calculate the residual, and determine whether the residual exceeds the limit. If it exceeds the limit, the depth tracking model is triggered and the displacement is corrected. The correction result is compared with the preset offset threshold and the package displacement anomaly information is output;

[0068] S3: Crop the ROI sub-image according to the packaging displacement anomaly information, call Canny detection to extract the edge contour, calculate the offset between the orthogonal projection X-axis length and the standard blade length, and generate the tool shape mark;

[0069] S4: Locate the cutting edge area based on the tool morphology mark, extract texture and shape features, calculate the contrast, energy and entropy of the gray-level co-occurrence matrix, analyze the contour parameters, and input them into the trained support vector machine classifier for defect recognition and output defect labels;

[0070] S5: Based on the defect label, displacement anomaly information and morphological mark, the random forest model is called to calculate the compensation parameters, perform confidence assessment and package output quality inspection results.

[0071] The alignment parameters specifically include translation, rotation angle, and scaling factor. The displacement anomaly information includes horizontal displacement deviation, vertical displacement deviation, and corrected residual value. The tool shape mark includes cutting edge curvature, blade length offset, and contour symmetry. The defect label specifically refers to crack type, notch size, and corrosion level. The quality inspection results include compensation angle, confidence score, and packaging level.

[0072] Specifically, if Figure 1 As shown, the steps of S1 are specifically as follows:

[0073] S101: Capture the initial image of the suture scalpel packaging, use Harris corner detection to traverse the pixel area, calculate the grayscale change rate, construct an autocorrelation matrix to calculate the grayscale response value of the corner point, screen candidate corner points based on the set threshold, call the Sobel operator to extract the gradient direction, calculate the cosine similarity with the preset angle interval, screen corner points with similarity higher than the tolerance threshold, and generate a set of corner point coordinates;

[0074] The tolerance threshold is the preset allowable deviation range of the gradient direction, which is determined by experimentally analyzing the gradient direction dispersion of valid corner points in multiple groups of samples.

[0075] The suture scalpel packaging image is collected using a CMOS industrial camera to perform single-frame image capture. After the image is grayscaled, a pixel-by-pixel traversal operation is performed. A 3×3 basic window sliding area is set, and the current pixel is taken as the center. The horizontal and vertical gradient values G of all pixels in the window under Sobel convolution are calculated respectively. xij ,G yij The average gradient of the X and Y directions of the center pixel i in the window is recorded as u xi 、u yi , calculate the grayscale response value of the corner point, using the formula:

[0076]

[0077] Among them, R i Represents the grayscale response value of the i-th pixel, Represents the Sobel gradient value in the X direction of the i-th pixel in the j-th window sampling, Represents the Sobel gradient value in the Y direction of the i-th pixel in the j-th window sampling, Represents the number of pixels in the window The average value of Represents the number of pixels in the window The average value of ε is a constant to avoid the denominator being zero. n represents the number of pixels in the 3×3 window, and ε=10 -5 , used to avoid the problem of the denominator being zero, selected in the sample example:

[0078] The sum of products is:

[0079]

[0080] Denominator calculation:

[0081] ∑(G xij -u xi ) 2 =44.67,∑(G yij -u yi )2 =36.67,

[0082]

[0083] Finally, the response value is:

[0084]

[0085] In order to determine the screening threshold R of the corner response value th , that is, R i In the research process, 500 suture scalpel packaging image samples were used as the statistical basis, and the corner response value R of the candidate pixels selected by Harris corner detection was calculated in each image. i The calculation of the response value is concentrated to form a response value distribution sequence {R1, R2, ..., R m}, where m is 100–300 corner points in each image and the total number of samples is greater than 50,000.

[0086] Next, we calculate the overall distribution of all corner point response values and obtain:

[0087] Mean: μ R =2.8;

[0088] Standard deviation: σ R =0.58;

[0089] The distribution range of the response values is concentrated in the interval [1.2, 4.0]. To ensure that the effective corner points have a certain degree of recognition and avoid the introduction of background pseudo corner points due to retaining too many weak responses, the threshold is set as follows:

[0090] R th =μ R -0.5·σ R =2.8-0.5·0.58=2.8-0.29=2.51;

[0091] Truncate the result down to a conservative value and get:

[0092] R th =2.5;

[0093] This setting ensures that approximately 70–75% of the effective strong corner points can be retained, while eliminating 25–30% of the weak corner points in the tail low response range. This range shows high background error and directional instability in experimental statistics. After elimination, the overall screening accuracy can be improved.

[0094] Table 1 Corner point response value distribution statistics sample table

[0095]

[0096]

[0097] As shown in Table 1, the threshold value is set by combining the statistical mean and standard deviation, which not only conforms to the discrete characteristics of the response value in the real image scene, but also effectively controls the range of corner point retention.

[0098] According to the set screening threshold R th =2.5, if R i >R th , then retain the point as a candidate corner point. Then use cosine similarity to determine whether the direction is consistent:

[0099] cos(Δθ)=cos(θ i -θ0),

[0100] where θ i is the direction of the corner point, θ0 = 45°, the angle tolerance is 5°, and the tolerance threshold is cos(5°)≈0.996. If the cosine of the corner point direction angle is higher than the threshold, it will be further retained, and finally a corner point coordinate set is formed;

[0101] S102: Based on the corner point coordinate set, call the camera internal parameters and distortion coefficients to perform distortion correction, convert to the robot arm base coordinate system through homogeneous coordinate transformation, superimpose the end position transformation matrix, and generate a physical coordinate set;

[0102] Based on the retained corner point set, by calling the camera intrinsic parameter matrix:

[0103]

[0104] And the distortion coefficients {k1 = -0.223, k2 = 0.150, p1 = 0.0009, p2 = 0.0012, k3 = -0.08} perform distortion correction on each corner point. Take the pixel point (u, v) = (128, 314) and normalize it to the camera coordinate system:

[0105]

[0106] Introducing the depth parameter λ = 500 mm, we get the three-dimensional point:

[0107]

[0108] Use the extrinsic transformation matrix:

[0109]

[0110] Convert the camera coordinates to the robot base coordinates and combine them with the end effector posture matrix T end Perform overlay and finally calculate the physical coordinates of the corner points, for example:

[0111] P world =[-80.12mm, 156.24mm, 100.5mm],

[0112] After transforming all corner points in sequence, a complete set of physical space coordinates is generated.

[0113] S103: Call the continuous frame coordinates of the same feature point in the physical coordinate set, calculate the three-axis Euclidean offset to construct a least squares residual matrix, select the offset samples within the trajectory error tolerance, fit the translation vector and rotation matrix, map the Euclidean offset of the feature point to a unified coordinate system, dynamically correct the initial offset estimate, and generate a packaging alignment parameter set;

[0114] The trajectory error tolerance is set according to the repeatability index of the robot arm, which constrains the offset to deviate from the maximum allowable value of the theoretical trajectory.

[0115] Call the spatial coordinates of the same feature point in the physical coordinate set in the continuous frame image, and set the continuous frame sampling value:

[0116] P1=[100.12, 150.32, 200.45], P2=[100.10, 150.34, 200.42],

[0117] P3=[100.11, 150.33, 200.43];

[0118] Calculate its three-axis Euclidean offset:

[0119] Δx=x t+1 -x t , Δy=y t+1 -y t , Δz=z t+1 -z t ;

[0120] Corresponding inter-frame residual:

[0121] ||ΔP1||=0.036mm, ||ΔP2||=0.017mm, ||ΔP3||=0.012mm;

[0122] The trajectory error tolerance is set to δ=0.02mm, and the following parameters are selected: ||ΔP t The sample points with ||<δ are used to fit the rigid body transformation and construct the least squares residual function:

[0123]

[0124] Finally, the rotation matrix R is obtained align With the translation vector t align , forming the package alignment parameter set:

[0125] {R align ,t align};

[0126] Specifically, if Figure 1 As shown, the steps of S2 are specifically as follows:

[0127] S201: Based on the packaging alignment parameters, matching feature points in adjacent frames are selected to construct a grayscale gradient spatiotemporal matrix. The motion vector is obtained by least squares solution. The horizontal and vertical components are extracted as the predicted displacement. The displacement is subtracted from the model reference displacement value item by item, and the absolute value of the residual is calculated to generate a residual vector set.

[0128] The model reference displacement values are compared with the actual calculated motion vector components to evaluate the accuracy of the motion estimation;

[0129] Based on the packaging alignment parameters, a multi-channel fusion tensor containing image grayscale, edge features, and color contrast is first constructed in each pair of image frames. In the feature extraction process, in addition to the response corner points directly obtained based on the grayscale gradient, each feature point is supplemented with its high-dimensional texture description vector. The description vector is extracted by inputting an encoding network composed of multi-layer convolution and feature enhancement modules into the image block. The network takes the local window of the image as input and outputs the embedded features around the feature point. Pair the embedded features of all feature points in the current frame with those in the previous frame to calculate feature similarity, specifically calculating the L2 distance of each pair of feature points And set the matching threshold ∈=0.15, if d ij <∈, it is considered that the feature points in the two frames are matched successfully, and a feature matching set after high-dimensional screening is constructed. Then, the matching point pairs are used to construct the image gradient matrix A and the grayscale difference matrix b, and the linear system is solved. Get the predicted motion vector Then the six-degree-of-freedom motion data of the end of the robotic arm is called, and the six-axis angle values [32°, 47°, 59°, 13°, 11°, 4°] are fed back and input into the inverse kinematics transformation function f -1 , solve the theoretical displacement (Δx, Δy) = (2.0, 1.0) in the current cycle, and make a difference with the predicted vector to obtain the residual vector. After taking the absolute value of each vector, a residual vector set is constructed for subsequent depth vector re-identification.

[0130] S202: Call the residual vector set, calculate the residual mean and standard deviation, set the residual screening threshold to a multiple range of the standard deviation, screen the image coordinates corresponding to the excess residuals, extract the spatiotemporal features of the image blocks, fuse the optical flow information with the model reference displacement value, and generate the corrected displacement;

[0131] Based on the existing residual vector set, firstly, the displacement history of all feature points in consecutive image frames is calculated. Combine into a three-dimensional time series and build an input structure Each sequence corresponds to the predicted displacement record of a feature point on three consecutive frames; in order to further determine whether the residual is caused by temporary occlusion or reflection of packaging materials, The input is fed into a lightweight recursive structure model, which uses a dual-channel time update mechanism to calculate the output sequence residual trend vector through the state unit. Used to replace the original residual value; based on the model output vector, the mean and standard deviation μ are calculated in the horizontal and vertical directions respectively x ′,σ x ′, to set the screening threshold τ x ′=2·σ x ';

[0132] Determine whether the residual of each point exceeds the limit in turn. If the residual of a point is greater than the value in any direction, it is determined to be abnormal. The feature point image block is expanded into a grayscale time series block along the time dimension, and the optical flow analysis is performed on the block to obtain the optical flow vector and the reference displacement value (r x ,r y ) are mixed to generate the corrected displacement, where the fusion formula is:

[0133]

[0134] Taking point P3 as an example, let r x =2.0, r y =1.0, after correction we get (1.18, 0.16), integrating all the corrected displacements to form the set V corr , used for the next step of abnormal statistics.

[0135] S203: Based on the corrected displacement, the difference between the length of each displacement vector and a preset offset threshold is calculated, and the number of vectors whose absolute value of the difference is greater than zero is counted. The proportion is compared with a preset abnormality determination threshold to determine whether the proportion exceeds the threshold, and displacement abnormality information is generated;

[0136] The abnormality determination threshold is determined based on the proportional relationship of the offset threshold.

[0137] For set V corr For each correction vector in , first calculate the modulus:

[0138]

[0139] Then compared with the preset offset threshold δ=1.0, the setting of the preset offset threshold δ=1.0 is based on actual data optimization: the experiment uses a data set containing 12,000 frames of images (with 3,850 anomaly points and 85,200 normal points marked). The analysis found that the spatial offset amplitude of normal points is concentrated in the range of 0.2 to 0.8, while the offset amplitude of anomaly points generally exceeds 1.2. In order to balance the detection sensitivity and false alarm rate, the gradient descent method is used to dynamically optimize the threshold: starting from the initial value of 0.5, after 150 rounds of iterative adjustment, the final threshold is stabilized at 1. The test results after optimization show that the system's capture rate of real anomalies is increased to 92.3%, while the false alarm rate is reduced to 5.8%, which is a significant improvement in comprehensive performance compared to the threshold set by manual experience (0.8). In the face of interference scenarios such as sudden changes in illumination and mechanical vibrations, the false alarm rate fluctuation range is stable at 5.2% to 6.5%, which is about 40% lower than before optimization. The difference ΔL is obtained i =L i -δ, if the difference is positive, the point is judged to be abnormal; in order to eliminate structural disturbance interference, such as multi-point misjudgment caused by periodic slip of tool or packaging edge, the correction vector (v corr,x , v corr,y ) and its past three frames of history vectors are input together into the abnormality discrimination network structure. The network uses the spatial attention mechanism to extract whether there is a high-density abnormal concentration pattern and outputs the abnormality confidence score s for each point. i ∈[0,1], set the confidence threshold θ=0.7, if s i >θ and ΔL i >0, it is finally confirmed as an abnormal point. The total number of abnormal points m and the total number of points n are counted, and the ratio p = m / n is obtained. It is compared with the threshold γ = 0.3 to determine the frame-level abnormality. If p>0.3, the frame is marked as having abnormal motion. The following integrated table shows the core data of the whole process:

[0140] Table 2 Abnormal determination table of fusion deep features and correction vectors

[0141]

[0142]

[0143] As shown in Table 2, combining the residual modulus and confidence level, we ultimately identified three points as abnormal, accounting for 50% of the total. This exceeded the 30% threshold, indicating that a displacement anomaly occurred in the current image frame. All depth path processing in the model uses the image feature sequence as input and involves three stages: residual screening, optical flow vector fusion, and anomaly classification. This ensures that the depth tracking structure is naturally embedded within the existing solution.

[0144] Specifically, if Figure 1 As shown, the steps of S3 are as follows:

[0145] S301: Based on the abnormal coordinates recorded in the displacement abnormality information, traverse the X-axis and Y-axis sets, extract the extreme values, calculate the side length of the rectangle surrounding the abnormal point, call the bilinear interpolation algorithm to smooth the boundary coordinates, and generate the ROI sub-image;

[0146] Based on the abnormal coordinates recorded in the displacement anomaly information, the coordinate sets of all abnormal points in the X-axis and Y-axis directions are first extracted and recorded as {x1, x2, ..., x n} and {y1, y2, …, y n}, traverse the two sets of coordinate values in turn, and call the maximum and minimum functions to calculate the extreme values in the set. For example, given the X coordinate set of the outlier point is [123, 180, 240, 305, 360, 420], and the Y coordinate set is [88, 92, 95, 99, 104, 110], the starting coordinate of the rectangle boundary is the upper left corner (x min ,y min )=(123,88), the lower right corner is (x max ,y max )=(420,110), based on which the side length of the rectangle is calculated to be l x =x max -x min =297 pixels, l y =y max -y min =22 pixels; to improve boundary continuity and subsequent image block segmentation accuracy, the image region edge point set is called on this basis, boundary coordinate points are interpolated between the four corners of the rectangular boundary, a bilinear interpolation operation is performed between the values of two adjacent pixels on each edge, the grayscale value of each interpolation point is calculated, and the boundary is extended in the direction of adjacent pixels, ultimately forming a smooth and closed ROI boundary outline. The corresponding area image is cropped from the original image according to the coordinate range to form an ROI sub-image.

[0147] S302: Call the ROI sub-image, calculate the horizontal and vertical grayscale gradient amplitudes of the pixels and merge them into an overall gradient map, calculate the gradient amplitude, bin the frequency, form an amplitude histogram, set high and low thresholds, filter pixels within the threshold range, connect adjacent pixels to form a closed curve, eliminate short segments, and obtain the edge coordinate set;

[0148] After calling the generated ROI sub-image, scan each pixel in the sub-image matrix pixel by pixel, and calculate the grayscale gradient value ΔG in the horizontal (x) and vertical (y) directions respectively based on the grayscale difference between the current pixel and the upper, lower, left and right adjacent pixels. x , ΔG y , then perform square summation on the gradient magnitudes in these two directions and take the square root to get the overall gradient magnitude of the pixel point The gradient amplitude of all pixels in the entire image is counted to form a grayscale histogram, and the high and low thresholds of the histogram are set according to the pixel distribution. For example, the 15th percentile is selected as the low threshold and is set as T l = 20, the 85th percentile is used as the high threshold, set as T h = 28, and judge whether the gradient values of all pixels are within [T l , T h ] interval, if it meets the requirements, it is determined to be a significant edge point; then, the adjacent pixel points that meet the requirements are grouped by connection operation, and the length of each group of pixel chains is detected. If the chain length is lower than the minimum valid edge length (for example, set to 8 pixels), it is eliminated, and only the pixel chains with the length meeting the requirements are retained to construct the closed curve contour. The pixel positions on the edges of all chains are extracted in the current image coordinate system to form a blade edge coordinate set.

[0149] S303: Based on the X-axis projection direction of the cutting edge coordinates, the frequency density of the coordinate points is counted, and the endpoints of the interval of the threshold number of consecutive valid coordinate points in the frequency density distribution are extracted. The distance between the endpoints is calculated as the actual blade length, and the difference between the actual blade length and the standard blade length parameter is calculated. The absolute value is taken as the blade length offset, and the offset is compared with the preset blade length tolerance threshold. If the offset exceeds the threshold, it is marked as a morphological abnormality, and a tool morphology mark is generated;

[0150] The threshold value of the number of valid coordinate points is calculated based on the tolerance range of the tool design size to calculate the minimum number of consecutive valid points;

[0151] The edge length tolerance threshold is set according to the tool wear limit value in the equipment maintenance manual.

[0152] Based on the extracted edge coordinate set, all edge point coordinates are first projected along the X-axis direction, that is, the number of Y coordinates corresponding to each set of X coordinate values is counted to form a frequency density array {f(x)}, and the frequency array is traversed to find the number of f(x)≥N at any continuous position. min The left and right endpoints of the segment are extracted as the effective edge segment range, where N min is the threshold of effective coordinate points. This value is calculated by inverse calculation from the tool tolerance range. For example, the design blade length of a standard tool is 31.7 pixels, corresponding to a 1mm length resolution of 8 pixels, and ±1mm wear is allowed. The corresponding minimum continuous length is 8×1=8 pixels. To improve stability, the threshold is set to 10 points, that is, N min =10; judge the continuous segments that meet the conditions in each frequency density curve. If the conditions are met in a certain X interval, extract its left and right endpoints x l , x r , calculate the spacing as L = x r -x l, which is the actual length of the current tool edge; for example, the starting and ending points of the frequency density continuous segment in the area where P3 is located are x l =208.6 and x r = 241.0, then the blade length is L = 32.4 pixels, calling the standard blade length L set by the system s = 31.7 pixels, and the blade length offset d is obtained by performing the difference calculation L =|LL s |=|32.4-31.7|=0.7 pixels, and then compare this value with the blade length tolerance threshold δ L = 0.5 pixels for comparison and judgment, if d L >δ L , then the blade segment is marked as a morphological abnormal point; applying this judgment logic to all valid feature points, statistics show that only the blade length deviation of P3 exceeds the tolerance value, and the remaining feature points are all qualified points.

[0153] Table 3 Cutting edge shape detection calculation table

[0154]

[0155] As shown in Table 3, P3 is the only one whose blade length offset is larger than the tolerance (0.7>0.5) and is judged as abnormal.

[0156] Specifically, if Figure 1 As shown, the steps of S4 are specifically as follows:

[0157] S401: Based on the coordinates of the abnormal area marked in the tool shape mark, all X / Y axis coordinate points in the image coordinate system are traversed, and the coordinate set marked as the specified abnormal category is screened. The range of the coordinate set is calculated as the rectangular area boundary, and the texture analysis area with abnormalities is constructed. The noise pixels are replaced with the mean of the neighborhood pixels to generate the cutting edge positioning mask;

[0158] Based on the abnormal area coordinate information stored in the tool shape mark, the coordinate point set of the image block marked as a defect is extracted one by one, and the corresponding abnormal number and category label are recorded. The X-axis and Y-axis positions of all pixels in the image coordinate system are double-nested and traversed. It is judged in turn whether each pixel falls within the coordinate range corresponding to the current specified abnormal number. If its X coordinate meets the x-axis, the pixel is identified as a defect. min ≤x i ≤x max , the Y coordinate satisfies y min ≤y i ≤y max, and its category label is the category corresponding to the current processing anomaly number, then the pixel is added to the current area processing set, and after recording all coordinate points, the difference between the maximum and minimum values in the X and Y directions is calculated as the pixel range in the horizontal and vertical directions respectively. For example, the coordinate range of the E1 area is x∈[210,240] and y∈[90,115], and the calculated pixel range X is 30 and Y is 25. On this basis, the 3×3 neighborhood grayscale matrix of all pixels in the area is called in turn, and the arithmetic mean of the remaining 8 pixels is taken after removing the center pixel value. For example, if the neighborhood grayscale value of a certain point is [112, 115, 116, 117, 110, 114, 113, 118], the mean is 114.375≈114. The original center value is replaced by this mean. This operation is performed on each pixel point in the abnormal area in turn to form a continuous, smooth, and isolated point noise-free binary mask image as the edge positioning area for further texture analysis.

[0159] S402: Calling the pixel data in the edge positioning mask, counting the grayscale co-occurrence matrix of the pixels with a set interval threshold along the set angle threshold direction, calculating the horizontal contrast as the weighted sum of the squares of the grayscale differences of adjacent pixels, and synchronously extracting the minimum circumscribed rectangle aspect ratio of the contour point set to generate a feature vector set;

[0160] In the generated mask image, the pixel block corresponding to each abnormal number is selected, and the gray-level co-occurrence matrix statistical operation is performed. The gray-level directions are set to 0°, 45°, and 90° to cover the horizontal, main diagonal, and vertical directions of the image. The pixel interval is set to 2 pixels in each direction. This value is determined jointly by the image spatial resolution and the target area size. The spatial resolution in the actual image is 0.125 mm / pixel, and the target area is about 25–35 pixels wide (about 3–4 mm). Taking the interval d = 2 pixels is a spatial perception range of 0.25 mm. Within this range, the local texture structure can be stably reflected while avoiding single-pixel level texture interference. The system measured that when d>3, the texture response is greatly weakened, and the contrast drops by more than 40%. Then, each pair of reference pixels and target pixels in each direction forms a gray-level pair (g i , g j ), cumulatively count the frequencies of all grayscale combinations and construct a 16-level grayscale co-occurrence matrix P(i, j), and calculate the horizontal contrast value according to the following formula:

[0161]

[0162] Where C represents the contrast value of the gray-level co-occurrence matrix at a specified angle, which is a statistic that measures the intensity of grayscale changes in an image area. i represents the grayscale level of the reference pixel corresponding to the row index in the gray-level co-occurrence matrix, and its value range is 0 to 15 (when the gray-level co-occurrence matrix is 16 levels). j represents the grayscale level of the target pixel corresponding to the column index in the gray-level co-occurrence matrix, and its value range is also 0 to 15. (ij) 2 represents the square of the difference between the grayscale value of the reference pixel and its corresponding target pixel, which is used to reflect the magnitude of the grayscale difference between pixels. P(i, j) represents the normalized probability frequency of the pixel pair with the reference pixel grayscale value i and the target pixel grayscale value j appearing in the entire image or the specified mask area, that is, the proportion of the element in the i-th row and j-th column of the co-occurrence matrix in the total number of all co-occurrence pairs.

[0163] For example, the grayscale combination frequency of the E1 region is mainly concentrated near the diagonal. If P(8, 9) = 0.10, P(9, 10) = 0.15, and P(10, 11) = 0.12, the contrast contribution is 0.10 + 0.15 + 0.12 = 0.37. Adding a farther pair such as (4, 12), P = 0.08, the overall contrast value is calculated to be 18.2; then the contour edge point set is extracted from the mask image, and the minimum circumscribed rectangle contour fitting method is called to calculate the length and width of its envelope rectangle. For example, the fitting width of the E1 region is 24, the height is 12, and the aspect ratio is r = 24 / 12 = 2.0. Combined with the contrast value, the feature vector of the current abnormal region is formed, such as [18.2, 2.0]. This process is performed synchronously on all abnormal numbers, and the final feature vector set is shown in Table 4.

[0164] Table 4 Abnormal area mask and texture feature extraction table

[0165]

[0166] As shown in Table 4, the contrast value and aspect ratio of the E2 region are both high, indicating that its edge contour fluctuates violently.

[0167] S403: Arrange the feature vector set in sample order, input it into the pre-trained support vector machine classifier, call the radial basis kernel function to calculate the feature space distance, determine the sample category according to the decision hyperplane sign, and output the defect label.

[0168] Arrange the eigenvectors corresponding to the abnormal numbers in the table in numerical order and construct the input sample matrix, where the eigenvector consists of two dimensions, namely the grayscale co-occurrence matrix contrast value and the minimum circumscribed rectangle aspect ratio of the region outline. For example, the eigenvector of E1 is [18.2, 2.0], E2 is [21.7, 2.8], and E3 is [17.5, 2.0]. Combine all eigenvectors to form a two-dimensional sample matrix X∈R 3×2, as the input sample set of the classification model, the sample set is input into the trained support vector machine classifier for defect type recognition. The classifier uses the radial basis kernel function, and its kernel function form is:

[0169]

[0170] Among them, x represents a sample point in the input space, x′ represents another sample point in the input space, K(x, x′) represents the kernel function value between point x and point x′, σ represents the bandwidth parameter of the RBF kernel function, σ is in mm, σ is adaptively optimized according to the spatial distribution law of the tool edge defect texture characteristics, and the kernel function bandwidth that matches the actual edge morphology is determined by counting the edge gradient amplitude in the defect texture image. The unit conversion is based on the image resolution of the tool detection equipment to ensure that the kernel function bandwidth is consistent with the tool entity feature size. Taking the E2 input vector [21.7, 2.8] and a support vector [20.5, 2.5] as an example for calculation, we have:

[0171] ||xx′|| 2 =(21.7-20.5) 2 +(2.8-2.5) 2 =1.44+0.09=1.53,

[0172] Set the kernel function parameter σ = 1, then the kernel value is:

[0173]

[0174] The kernel value and the weight coefficient a of the support vector i Its category label y i After multiplication, the kernel response values of all support vectors are accumulated and the bias term b is added to obtain the classification decision function value:

[0175]

[0176] Among them, a i Represents the weight coefficient of the support vector, y i represents the category label of the support vector, and b is the bias term.

[0177] If the sign of the value is positive, the input sample is judged as a defect area, and if the sign is negative, it is a normal area. The judgment process does not require adjusting the original features, and is only based on its relative position in the feature boundary constructed by the support vector. i =0.6 (assuming that the support vector is a typical value of non-boundary samples), y i= + 1 (defect class support vector), b = 0.43 (set according to the training results). The training data of the support vector machine classifier comes from 210 manually annotated tool image samples. Each image is annotated with 3 regions, resulting in a total of 630 training samples, including 420 normal classes and 210 defect classes. The defect samples are distributed as follows: 90 edge cracks, 60 internal chipping, and 60 mild wear. The input features are normalized according to the maximum and minimum normalization and then input into the model training. The radial basis kernel function is used and the optimal parameters σ = 1 and penalty factor C = 1.0 are selected through 10-fold cross validation. Finally, the classification accuracy of 94.1%, defect recognition precision of 91.3%, defect recall rate of 88.6%, and F1 score of 89.9 are achieved in the test set, demonstrating that the model has good classification stability and generalization ability. With the support of this model, E2 is judged as an edge crack abnormality, while E1 and E3 are judged as normal areas. The corresponding defect annotations will be combined with the image coordinates to output the final label.

[0178] Specifically, if Figure 1 As shown, the steps of S5 are specifically as follows:

[0179] S501: Based on the defect label, displacement anomaly information, and tool shape mark, the defect type is encoded as a numerical vector, the weighted statistical features of the offset error are extracted, and the shape mark level information is integrated to construct a three-dimensional feature combination. The random forest model is called for parallel prediction to generate compensation parameter prediction values;

[0180] The predicted value of compensation parameters is the estimated value of modified tool parameters output by the random forest model;

[0181] Based on the defect label, displacement anomaly information and tool morphology mark, the defect label is first discretized and each defect category is converted into an independent vector, where defect A, defect B and defect C correspond to three-dimensional vectors [1, 0, 0], [0, 1, 0] and [0, 0, 1] respectively. In this way, non-numerical information can be effectively mapped to the calculation space. Then, the corresponding tool morphology mark level is extracted for each sample, and the level is quantized into an integer value, with a slight mark assigned to 1, a moderate mark assigned to 2, and a severe mark assigned to 3. Furthermore, for each defect sample, the displacement anomaly presented in the image space is obtained and multiplied by the morphology level to obtain the weighted displacement deviation value. For example, the morphology level of the defect A sample is 1 , the displacement is 0.8mm, then the weighted deviation is 0.8×1=0.8mm. For the defect C sample, the morphological grade is 3 and the displacement is 0.5mm, then the weighted deviation is 0.5×3=1.5mm. Finally, the information of the above three dimensions is combined to form a feature vector of length 5, which is the label vector (3 dimensions), the morphological grade (1 dimension) and the weighted offset (1 dimension). The feature combination is input into the random forest model channel for parallel processing after structural arrangement. Each decision subtree independently predicts the compensation parameters of the input sample. For example, for the defect B input feature [0,1,0,2,2.4], the corresponding prediction value is 1.68mm. The prediction value of each tree is recorded and summarized to form the compensation parameter prediction value result.

[0182] S502: Based on the predicted value of the compensation parameter, the independent prediction results of the decision trees in the random forest model are counted, the fluctuation standard deviation of the prediction results is calculated, and the out-of-bag error rate during the random forest training process is extracted and weighted linear combination is performed to generate a confidence coefficient;

[0183] The confidence coefficient is an indicator to measure the credibility of the prediction results of compensation parameters;

[0184] The out-of-bag error rate is the average error rate generated by the random forest model during the training phase for data samples that do not participate in tree building;

[0185] Based on the compensation parameter prediction values generated above, the independent prediction outputs of each decision tree are extracted from the random forest. The difference between the prediction value of each tree and the final compensation parameter mean is set as δ s , which is used to measure the consistency and deviation of the prediction. To quantify the confidence level, the confidence coefficient is calculated using the formula:

[0186]

[0187] Among them, C represents the confidence coefficient, q represents the number of independent decision trees in the random forest model, and δ s Represents the difference between the prediction value of the sth decision tree and the prediction value of the compensation parameter, in pixels. represents the mean of all decision tree difference values in pixels, and ∈ represents the out-of-bag error rate in the random forest training records.

[0188] The number of decision trees is fixed at 5 during the random forest modeling phase. This value is set by the pre-training model parameter setting. A small number of trees is selected to ensure that the computational load is controllable and the prediction results have structural dispersion. 5 trees is a common configuration for small and medium-sized detection models and is suitable for real-time prediction tasks.

[0189] The out-of-bag error rate is 0.08. During the construction of the random forest model, each tree is constructed by sampling from the training set using the bootstrap method. The unsampled part is called the out-of-bag sample. The model predicts the out-of-bag samples and compares them with the labels to obtain the error rate. After calculating the mean of 200 groups of samples, the out-of-bag error rate is stable at 0.08. In multiple independent training batches, its range remains between 0.07–0.10. This value is selected from a representative training experiment record.

[0190] The mean difference value is 1.4. The prediction outputs of 5 trees for a sample are 1.2, 1.5, 1.3, 1.6 and 1.4 (in pixels), and the mean difference value is obtained by statistically averaging.

[0191] After the calculation, the denominator normalization term is:

[0192]

[0193] Then, the confidence sub-item calculation is performed on each tree, and the absolute value of the deviation is {0.2, 0.1, 0.1, 0.2, 0.0};

[0194] The weighted terms are {0.0153, 0.0076, 0.0076, 0.0153, 0.0000};

[0195] The squared terms are {1.44, 2.25, 1.69, 2.56, 1.96};

[0196] The total term after combination is {1.4553, 2.2576, 1.6976, 2.5753, 1.9600}, and the final mean is:

[0197]

[0198] Table 5 Confidence coefficient calculation details

[0199]

[0200] As shown in Table 5, the prediction differences of all decision trees are normalized and combined with the square term to form the confidence sub-term. The final average value is obtained to obtain the confidence coefficient C = 1.9892.

[0201] S503: Pair and encapsulate the confidence coefficient and the compensation parameter prediction value, reference the compensation deviation limit as the judgment threshold, compare the confidence coefficient with the limit, and if the difference is less than zero, add a review-required flag field to generate a quality inspection result;

[0202] The allowable limit of compensation deviation is set according to the tolerance range of the tool package and its operating conditions, and is determined by evaluating the tool manufacturing accuracy requirements and the error tolerance in the actual processing environment.

[0203] The generated compensation parameter prediction value and the confidence coefficient C = 1.9892 are encapsulated into a structured output unit in a one-to-one correspondence. Based on the dual verification of the tool packaging tolerance (±0.8mm) and the process capability index (Cpk = 1.45), the compensation deviation allowable limit is finally set to 2.0. Comparing the current confidence with the limit, the difference is calculated to be 1.9892-2.0 = -0.0108. According to the judgment criteria, if the difference value is less than zero, it means that the confidence is insufficient and a review mark addition operation needs to be performed. The corresponding field is named "needs review", the data type is Boolean, and the field value is set to True. The final generated structure data field includes three items: "compensation prediction value", "confidence coefficient" and "needs review". The example is as follows: [1.68, 1.9892, True], which constitutes a data information unit for subsequent quality inspection and manual review processing.

[0204] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for detecting the packaging quality of suture scalpels based on visual nerves, characterized in that: The following steps are involved: S1: Collect the initial image of the suture scalpel packaging, perform corner detection and gradient direction screening, extract the corner coordinates to complete the pixel physical coordinate mapping, convert it to a unified coordinate system, calculate the Euclidean offset of the feature points to obtain the packaging alignment parameters; S2: Calculate the initial value of the optical flow displacement based on the package alignment parameters, extract the predicted value to calculate the residual, determine whether the residual exceeds the limit, and if it exceeds the limit, trigger the depth tracking model and correct the displacement, compare the correction result with the preset offset threshold, and output package displacement abnormality information; S3: Crop the ROI sub-image according to the packaging displacement abnormality information, call Canny detection to extract the edge contour, calculate the offset between the orthogonal projection X-axis length and the standard blade length, and generate the tool shape mark; S4: Locate the cutting edge area based on the tool morphological mark, extract texture and shape features, calculate the contrast, energy and entropy of the gray-level co-occurrence matrix, analyze the contour parameters, and input them into the trained support vector machine classifier for defect recognition, and output the defect label; S5: Based on the defect label, displacement anomaly information and morphological mark, the random forest model is called to calculate the compensation parameters, and the confidence assessment and packaging output quality inspection results are performed.

2. The suture scalpel packaging quality detection method based on visual nerve according to claim 1 is characterized in that: The corner detection and gradient direction screening extract structural feature points with directional stability and spatial positioning accuracy, which are input into the depth tracking model and support vector machine classifier as initial features and are fused with the extracted spatiotemporal texture features; The depth tracking model is a neural network with a three-layer convolutional structure, which extracts spatiotemporal features and generates displacement correction values; The Canny edge detection method performs Gaussian filtering on the image to reduce noise, calculates pixel gradients to identify edge strength, filters edge pixels based on gradient amplitude, and uses a double threshold method to connect edges; The gray-level co-occurrence matrix adopts a multi-directional feature extraction method and performs gray-level quantization processing; The support vector machine classifier uses the high-confidence texture features output by the depth tracking model as input to complete the classification and identification of tool defect types; The alignment parameters specifically include translation, rotation angle, and scaling factor; the displacement anomaly information includes horizontal displacement deviation, vertical displacement deviation, and correction residual value; the tool shape mark includes cutting edge curvature, blade length offset, and contour symmetry; the defect label specifically refers to crack type, notch size, and corrosion level; the quality inspection results include compensation angle, confidence score, and packaging level.

3. The suture scalpel packaging quality detection method based on visual nerve according to claim 1, characterized in that: The specific steps of S1 include: S101: Acquire an initial image of the suture scalpel packaging, use corner detection to traverse the pixel area, calculate the grayscale change rate, construct a gradient autocorrelation matrix to calculate the grayscale response value of the corner point, screen candidate corner points based on a set threshold, call the Sobel operator to extract the gradient direction, calculate the cosine similarity with the preset angle interval, screen corner points with similarity higher than the tolerance threshold, and generate a set of corner point coordinates; The tolerance threshold is a preset allowable deviation range of the gradient direction, which is determined by experimentally analyzing the gradient direction dispersion of valid corner points in multiple groups of samples. S102: Based on the corner point coordinate set, call the camera intrinsic parameters and distortion coefficients to perform distortion correction, convert to the robot arm base coordinate system through homogeneous coordinate transformation, superimpose the end position transformation matrix, and generate a physical coordinate set; S103: Calling the continuous frame coordinates of the same feature point in the physical coordinate set, calculating the three-axis Euclidean offset to construct a least squares residual matrix, screening the offset samples within the trajectory error tolerance, fitting the translation vector and rotation matrix, mapping the Euclidean offset of the feature point to a unified coordinate system, dynamically correcting the initial offset estimate, and generating a packaging alignment parameter set; The trajectory error tolerance is set according to the repeatability index of the robot arm, and constrains the offset to deviate from the maximum allowable value of the theoretical trajectory.

4. The suture scalpel packaging quality detection method based on visual nerve according to claim 3 is characterized in that: The grayscale response value of the corner point is calculated using the formula: Among them, R i Represents the grayscale response value of the i-th pixel, Represents the Sobel gradient value in the X direction of the i-th pixel in the j-th window sampling, Represents the Sobel gradient value in the Y direction of the i-th pixel in the j-th window sampling, Represents the number of pixels in the window The average value of Represents the number of pixels in the window , ε is a small positive number that avoids the denominator being zero, and n represents the number of pixels in the 3×3 window.

5. The suture scalpel packaging quality inspection method based on visual nerve according to claim 3, characterized in that: The specific steps of S2 include: S201: Based on the packaging alignment parameters, matching feature points in adjacent frames are selected, a grayscale gradient spatiotemporal matrix is constructed, a least squares solution is obtained to obtain a motion vector, horizontal and vertical components are extracted as predicted displacements, and the predicted displacements are subtracted item by item with the model reference displacement value to calculate the absolute value of the residual and generate a residual vector set; The model reference displacement value is compared with the actual calculated motion vector component to evaluate the accuracy of the motion estimation; S202: calling the residual vector set, calculating the residual mean and standard deviation, setting the residual screening threshold to a multiple range of the standard deviation, screening the image coordinates corresponding to the excess residuals, extracting the spatiotemporal features of the image blocks, fusing the optical flow information with the model reference displacement value, and generating a corrected displacement; S203: Based on the corrected displacement, the difference between the length of each displacement vector and a preset offset threshold is calculated, the number of vectors whose absolute value of the difference is greater than zero is counted, the proportion is compared with a preset abnormality determination threshold, and whether the proportion exceeds the threshold is determined, and displacement abnormality information is generated; The abnormality determination threshold is determined according to the proportional relationship of the offset threshold.

6. The suture scalpel packaging quality inspection method based on visual nerve according to claim 5, characterized in that: The specific steps of S3 include: S301: Based on the abnormal coordinates recorded in the displacement abnormality information, traverse the X-axis and Y-axis sets, extract the extreme values, calculate the side length of the rectangle surrounding the abnormal point, call the bilinear interpolation algorithm to smooth the boundary coordinates, and generate the ROI sub-image; S302: Calling the ROI sub-image, calculating the horizontal and vertical grayscale gradient amplitudes of the pixels and merging them into an overall gradient map, calculating the gradient amplitude, binning and counting the frequencies, forming an amplitude histogram, setting high and low thresholds, filtering pixels within the threshold range, connecting adjacent pixels to form a closed curve, eliminating short segments, and obtaining a set of blade edge coordinates; S303: Based on the X-axis projection direction of the cutting edge coordinates, the frequency density of the coordinate points is counted, and the endpoints of the interval of the threshold number of consecutive valid coordinate points in the frequency density distribution are extracted. The distance between the endpoints is calculated as the actual blade length, and the difference between the actual blade length and the standard blade length parameter is calculated. The absolute value is taken as the blade length offset, and the offset is compared with the preset blade length tolerance threshold. If the offset exceeds the threshold, it is marked as a morphological abnormality, and a tool morphology mark is generated; The threshold value of the number of valid coordinate points is calculated based on the tolerance range of the tool design size to calculate the minimum number of consecutive valid points; The edge length tolerance threshold is set according to the tool wear limit value in the equipment maintenance manual.

7. The suture scalpel packaging quality inspection method based on visual nerve according to claim 6, characterized in that: The specific steps of S4 include: S401: Based on the coordinates of the abnormal area marked in the tool shape mark, traverse all X / Y axis coordinate points in the image coordinate system, filter the coordinate set marked as the specified abnormal category, calculate the range of the coordinate set as the rectangular area boundary, construct the abnormal texture analysis area, replace the noise pixels with the neighborhood pixel mean, and generate the cutting edge positioning mask; S402: Calling the pixel data in the edge positioning mask, counting the grayscale co-occurrence matrix of the pixels with a set interval threshold along the direction of the set angle threshold, calculating the horizontal contrast as the weighted sum of the squares of the grayscale differences of adjacent pixels, and synchronously extracting the minimum circumscribed rectangle aspect ratio of the contour point set to generate a feature vector set; S403: Arrange the feature vector set in sample order, input it into the pre-trained support vector machine classifier, call the radial basis kernel function to calculate the feature space distance, determine the sample category according to the decision hyperplane sign, and output the defect label.

8. The suture scalpel packaging quality inspection method based on visual nerve according to claim 7, characterized in that: The radial basis kernel function is called to calculate the feature space distance using the formula: Where x represents a sample point in the input space, x′ represents another sample point in the input space, K(x, x′) represents the kernel function value between points x and x′, σ represents the bandwidth parameter of the RBF kernel function, and the unit of σ is mm. σ is adaptively optimized according to the spatial distribution law of the defect texture characteristics of the tool edge. The kernel function bandwidth that matches the actual edge morphology is determined by statistics on the edge gradient amplitude in the defect texture image. The unit conversion is based on the image resolution of the tool detection equipment.

9. The suture scalpel packaging quality inspection method based on visual nerve according to claim 7, characterized in that: The specific steps of S5 include: S501: Based on the defect label, displacement anomaly information and tool morphology mark, the defect type is encoded as a numerical vector, the weighted statistical features of the offset error are extracted, and the morphology mark level information is integrated to construct a three-dimensional feature combination, and a random forest model is called for parallel prediction to generate a compensation parameter prediction value; The compensation parameter prediction value is a modified tool parameter estimation value output by the random forest model; S502: Based on the predicted value of the compensation parameter, the independent prediction results of the decision trees in the random forest model are counted, the fluctuation standard deviation of the prediction results is calculated, and the out-of-bag error rate during the random forest training process is extracted and weighted linear combination is performed to generate a confidence coefficient; The confidence coefficient is an indicator for measuring the credibility of the compensation parameter prediction results; The out-of-bag error rate is the average error rate generated by the random forest model during the training phase for data samples that do not participate in tree building; S503: Pair and encapsulate the confidence coefficient and the predicted value of the compensation parameter, reference the compensation deviation limit as a judgment threshold, compare the confidence coefficient with the limit, and if the difference is less than zero, add a review-required flag field to generate a quality inspection result; The allowable limit of the compensation deviation is set according to the tolerance range of the tool package and its operating conditions, and is determined by evaluating the tool manufacturing accuracy requirements and the error tolerance in the actual processing environment.

10. The suture scalpel packaging quality inspection method based on visual nerve according to claim 9, characterized in that: The confidence coefficient adopts the formula: Among them, C represents the confidence coefficient, q represents the number of independent decision trees in the random forest model, and δ s Represents the difference between the prediction value of the sth decision tree and the prediction value of the compensation parameter, in pixels. represents the mean of all decision tree difference values in pixels, and ∈ represents the out-of-bag error rate in the random forest training records.

Images