Full-process Defect Detection Method for Endoscopic Sampling Forceps Based on Machine Vision

Abstract

The present invention relates to the technical field of medical device quality inspection, in particular to a full-process defect detection method for endoscopic sampling forceps based on machine vision. The present invention proposes to process image data through a multi-scale geometric perception feature extractor to generate a feature map set, and use a part-adaptive attention module to identify key parts. A defect-sensitive residual network is used to enhance relevant defect features, and a medical-grade precise segmentation module is used to achieve defect segmentation. Combining an expert knowledge-guided self-calibration mechanism and a defect ontology knowledge base, this method can effectively evaluate and calibrate detection uncertainty, ensure detection accuracy, has strong adaptability, can efficiently extract defect features of different scales, and improve the quality inspection efficiency of endoscopic sampling forceps.

Description

Technical Field

[0001] The present invention relates to the technical field of medical device quality inspection, and particularly to a full-process defect detection method for endoscopic sampling forceps based on machine vision. Background Art

[0002] As a common minimally invasive medical device, endoscopic sampling forceps play a crucial role in clinical diagnosis and treatment. Since it directly contacts human tissues and performs biopsy operations, its quality is closely related to patient safety. Traditional quality inspection of endoscopic sampling forceps mainly relies on manual visual inspection, which has problems such as low efficiency, inconsistent standards, easy fatigue, and difficulty in detecting minor defects.

[0003] Currently, there are some defect detection methods based on machine vision in the industrial field. For example, Chinese Patent CN111681232B discloses a "defect detection method for industrial welding images based on semantic segmentation", which realizes the automatic detection of welding defects through deep learning. However, such methods have the following limitations:

[0004] 1. The application scenario is single, only for industrial welding images, and cannot adapt to the special structure and material characteristics of medical devices such as endoscopic sampling forceps;

[0005] 2. Lack the ability to differentially process multiple parts of the instrument, and cannot perform targeted detection according to the importance and risk level of different parts;

[0006] 3. The accuracy does not meet the high standards required for medical devices, and there is a risk of missed detection especially in the detection of minor defects;

[0007] 4. Lack the guidance and calibration mechanism of expert knowledge, and the interpretability of the detection results is poor, making it difficult to support the quality control decision-making of medical devices;

[0008] 5. Unable to adapt to the changes in defect characteristics at different stages of the full life cycle of endoscopic sampling forceps.

[0009] With the increasingly strict regulatory requirements in the medical device industry, especially the higher requirements of regulatory agencies such as FDA and NMPA for the Good Manufacturing Practice (GMP) of medical device production, there is an urgent need for a high-precision and full-process defect detection method specifically for endoscopic sampling forceps. Summary of the Invention

[0010] In view of the above technical problems, the present invention provides a full-process defect detection method for endoscopic sampling forceps based on machine vision, aiming to achieve high-precision and fully automatic defect detection of endoscopic sampling forceps and meet the strict quality control requirements of medical-grade devices.

[0011] The present invention proposes a full - process defect detection method for endoscopic sampling forceps based on machine vision, including:

[0012] Obtain the image data of the endoscopic sampling forceps;

[0013] Process the image data through a multi - scale geometric perception feature extractor to generate a multi - scale feature map set;

[0014] Based on the multi - scale feature map set, identify three key parts of the endoscopic sampling forceps, namely the jaw, hinge, and rod body, through a part - adaptive attention module, and generate a weighted attention feature map;

[0015] Based on the weighted attention feature map, enhance the features related to the defect type through a defect - sensitive residual network to generate a defect - sensitive feature map;

[0016] Based on the defect - sensitive feature map, generate a defect segmentation result of the endoscopic sampling forceps through a medical - grade precise segmentation module;

[0017] Based on the defect segmentation result, evaluate the uncertainty of the detection result through an expert - knowledge - guided self - calibration mechanism, and when the uncertainty exceeds a preset threshold, calibrate the detection result based on the endoscopic sampling forceps defect ontology knowledge base.

[0018] Preferably, the multi - scale geometric perception feature extractor processes the image data, specifically including:

[0019] Construct a geometric perception convolution kernel including a standard convolution branch, a depth - separable convolution branch, and a point convolution branch;

[0020] Based on the geometric features of different components of the endoscopic sampling forceps, adaptively adjust the weight coefficients of the geometric perception convolution kernel through the following formula:

[0021] ,

[0022] where, is the weight coefficient of the i - th branch; GAP is the global average pooling operation; FC is the fully - connected layer; is the output feature map of the i - th branch; Softmax is the normalization function;

[0023] Capture the direction features of the three key parts of the jaw, hinge, and rod body through an angle - sensitive pooling layer, and the formula is:

[0024] ,

[0025] where, ASP is the angle - sensitive pooling operation; F is the input feature map; θ is the rotation angle, and the value range is [0°, 45°, 90°, 135°]; is the rotation transformation matrix for the angle; MaxPool is the max pooling operation; represents the matrix multiplication operation;

[0026] Generate multi-scale feature maps , where: represents the feature map with a resolution of the original image times; s is the scale factor.

[0027] Preferably, the part adaptive attention module identifies the key parts of the endoscopic sampling forceps, specifically including:

[0028] Calculate the channel-spatial attention maps of the three key parts of the endoscopic sampling forceps, namely the jaws, hinge, and shaft. The formula is:

[0029] ,

[0030] where, is the attention map of part , jaws, hinge, shaft is the feature map of part ; ChannelPool is the channel pooling operation, SpatialPool is the spatial pooling operation; and are learnable weight parameter matrices, ReLU is the rectified linear unit activation function, is the sigmoid activation function, Conv is the convolution operation, · represents the matrix multiplication or element-wise multiplication operation;

[0031] Establish a relationship model between parts to capture the collaborative relationship between each part of the sampling forceps. The formula is:

[0032] ,

[0033] where, is the relationship feature between part and , is the tanh activation function, and are the feature representations of part and part respectively; represents the feature concatenation operation, is the weight parameter matrix for relationship modeling; is the bias parameter vector; is the tanh activation function;

[0034] Assign different weights to the feature maps according to the defect risk degree of the parts to generate weighted attention feature maps 。

[0035] Preferably, the defect-sensitive residual network specifically includes:

[0036] For the specific defect types of the endoscopic sampling forceps, including surface scratches, jaw deformation, hinge looseness, coating peeling, and cracks, a dedicated residual unit is designed, and the formula is:

[0037] ,

[0038] Among them, is the output feature map, is the input feature map, is the standard residual path, is the defect-sensitive skip connection;

[0039] Specialized residual paths are designed for different types of defects:

[0040] For surface scratches, multi-directional Gabor filters are used to enhance texture features;

[0041] For jaw deformation, a residual connection with shape prior constraints is used;

[0042] For hinge looseness, a temporal difference module is combined to capture dynamic features;

[0043] For coating peeling, a reflectivity analysis unit is introduced to enhance material change features;

[0044] For cracks, direction-sensitive dilated convolutions are used to amplify fine structures;

[0045] By cascading multiple residual units, a defect-sensitive feature map is generated 。

[0046] Preferably, the medical-grade precise segmentation module specifically includes:

[0047] A segmentation network guided by reverse attention is designed to enhance the defect boundary extraction ability through foreground-background contrast;

[0048] A medical-specific boundary enhancement loss function is adopted, and the formula is:

[0049] ,

[0050] Among them, is the pixel weight, is the defect category probability, is the pixel to the distance of the nearest defect boundary, is the boundary attention coefficient;

[0051] The defect boundary is iteratively optimized through a multi-stage boundary refinement strategy, and the formula is:

[0052] ,

[0053] where is the boundary at the t-th iteration; t is the number of iterations; is the boundary at the -th iteration; is the defect-sensitive feature map; Refine is the boundary refinement function;

[0054] The rough segmentation result, the boundary refinement result, and the multi-scale fusion result are combined to generate the final endoscopic sampling forceps defect segmentation result.

[0055] Preferably, the expert knowledge-guided self-calibration mechanism specifically includes:

[0056] Construct an endoscopic sampling forceps defect ontology knowledge base, which contains defect cases annotated by experts;

[0057] Calculate the uncertainty based on the attention map through the following formula:

[0058] ,

[0059] where U is the uncertainty metric value of the detection result; is the defect class probability; log is the natural logarithm function, and λ is the balance parameter; is the attention map 's gradient; · represents the L1 norm; ∑ represents the summation operation over all defect classes;

[0060] When the uncertainty exceeds the preset threshold , expert knowledge-guided self-calibration is performed through the following formula:

[0061] ,

[0062] where is the calibrated defect class probability, is the original prediction probability, is the dynamic weight coefficient, is the expert prior probability based on the knowledge base ;

[0063] Generate a calibrated endoscopic sampling forceps defect detection report, which includes defect type, location, severity, and repair suggestions.

[0064] Preferably, the construction of the endoscope sampling forceps defect ontology knowledge base specifically includes:

[0065] Collect various defect samples that occur during the production, use, and maintenance of endoscope sampling forceps;

[0066] Invite at least three medical device quality inspection experts with more than five years of experience to label the defect samples;

[0067] Establish a structured description for each defect sample, including defect type, appearance characteristics, cause analysis, and impact assessment;

[0068] Establish a defect similarity measurement standard for quickly retrieving similar defect cases;

[0069] Regularly update the content of the knowledge base to incorporate newly discovered defect types and characteristics.

[0070] Preferably, it also includes a data preprocessing step:

[0071] Perform illumination normalization on the acquired endoscope sampling forceps images to eliminate the influence of uneven illumination;

[0072] Adopt a specular reflection suppression algorithm based on reflection characteristics to reduce the interference of highlights on the metal surface to detection;

[0073] Perform perspective correction on the images according to the geometric characteristics of the endoscope sampling forceps to ensure the proportional consistency of key parts;

[0074] Enhance image details through contrast-limited adaptive histogram equalization;

[0075] Perform multi-scale noise suppression on the images while retaining the subtle features of the defects.

[0076] Preferably, before acquiring the image data of the endoscope sampling forceps, it also includes an acquisition strategy optimization step:

[0077] Based on the 3D CAD model of the endoscope sampling forceps, determine the key detection perspectives and the best imaging distances;

[0078] Design a specific light source layout, including the positions, angles, and intensities of the main light source and the auxiliary light source;

[0079] For different types of potential defects, formulate corresponding image acquisition parameters, including exposure time, aperture size, and focusing position;

[0080] Evaluate the quality of the acquired images, and when the image quality is lower than the preset standard, automatically adjust the acquisition parameters and re-acquire.

[0081] According to the method described above, after the defect detection of the endoscopic sampling forceps is completed, it further includes a quality closed-loop control step:

[0082] Record the defect detection results of each batch of endoscopic sampling forceps, including defect types, positions, sizes, and distribution characteristics;

[0083] Analyze the temporal and spatial distribution patterns of the defects to identify potential systematic problems;

[0084] Generate improvement suggestions for the production process, including material selection, processing parameters, assembly processes, and quality inspection standards;

[0085] Associate the detection results with the patient safety risk level and establish a risk warning mechanism for defects in endoscopic sampling forceps;

[0086] Regularly evaluate and optimize the performance of the detection system, including monitoring and improving the detection rate, false alarm rate, and processing speed.

[0087] The present invention forms a complete information processing pipeline through five progressive processing modules. The output of each module provides more refined features for the next module, achieving step-by-step refinement of the defect detection of endoscopic sampling forceps. The method has the following beneficial effects:

[0088] 1. Strong adaptability: Through the multi-scale geometric perception feature extractor, it can adapt to the geometric characteristics of different parts of the endoscopic sampling forceps and effectively extract defect features of various scales;

[0089] 2. Precise positioning: The part-adaptive attention module can allocate attention weights according to the importance of different parts of the endoscopic sampling forceps, achieving key attention to high-risk areas;

[0090] 3. Defect sensitivity: For typical defects such as surface scratches, jaw deformation, hinge looseness, coating peeling, and cracks on the endoscopic sampling forceps, a special feature enhancement mechanism is designed to improve the detection accuracy;

[0091] 4. Medical-grade precision: Through precise boundary segmentation and multi-stage refinement, it meets the high-precision detection requirements of medical devices and can identify defects as small as 0.01 mm²;

[0092] 5. Strong interpretability: Combining the expert knowledge base and the self-calibration mechanism, it provides interpretable detection results and quality improvement suggestions to support the production decision-making of medical devices;

[0093] 6. Full-process adaptation: It covers the entire life cycle of endoscopic sampling forceps from production, use to maintenance, and realizes quality closed-loop control. Description of the Drawings

[0094] Figure 1It is the overall flowchart of the full - process defect detection method for endoscopic sampling forceps based on machine vision provided by the present invention;

[0095] Figure 2 It is the structural schematic diagram of the multi - scale geometric perception feature extractor provided by the present invention;

[0096] Figure 3 It is the structural schematic diagram of the part - adaptive attention module provided by the present invention;

[0097] Figure 4 It is the structural schematic diagram of the defect - sensitive residual network provided by the present invention;

[0098] Figure 5 It is the structural schematic diagram of the medical - grade precise segmentation module provided by the present invention;

[0099] Figure 6 It is the flowchart of the expert - knowledge - guided self - calibration mechanism provided by the present invention;

[0100] Figure 7 It is the structural schematic diagram of the endoscopic sampling forceps defect ontology knowledge base provided by the present invention;

[0101] Figure 8 It is the comparison chart of defect detection effects in the application example of the present invention. Detailed implementation manners

[0102] Please refer to the attached Figure 1-8 , and the present invention will be described in detail below with reference to the accompanying drawings and embodiments. These embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0103] Example 1: Overall process of the full - process defect detection method for endoscopic sampling forceps based on machine vision

[0104] Referring to Figure 1 , the full - process defect detection method for endoscopic sampling forceps based on machine vision provided by the present invention mainly includes the following steps:

[0105] First, obtain the image data of the endoscopic sampling forceps. Preferably, a high - resolution industrial camera is used to obtain the image of the endoscopic sampling forceps, and the resolution is not less than 2048×1536 pixels to ensure capturing tiny defect features. During the image acquisition process, a diffuse reflection light source is used to avoid strong reflections on the metal surface, and a fixed fixture is used to ensure the consistency of the position and angle of the sampling forceps.

[0106] Next, the image data is processed by a multi-scale geometric perception feature extractor to generate a multi-scale feature map set. The feature extractor designs a dedicated geometric perception convolution kernel, which can adapt to the geometric characteristics of different parts such as the jaws, hinges, and shafts of the endoscopic sampling forceps, and effectively extract defect features of various scales.

[0107] Then, based on the multi-scale feature map set, a part-adaptive attention module is used to identify the three key parts of the jaws, hinges, and shafts of the endoscopic sampling forceps, and generate a weighted attention feature map. The module assigns attention weights according to the defect risk levels of different parts, and focuses on high-risk areas.

[0108] Next, based on the weighted attention feature map, a defect-sensitive residual network is used to enhance the features related to the defect type, and generate a defect-sensitive feature map. The network designs a dedicated feature enhancement mechanism for typical defects such as surface scratches, jaw deformation, hinge looseness, coating peeling, and cracks on the endoscopic sampling forceps.

[0109] Subsequently, based on the defect-sensitive feature map, a medical-grade precise segmentation module is used to generate the defect segmentation result of the endoscopic sampling forceps. The module meets the high-precision detection requirements of medical devices through precise boundary segmentation and multi-stage refinement, and can identify defects as small as 0.01 mm².

[0110] Finally, based on the defect segmentation result, an expert knowledge-guided self-calibration mechanism is used to evaluate the uncertainty of the detection result, and when the uncertainty exceeds a preset threshold, the detection result is calibrated based on the endoscopic sampling forceps defect ontology knowledge base. The mechanism combines an expert knowledge base and a self-calibration algorithm to provide interpretable detection results and quality improvement suggestions.

[0111] Embodiment 2: Implementation method of the multi-scale geometric perception feature extractor

[0112] Refer to Figure 2 , this embodiment details the implementation method of the multi-scale geometric perception feature extractor, which can effectively handle the feature extraction problem under the complex geometric structure of the endoscopic sampling forceps.

[0113] The multi-scale geometric perception feature extractor first constructs a geometric perception convolution kernel (GACK) containing three branches: a standard convolution branch (3×3), a depthwise separable convolution branch (5×5), and a pointwise convolution branch (1×1). The standard convolution branch mainly captures local structural features, the depthwise separable convolution branch expands the receptive field to obtain a larger range of context information, and the pointwise convolution branch realizes feature integration through channel dimension fusion.

[0114] Based on the geometric features of different components of the endoscopic sampling forceps, the feature extractor dynamically adjusts the importance of each branch through an adaptive weight adjustment mechanism. The weight coefficient calculation formula is:

[0115] ,

[0116] where is the weight coefficient of the i-th branch, indicating the importance of this branch in feature extraction; is the output feature map of the i-th branch; GAP is the global average pooling operation, used to compress the feature map into a single feature vector; FC is the fully connected layer, used to map the feature vector to the weight coefficient; Softmax is the normalization function, ensuring that the sum of all branch weights is 1.

[0117] To better capture the directional features of the key parts of the texture endoscopic sampling forceps, the feature extractor also introduces an angle-sensitive pooling layer (ASP). This layer enhances the directional sensitivity by applying the max pooling operation at different angles, and the formula is:

[0118] ,

[0119] where is the angle-sensitive pooling operation; is the input feature map; θ is the rotation angle, and the value range is {0°, 45°, 90°, 135°}; is the rotation transformation matrix of angle θ; MaxPool is the max pooling operation; represents the matrix multiplication operation. This angle-sensitive design is particularly suitable for detecting directional defects on the sampling forceps, such as surface scratches and cracks.

[0120] The output of the feature extractor is a multi-scale feature map set , where represents the feature map with a resolution of times the original image. s is the scale factor. The multi-scale feature design enables the system to simultaneously focus on large-scale structures (such as overall deformation of the forceps jaws) and small-scale details (such as tiny surface scratches).

[0121] In practical applications, the feature extractor uses 64 standard convolutional kernels, 48 depth convolutional kernels, and 32 point convolutional kernels, and the activation function is selected as the Swish function sigmoid , where is set to 1.5 to provide a smoother gradient flow. The initial learning rate is set to 0.002 and decays by 10% every 25 training epochs to ensure training stability.

[0122] Example 3: Implementation method of the part adaptive attention module

[0123] Reference Figure 3 , this embodiment details the implementation method of the part adaptive attention module, which can adaptively allocate attention resources according to the importance and risk degree of different parts of the endoscopic sampling forceps.

[0124] The part adaptive attention module first identifies three key parts of the endoscopic sampling forceps: the jaws, the hinge, and the shaft. The jaws directly contact the tissue for sampling and are the most critical part in terms of function; the hinge controls the opening and closing of the jaws and is the core of the mechanical structure; the shaft connects the operating end and the working end and transmits the operating force.

[0125] For these three key parts, the module calculates the channel - spatial joint attention map, and the formula is:

[0126] ,

[0127] Among them, is the attention map of part , indicating the degree of attention to the features of this part; represents the key parts of the endoscopic sampling forceps, the jaws, the hinge, the shaft is the feature map of part ; ChannelPool is the channel pooling operation, which is used to aggregate the features in the channel dimension; SpatialPool is the spatial pooling operation, which is used to aggregate the features in the spatial dimension; and are learnable weight parameter matrices; ReLU is the rectified linear unit activation function, defined as ReLU is the sigmoid activation function, defined as Conv is the convolution operation; · represents the matrix multiplication or element-wise multiplication operation.

[0128] In addition, since the defects of the endoscopic sampling forceps often have a correlation (for example, hinge loosening may cause jaw deformation), the module designs a part - to - part relationship modeling unit, and the formula is:

[0129] ,

[0130] Among them, is the relationship feature between part and part , indicating the degree of mutual influence between the two parts; and are the feature representations of part and part respectively; Represents a feature connection operation that concatenates two feature vectors along the channel dimension. Is the weight parameter matrix for relationship modeling; Is the bias parameter vector; Is the tanh activation function, defined as .

[0131] Based on risk assessment, different parts are assigned different initial weights: the initial weight of the jaw area is 1.5 (high-risk area), the hinge area is 1.2 (medium-risk area), and the rod area is 1.0 (low-risk area). These initial weights will be automatically adjusted according to the actual sample distribution during the training process.

[0132] In practical applications, the attention channel compression rate r is set to 16 to balance computational efficiency and expressive ability. The module finally outputs a weighted attention feature map , which serves as the input to the next-stage defect-sensitive residual network.

[0133] Example 4: Implementation method of the defect-sensitive residual network

[0134] Refer to Figure 4 , this example details the implementation method of the defect-sensitive residual network, which is specifically designed to enhance the typical defect features of the endoscopic sampling forceps and improve the detection accuracy.

[0135] The defect-sensitive residual network designs specialized residual units for five main types of defects of the endoscopic sampling forceps (surface scratches, jaw deformation, hinge looseness, coating peeling, and cracks). The basic structure of the residual unit is:

[0136] ,

[0137] Among them, Is the output feature map of the residual unit; Is the input feature map of the residual unit; Is the standard residual path, indicating the function that performs a non-linear transformation on the input through the weight parameter ; Is the defect-sensitive skip connection, representing a feature enhancement path designed for specific defect types; + represents the element-wise addition operation. Different from the traditional residual network, this network adds a defect-sensitive skip connection , and the formula is:

[0138] ,

[0139] Among them, Is the output of the defect-sensitive skip connection; It is a defect type correlation function used to enhance the feature channels related to specific defect types; is an activation function; Conv is a convolution operation; · represents an element-wise multiplication operation.

[0140] For different types of defects, the network designs specialized residual paths:

[0141] For surface scratches, multi-directional Gabor filters are used to enhance texture features. The parameters of the Gabor filters are set to 8 directions and 3 scales to effectively capture the strongly directional scratch features.

[0142] For jaw deformation, a residual connection with shape prior constraints is introduced. By calculating the shape difference from the standard jaw template, the feature response in the deformed area is enhanced, making the network more sensitive to deformation features.

[0143] For hinge looseness, a temporal difference module is combined to capture dynamic features. By analyzing the motion characteristics during the hinge opening and closing process, abnormal motion patterns are detected, which is crucial for the detection of functional defects.

[0144] For coating peeling, a reflectivity analysis unit is introduced to enhance the material change features. By analyzing the reflection characteristics at different incident angles, the change in the coating state is identified to effectively distinguish the normal surface and the peeling area.

[0145] For cracks, direction-sensitive dilated convolutions are used to magnify fine structures. The dilation rate sequence is set to {1, 2, 4, 8, 4, 2, 1} to expand the receptive field without increasing the number of parameters and enhance the perception ability of fine cracks.

[0146] The network consists of 8 residual blocks in total. The defect type coding dimension is 64. The skip connections adopt a combination of additive and multiplicative gating mechanisms to simultaneously retain the original features and enhance the defect features. The network outputs defect-sensitive feature maps , with a feature dimension of 256, providing a high-quality feature representation for subsequent precise segmentation.

[0147] Example 5: Implementation method of the medical-grade precise segmentation module

[0148] Refer to Figure 5 , this example details the implementation method of the medical-grade precise segmentation module, which realizes pixel-level precise segmentation for the high-precision requirements of medical devices.

[0149] The medical-grade precise segmentation module first designs a reverse attention-guided segmentation network to enhance the defect boundary extraction ability through foreground-background contrast. This network not only focuses on the defect area itself but also particularly emphasizes the boundary features between the defect and the normal area to improve the segmentation accuracy.

[0150] To better capture the defect boundaries, the module introduces a boundary enhancement loss function dedicated to medical applications:

[0151] ,

[0152] where: is the value of the boundary enhancement loss function; is the pixel weight, reflecting the importance of the pixel in the segmentation task; is the defect class probability, representing the predicted probability that the pixel belongs to the defect class; is the distance from the pixel to the nearest defect boundary; is the boundary attention coefficient, set to 0.8; denotes the summation operation over all pixels. This loss function design enables the network to pay more attention to the boundary regions during training, improving the boundary localization accuracy.

[0153] The segmentation process is divided into three stages: First, a rough segmentation is performed to obtain the initial defect region; then, the defect boundary is iteratively optimized through boundary refinement, with the formula:

[0154] ,

[0155] where, is the boundary at the t-th iteration; t is the iteration number; is the boundary at the -th iteration; is the defect-sensitive feature map, providing the feature information required for boundary refinement; Refine is the boundary refinement function, which uses wavelet transform to enhance the boundary features. The boundary refinement is iterated 3 times, and the accuracy of the boundary is further improved in each iteration. Finally, through multi-scale fusion, the segmentation results at different resolutions are integrated to generate the final segmentation result.

[0156] In practical applications, the boundary determination threshold is set to 0.65, and the wavelet transform uses 4-level decomposition to balance detail preservation and noise suppression. At the same time, according to the requirements of medical device standards, the minimum defect area that the system can identify is 0.01 mm², meeting the detection accuracy requirements at the medical level.

[0157] The segmentation module finally outputs a pixel-level defect segmentation map M, with a value range of {0, 1, 2, 3, 4, 5}, where 0 represents the background, and 1 - 5 respectively represent five types of defects: surface scratches, jaw deformation, hinge looseness, coating peeling, and cracks.

[0158] Example 6: Implementation method of an expert knowledge-guided self-calibration mechanism

[0159] Refer to Figure 6, this embodiment details the implementation method of an expert knowledge-guided self-calibration mechanism, which combines expert knowledge and uncertainty assessment to improve the reliability and interpretability of detection results.

[0160] The expert knowledge-guided self-calibration mechanism first constructs an endoscope sampling forceps defect ontology knowledge base, which contains defect cases annotated by experts. Each defect case in the knowledge base contains structured information such as defect type, appearance characteristics, cause analysis, and impact assessment, providing expert-level knowledge support for the system.

[0161] The mechanism calculates the uncertainty of the detection result through an attention map-based uncertainty assessment module:

[0162] ,

[0163] where U is the uncertainty metric value of the detection result; is the defect class probability, representing the probability that a pixel belongs to the i-th type of defect; log is the natural logarithm function, λ is a balance parameter that controls the weight of the attention gradient term, and λ is set to 0.3; is the attention map is the gradient of · represents the L1 norm, and the calculation method is the sum of the absolute values of all elements; ∑ represents the summation operation over all defect classes. This uncertainty metric comprehensively considers the entropy of the prediction probability and the stability of the attention, and can more comprehensively evaluate the reliability of the detection result.

[0164] When the uncertainty U exceeds the preset threshold θ (set to 0.75), the system triggers the self-calibration process. The self-calibration adopts an expert knowledge-guided method, and the formula is:

[0165] ,

[0166] where, is the calibrated defect class probability; is the original prediction probability; is the dynamic weight coefficient that controls the influence degree of expert knowledge, initially set to 0.6, and gradually decreases as the number of samples increases; is the expert prior probability based on the knowledge base By retrieving cases similar to the previous sample in the knowledge base, the probability distribution annotated by experts is extracted.

[0167] Dynamic weight coefficient The initial value is set to 0.6 and gradually decreases as the number of samples increases, reflecting the process of gradually reducing the dependence on expert knowledge as the system accumulates experience. The minimum confidence level set by the system is 0.85, and detection results below this value will be marked as requiring manual review.

[0168] The self - calibration mechanism also includes a closed - loop feedback function, which extracts the characteristics of high - uncertainty regions, queries the knowledge base to generate calibration suggestions, and updates the model. The knowledge base is updated monthly to incorporate newly discovered defect types and characteristics, maintaining the timeliness of the knowledge base.

[0169] Example 7: Construction method of the defect ontology knowledge base for endoscopic sampling forceps

[0170] Refer to Figure 7 , this example details the construction method of the defect ontology knowledge base for endoscopic sampling forceps, which provides expert knowledge support for the self - calibration mechanism.

[0171] The construction of the defect ontology knowledge base for endoscopic sampling forceps first collects various defect samples that occur during the production, use, and maintenance of endoscopic sampling forceps. The collection channels include multiple sources such as offline production inspection, clinical use feedback, and maintenance records, ensuring the diversity and representativeness of the samples.

[0172] The collected defect samples are labeled by at least three medical device quality inspection experts with more than five years of experience. The labeling is carried out in a double - blind manner, with each sample independently labeled by two experts and then confirmed and arbitrated by a third expert to ensure the labeling quality.

[0173] A structured description is established for each defect sample, including the following content:

[0174] Defect type: Clearly classified as surface scratches, jaw deformation, hinge looseness, coating peeling, or cracks;

[0175] Appearance features: Describe in detail the visual features such as the shape, size, location, and color of the defect;

[0176] Cause analysis: Explain the possible formation reasons of the defect, such as material problems, processing problems, use and wear, etc.;

[0177] Impact assessment: Evaluate the degree of impact of the defect on function and safety, divided into five levels: extremely low, low, medium, high, and extremely high;

[0178] Treatment suggestions: Provide treatment plans for this type of defect, such as rework, scrapping, restricted use, etc.

[0179] To achieve fast retrieval, the knowledge base establishes a defect similarity measurement standard. The similarity calculation comprehensively considers appearance features, location information, and context environment, and adopts the weighted cosine similarity calculation method. Preferably, the weight of appearance features is 0.5, the weight of location information is 0.3, and the weight of context environment is 0.2.

[0180] The knowledge base adopts a regular update mechanism, and is reviewed by an expert team every month to incorporate newly discovered defect types and features. When there are more than 50 new samples accumulated or new types of defects are discovered, the knowledge base update process is triggered. The update process includes four steps: data cleaning, feature extraction, expert annotation, and data integration.

[0181] Through long-term accumulation, the knowledge base already contains more than 240 defect cases, covering various defects in the entire life cycle of the endoscope sampling forceps, providing rich expert knowledge support for the self-calibration mechanism.

[0182] Embodiment 8: Data preprocessing method

[0183] This embodiment details the data preprocessing method, which provides high-quality input data for subsequent feature extraction and defect detection.

[0184] Data preprocessing first performs illumination normalization on the acquired endoscope sampling forceps images to eliminate the influence of uneven illumination. The multi-scale Retinex algorithm is used to decompose the image into a reflection component and an illumination component, and illumination normalization is achieved by adjusting the illumination component, while retaining the defect information in the reflection component.

[0185] Next, a specular reflection suppression algorithm based on reflection characteristics is adopted to reduce the interference of highlights on the metal surface to detection. This algorithm analyzes polarization characteristics, identifies and suppresses the specular reflection area, and restores the image details blocked by highlights. Preferably, a dual-threshold method is used to identify the specular reflection area, and the thresholds are set to 1.5 times and 2 times the average brightness of the image respectively.

[0186] Then, according to the geometric characteristics of the endoscope sampling forceps, perspective correction is performed on the image to ensure the proportional consistency of key parts. The correction process is based on a predefined standard contour template of the sampling forceps, and image correction is achieved through affine transformation, so that images of each batch have a consistent perspective and proportion.

[0187] Subsequently, contrast-limited adaptive histogram equalization (CLAHE) is used to enhance image details. The parameters of the CLAHE algorithm are set as follows: block size 8×8, contrast limit factor 3.0, and distribution type Rayleigh distribution, in order to suppress noise amplification while enhancing details.

[0188] Finally, multi-scale noise suppression is performed on the image while preserving the subtle features of the defects. A method combining bilateral filtering and guided filtering is adopted to preserve edge and texture information while suppressing noise. Preferably, the standard deviation of the bilateral filtering in the spatial domain is set to 3.0, the standard deviation in the range domain is set to 0.1, the window radius of the guided filtering is set to 5, and the regularization parameter is set to 0.01.

[0189] After the above preprocessing steps, an endoscope sampling forceps image with improved quality is obtained, providing a good data basis for subsequent feature extraction and defect detection.

[0190] Example 9: Acquisition Strategy Optimization Method

[0191] This example details the acquisition strategy optimization method, which improves the quality of the original data by optimizing the image acquisition parameters and provides better input for subsequent processing.

[0192] The acquisition strategy optimization first determines the key detection viewpoints and the best imaging distance based on the 3D CAD model of the endoscope sampling forceps. Through virtual imaging simulation, the coverage of key parts such as the jaw, hinge, and rod body at different viewpoints is analyzed to determine the optimal viewpoint combination, usually including the front view, side view, and 45° inclined view Figure 3 as three standard viewpoints. The best imaging distance is determined according to the image resolution and depth-of-field requirements. For a camera with a resolution of 2048×1536, the preferred imaging distance is 150mm - 200mm.

[0193] Next, a specific light source layout is designed, including the position, angle, and intensity of the main light source and the auxiliary light source. The main light source uses a ring-shaped LED light source to provide the main illumination; the auxiliary light source is a point light source that supplements the illumination from a specific angle to enhance the visibility of the defects. The core principle of the light source layout is that the main light source provides uniform basic illumination, and the auxiliary light source enhances the defect contrast by incident from a specific angle. For metal surface detection, the intensity of the main light source is set to 2 - 3 times that of the auxiliary light source, and the angle between the main light source and the object to be photographed is 15° - 30°.

[0194] Then, corresponding image acquisition parameters are formulated for different types of potential defects, including the exposure time, aperture size, and focusing position. For example, for surface scratches and cracks, a shorter exposure time (10 - 15ms), a smaller aperture (F8 - F11), and precise focusing are used to enhance the edge sharpness; for jaw deformation and hinge looseness, a medium exposure time (20 - 30ms), a medium aperture (F5.6 - F8), and panoramic focusing are used to ensure the overall shape is clear; for coating peeling, a longer exposure time (30 - 40ms), a larger aperture (F4 - F5.6), and surface focusing are used to enhance the visibility of the material texture.

[0195] Finally, evaluate the quality of the captured images. When the image quality is lower than the preset standard, automatically adjust the acquisition parameters and re-acquire the images. The image quality evaluation metrics include four aspects: sharpness, contrast, noise level, and dynamic range, which are quantitatively evaluated through gradient magnitude distribution, histogram uniformity, signal-to-noise ratio, and standard deviation of gray-level distribution. When any of the metrics is lower than the threshold (sharpness < 0.6, contrast < 0.5, signal-to-noise ratio < 30 dB, dynamic range < 0.7), the system automatically adjusts the corresponding parameters and triggers the re-acquisition process.

[0196] Example 10: Quality Closed-Loop Control Method

[0197] This example details the quality closed-loop control method, which feeds back the defect detection results into production control and product improvement to achieve closed-loop management of quality.

[0198] Quality closed-loop control first records the defect detection results of each batch of endoscopic sampling forceps, including defect type, location, size, and distribution characteristics. The detection results are aggregated by batch to generate a batch quality report, which contains statistical information such as defect rate, defect type distribution, severity distribution, etc., providing a data basis for quality trend analysis.

[0199] Next, analyze the temporal and spatial distribution patterns of defects to identify potential systematic problems. Through time series analysis, identify the fluctuation trend and periodic changes of the defect rate; through spatial distribution analysis, identify high-incidence defect areas and defect clustering phenomena; through correlation analysis, identify the correlation and co-occurrence patterns between different defect types. These analysis results help to discover systematic problems in the production process, such as equipment wear, operation deviation, and material fluctuation.

[0200] Then, based on the analysis results, generate improvement suggestions for the production process, including material selection, processing parameters, assembly process, and quality inspection standards. The improvement suggestions adopt PDCA (Plan-Do-Check-Act) cycle management, and each suggestion contains clear improvement goals, implementation steps, verification methods, and expected effects. For example, when an increase in the number of loose hinge defects is detected, the system may suggest adjusting the hinge assembly torque or replacing the assembly tool, and provide specific parameter adjustment ranges and verification schemes.

[0201] Next, associate the detection results with the patient safety risk level to establish a risk warning mechanism for endoscopic sampling forceps defects. The risk level is divided into five levels: extremely low (only affecting appearance), low (slightly affecting function but not affecting use), medium (affecting function but acceptable), high (significantly affecting function and requiring rework), and extremely high (possibly causing a safety accident and must be scrapped). The system automatically evaluates the risk level according to the defect type, location, and severity, and triggers an automatic warning when high-risk and extremely high-risk situations occur to ensure that problems are handled in a timely manner.

[0202] Finally, regularly evaluate and optimize the performance of the detection system, including monitoring and improving the detection rate, false alarm rate, and processing speed. The system performance evaluation uses confusion matrix analysis to calculate indicators such as the detection rate, precision rate, and F1 score for each type of defect, identifying the advantages and disadvantages of the system; at the same time, monitor the real-time processing ability of the system to ensure that it meets the production line speed requirements. Based on the evaluation results, the system regularly updates the model parameters, optimizes the algorithm structure, and adjusts the detection strategy to continuously improve the detection performance.

[0203] Through the above quality closed-loop control method, the present invention not only achieves accurate detection of the defects of the endoscopic sampling forceps, but also establishes a complete closed-loop from detection to improvement, providing a full-process solution for the quality management of medical devices.

[0204] The following shows the technical effects of the present invention through a practical application case.

[0205] Deploy the defect detection system of the present invention on the production line of endoscopic sampling forceps of a certain medical device manufacturing enterprise, and compare the detection effects of the traditional manual detection method and the method of the present invention. The test is carried out on 500 samples of endoscopic sampling forceps, and the samples contain various types of defects, including surface scratches, jaw deformation, hinge looseness, coating peeling, and cracks.

[0206] As Figure 8 shown, the comparison results between the method of the present invention and the manual detection method show that:

[0207] 1. Detection rate: The overall defect detection rate of the method of the present invention reaches 96.8%, while that of manual detection is 82.3%. Especially in the detection of micro-defects (area < 0.05 mm²), the detection rate of the method of the present invention is 94.2%, while that of manual detection is only 65.7%;

[0208] 2. Accuracy rate: The defect classification accuracy rate of the method of the present invention is 93.5%, and that of manual detection is 89.1%;

[0209] 3. Consistency: The consistency of the detection results of different operators using the method of the present invention reaches 95.7%, while the consistency of manual detection by different personnel is only 78.2%;

[0210] 4. Efficiency: The average detection time of the method of the present invention is 0.25 seconds per piece, while manual detection requires an average of 15 seconds per piece, and the efficiency is increased by 60 times;

[0211] 5. Interpretability: The method of the present invention can provide the location, type, severity, and cause analysis of the defects, supporting production decision-making and quality improvement.

[0212] This application example fully demonstrates the remarkable technical effects of the present invention in the field of defect detection of endoscopic sampling forceps, and verifies its application value in improving the quality control level of medical devices.

[0213] In summary, the full-process defect detection method for endoscopic sampling forceps based on machine vision provided by the present invention realizes high-precision and fully automatic defect detection of endoscopic sampling forceps through innovative technologies such as multi-scale geometric perception feature extraction, part-adaptive attention allocation, defect-sensitive residual learning, medical-grade precise segmentation, and expert knowledge-guided self-calibration, meets the strict quality control requirements of medical devices, and has significant practical value and promotion prospects.

[0214] The above are only the preferred embodiments of the present invention, and do not limit the patent protection scope of the present invention. Any equivalent structural transformation made by using the content of the specification and drawings of the present invention under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A full-process defect detection method for endoscopic sampling forceps based on machine vision, characterized in that: include: Acquire image data of an endoscopic sampling forceps; Processing the image data through a multi-scale geometry-aware feature extractor to generate a multi-scale feature atlas; Based on the multi-scale feature atlas, three key parts of the endoscope sampling forceps, namely the jaws, hinges and rod body, are identified by a part-adaptive attention module to generate a weighted attention feature map; Based on the weighted attention feature map, features related to the defect type are enhanced through a defect-sensitive residual network to generate a defect-sensitive feature map; Based on the defect sensitive feature map, generating an endoscopic sampling forceps defect segmentation result through a medical-grade precise segmentation module; Based on the defect segmentation result, the uncertainty of the detection result is evaluated through a self-calibration mechanism guided by expert knowledge, and when the uncertainty exceeds a preset threshold, the detection result is calibrated based on an endoscopic sampling forceps defect ontology knowledge base; The multi-scale geometry-aware feature extractor processes the image data, specifically comprising: Construct a geometry-aware convolution kernel consisting of a standard convolution branch, a depth-wise separable convolution branch, and a point-wise convolution branch; Based on the geometric features of different components of the endoscopic sampling forceps, the weight coefficient of the geometry-aware convolution kernel is adaptively adjusted by the following formula: ， in, is the weight coefficient of the i-th branch; GAP is the global average pooling operation; FC is the fully connected layer; is the output feature map of the i-th branch; Softmax is the normalization function; Through the angle-sensitive pooling layer, the directional features of the three key parts of the jaw, hinge and rod are captured. The formula is: , Among them, ASP is the angle-sensitive pooling operation; F is the input feature map; θ is the rotation angle, and the value range is [0°, 45°, 90°, 135°]; is the rotation transformation matrix of the angle; MaxPool is the maximum pooling operation; Represents a matrix multiplication operation; Generating multi-scale feature atlas ,in: Indicates the resolution of the original image times the feature map; s is the scale factor; The defect-sensitive residual network specifically includes: Aiming at the unique defect types of endoscopic sampling forceps, including surface scratches, jaw deformation, hinge loosening, coating peeling and cracks, a special residual unit is designed, and the formula is: ， in, is the output feature map, is the input feature map, is the standard residual path, for defect-sensitive skip connections; Design specialized residual paths for different types of defects: For surface scratches, a multi-directional Gabor filter is used to enhance texture features; For jaw deformation, residual connections with shape prior constraints are used; For hinge looseness, the temporal difference module is combined to capture dynamic features; For coating peeling, a reflectivity analysis unit is introduced to enhance the material change characteristics; For cracks, direction-sensitive dilated convolution is used to amplify the fine structure; Generate defect-sensitive feature maps by cascading multiple residual units ; The self-calibration mechanism guided by expert knowledge specifically includes: Construct an ontological knowledge base of endoscopic sampling forceps defects, including defect cases annotated by experts; The uncertainty based on the attention map is calculated by the following formula: , Among them, U is the uncertainty measurement value of the detection result; is the defect category probability; log is the natural logarithm function, λ is the balance parameter; For the attention map The gradient of · represents the L1 norm; ∑ represents the summation operation of all defect categories; When uncertainty Exceeding the preset threshold When , the expert knowledge-guided self-calibration is performed by the following formula: , in, is the defect category probability after calibration, is the original predicted probability, is the dynamic weight coefficient, Knowledge-based The expert prior probability of Generates calibrated endoscopic forceps defect inspection reports including defect type, location, severity and repair recommendations; The construction of the endoscopic sampling forceps defect ontology knowledge base specifically includes: Collect samples of various defects that occur during the production, use and maintenance of endoscopic sampling forceps; Invite at least three medical device quality inspection experts with more than five years of experience to mark the defective samples; Create a structured description for each defect sample, including defect type, appearance characteristics, cause analysis, and impact assessment; Establish defect similarity metrics for quick retrieval of similar defect cases; The knowledge base content is updated regularly to include newly discovered defect types and characteristics.

2. The method according to claim 1, characterized in that The part-adaptive attention module identifies key parts of the endoscopic sampling forceps, specifically including: The channel-space attention map of the three key parts of the endoscope sampling forceps jaws, hinges and rods is calculated using the formula: ， in, For the part The attention map, Jaw, hinge, lever For the part feature map; ChannelPool is the channel pooling operation, and SpatialPool is the spatial pooling operation; and is the learnable weight parameter matrix, ReLU is the rectified linear unit activation function, is the sigmoid activation function, Conv is the convolution operation, and · represents the matrix multiplication or element-by-element multiplication operation; The inter-part relationship model is established to capture the synergistic relationship between the various parts of the sampling clamp. The formula is: ， in, For the part and The relationship characteristics between is the tanh activation function, and Part and parts The feature representation of represents the feature connection operation, A matrix of weight parameters for modeling relationships; is the bias parameter vector; is the tanh activation function; According to the defect risk level of the part, different weights are assigned to the feature map to generate a weighted attention feature map .

3. The method according to claim 1, characterized in that The medical-grade precise segmentation module specifically includes: Design a reverse attention-guided segmentation network to enhance defect boundary extraction capabilities through foreground-background contrast; The medical-specific boundary enhancement loss function is used, and the formula is: , in, is the pixel weight, is the defect category probability, Pixel The distance to the nearest defect boundary, is the boundary attention coefficient; The defect boundary is iteratively optimized through a multi-stage boundary refinement strategy, and the formula is: , in, is the boundary of the tth iteration; t is the number of iterations; For the The boundary of the iteration; is the defect sensitive feature map; Refine is the boundary refining function; The rough segmentation results, boundary refinement results and multi-scale fusion results are combined to generate the final endoscopic sampling forceps defect segmentation results.

4. The method according to any one of claims 1 to 3, characterized in that: It also includes data preprocessing steps: Perform illumination normalization on the acquired endoscopic sampling forceps images to eliminate the influence of uneven illumination; Adopt the specular reflection suppression algorithm based on reflection characteristics to reduce the interference of metal surface highlights on detection; According to the geometric characteristics of the endoscopic sampling forceps, the image is perspective corrected to ensure the proportional consistency of key parts; Enhance image details through contrast-limited adaptive histogram equalization; Multi-scale noise suppression is performed on the image while retaining the subtle features of the defects.

5. The method according to claim 1, characterized in that Before acquiring image data of the endoscopic sampling forceps, the acquisition strategy optimization step is also included: Based on the 3D CAD model of the endoscopic sampling forceps, determine the key detection viewing angle and optimal imaging distance; Design specific lighting layout, including the position, angle and intensity of main and auxiliary light sources; Develop corresponding image acquisition parameters for different types of potential defects, including exposure time, aperture size, and focus position; The quality of the acquired images is evaluated. When the image quality is lower than the preset standard, the acquisition parameters are automatically adjusted and the image is re-acquired.

6. The method according to claim 1, after completing the defect detection of the endoscopic sampling forceps, further comprises a quality closed-loop control step: Record the defect detection results of each batch of endoscopic sampling forceps, including defect type, location, size and distribution characteristics; Analyze the temporal and spatial distribution patterns of defects to identify potential systemic problems; Generate improvement suggestions for production processes, including material selection, processing parameters, assembly processes and quality inspection standards; Correlate the test results with the patient safety risk level and establish a risk warning mechanism for endoscopic sampling forceps defects; Regularly evaluate and optimize the performance of the detection system, including monitoring and improvement of detection rate, false alarm rate and processing speed.

Images