Endoscope sampling clamp full-process defect detection method based on machine vision

Abstract

The invention relates to the technical field of quality detection of medical instruments, in particular to a full-process defect detection method for endoscope sampling forceps based on machine vision, and provides a method for processing image data through a multi-scale geometric perception feature extractor, generating a feature image set, identifying key parts by utilizing a part self-adaptive attention module, and detecting the defects of the endoscope sampling forceps in the whole process. The defect sensitive residual network is used for enhancing related defect features, the medical-level precise segmentation module realizes defect segmentation, and the self-calibration mechanism guided by expert knowledge and the defect ontology knowledge base are combined, so that the method can effectively evaluate and calibrate the detection uncertainty and ensure the detection precision, is high in adaptability, can efficiently extract defect features of different scales, and has a good application prospect. And the quality detection efficiency of the endoscope sampling forceps is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical device quality inspection, and in particular to a full-process defect inspection method for endoscopic sampling forceps based on machine vision. Background Art

[0002] As a common minimally invasive medical device, endoscopic sampling forceps plays a vital role in clinical diagnosis and treatment. Since they directly contact human tissue and perform biopsy operations, their quality is closely related to patient safety. Traditional endoscopic sampling forceps quality inspection mainly relies on manual visual inspection, which has problems such as low efficiency, inconsistent standards, easy fatigue and difficulty in detecting minor defects.

[0003] At present, there are some defect detection methods based on machine vision in the industrial field. For example, Chinese patent CN111681232B discloses a "method for detecting industrial welding image defects based on semantic segmentation", which realizes automatic detection of welding defects through deep learning. However, this type of method has the following limitations:

[0004] 1. The application scenario is single, only for industrial welding images, and cannot adapt to the special structure and material characteristics of medical devices such as endoscopic sampling forceps;

[0005] 2. Lack of differentiated processing capabilities for multiple parts of the device, and inability to conduct targeted testing based on the importance and risk level of different parts;

[0006] 3. The accuracy does not meet the high standards of medical devices, especially in the detection of minor defects, there is a risk of missed detection;

[0007] 4. Lack of expert knowledge guidance and calibration mechanism, poor interpretability of test results, and difficulty in supporting medical device quality control decisions;

[0008] 5. Unable to adapt to the changes in defect characteristics of endoscopic sampling forceps at different stages of their life cycle.

[0009] With the increasingly stringent regulatory requirements in the medical device industry, especially the higher requirements of regulatory agencies such as FDA and NMPA for medical device production good management practices (GMP), there is an urgent need for a high-precision, full-process defect detection method specifically for endoscopic sampling forceps. Summary of the invention

[0010] In response to the above technical problems, the present invention provides a full-process defect detection method for endoscopic sampling forceps based on machine vision, aiming to achieve high-precision and fully automatic defect detection of endoscopic sampling forceps to meet the strict quality control requirements of medical-grade equipment.

[0011] The present invention proposes a full-process defect detection method for endoscopic sampling forceps based on machine vision, including:

[0012] Acquire image data of endoscopic sampling forceps;

[0013] Processing the image data through a multi-scale geometry-aware feature extractor to generate a multi-scale feature atlas;

[0014] Based on the multi-scale feature atlas, three key parts of the endoscope sampling forceps, namely the jaws, hinges and rod body, are identified by a part-adaptive attention module to generate a weighted attention feature map;

[0015] Based on the weighted attention feature map, features related to the defect type are enhanced through a defect-sensitive residual network to generate a defect-sensitive feature map;

[0016] Based on the defect sensitive feature map, generating an endoscopic sampling forceps defect segmentation result through a medical-grade precise segmentation module;

[0017] Based on the defect segmentation result, the uncertainty of the detection result is evaluated through a self-calibration mechanism guided by expert knowledge, and when the uncertainty exceeds a preset threshold, the detection result is calibrated based on an endoscopic sampling forceps defect ontology knowledge base.

[0018] Preferably, the multi-scale geometry-aware feature extractor processes the image data, specifically comprising:

[0019] Construct a geometry-aware convolution kernel consisting of a standard convolution branch, a depth-wise separable convolution branch, and a point-wise convolution branch;

[0020] Based on the geometric features of different components of the endoscopic sampling forceps, the weight coefficient of the geometry-aware convolution kernel is adaptively adjusted by the following formula:

[0021] ,

[0022] in, is the weight coefficient of the i-th branch; GAP is the global average pooling operation; FC is the fully connected layer; is the output feature map of the i-th branch; Softmax is the normalization function;

[0023] Through the angle-sensitive pooling layer, the directional features of the three key parts of the jaw, hinge and rod are captured. The formula is:

[0024] ,

[0025] Among them, ASP is the angle-sensitive pooling operation; F is the input feature map; θ is the rotation angle, and the value range is [0°, 45°, 90°, 135°]; is the rotation transformation matrix of the angle; MaxPool is the maximum pooling operation; Represents a matrix multiplication operation;

[0026] Generating multi-scale feature atlas ,in: Indicates the resolution is the original image times the feature map; s is the scale factor.

[0027] Preferably, the part-adaptive attention module identifies key parts of the endoscopic sampling forceps, specifically including:

[0028] The channel-space attention map of the three key parts of the endoscope sampling forceps jaws, hinges and rods is calculated using the formula:

[0029] ,

[0030] in, For the part The attention map, Jaw, hinge, lever For the part feature map; ChannelPool is the channel pooling operation, and SpatialPool is the spatial pooling operation; and is the learnable weight parameter matrix, ReLU is the rectified linear unit activation function, is the sigmoid activation function, Conv is the convolution operation, and · represents the matrix multiplication or element-by-element multiplication operation;

[0031] The inter-part relationship model is established to capture the synergistic relationship between the various parts of the sampling clamp. The formula is:

[0032] ,

[0033] in, For the part and The relationship characteristics between is the tanh activation function, and Part and parts The feature representation of represents the feature connection operation, A matrix of weight parameters for modeling relationships; is the bias parameter vector; is the tanh activation function;

[0034] According to the defect risk level of the part, different weights are assigned to the feature map to generate a weighted attention feature map .

[0035] Preferably, the defect-sensitive residual network specifically includes:

[0036] Aiming at the unique defect types of endoscopic sampling forceps, including surface scratches, jaw deformation, hinge loosening, coating peeling and cracks, a special residual unit is designed, and the formula is:

[0037] ,

[0038] in, is the output feature map, is the input feature map, is the standard residual path, for defect-sensitive skip connections;

[0039] Design specialized residual paths for different types of defects:

[0040] For surface scratches, a multi-directional Gabor filter is used to enhance texture features;

[0041] For jaw deformation, residual connections with shape prior constraints are used;

[0042] For hinge looseness, the temporal difference module is combined to capture dynamic features;

[0043] For coating peeling, a reflectivity analysis unit is introduced to enhance the material change characteristics;

[0044] For cracks, direction-sensitive dilated convolution is used to amplify the fine structure;

[0045] Generate defect-sensitive feature maps by cascading multiple residual units .

[0046] Preferably, the medical-grade precise segmentation module specifically comprises:

[0047] Design a reverse attention-guided segmentation network to enhance defect boundary extraction capabilities through foreground-background contrast;

[0048] The medical-specific boundary enhancement loss function is used, and the formula is:

[0049] ,

[0050] in, is the pixel weight, is the defect category probability, Pixel The distance to the nearest defect boundary, is the boundary attention coefficient;

[0051] The defect boundary is iteratively optimized through a multi-stage boundary refinement strategy, and the formula is:

[0052] ,

[0053] in, is the boundary of the tth iteration; t is the number of iterations; For the The boundary of the iteration; is the defect sensitive feature map; Refine is the boundary refining function;

[0054] The rough segmentation results, boundary refinement results and multi-scale fusion results are combined to generate the final endoscopic sampling forceps defect segmentation results.

[0055] Preferably, the expert knowledge-guided self-calibration mechanism specifically includes:

[0056] Construct an ontological knowledge base of endoscopic sampling forceps defects, including defect cases annotated by experts;

[0057] The uncertainty based on the attention map is calculated by the following formula:

[0058] ,

[0059] Among them, U is the uncertainty measurement value of the detection result; is the defect category probability; log is the natural logarithm function, λ is the balance parameter; For the attention map The gradient of · represents the L1 norm; ∑ represents the summation operation of all defect categories;

[0060] When uncertainty Exceeding the preset threshold When , the expert knowledge-guided self-calibration is performed by the following formula:

[0061] ,

[0062] in, is the defect category probability after calibration, is the original predicted probability, is the dynamic weight coefficient, Knowledge-based The expert prior probability of

[0063] Generates calibrated endoscopic forceps defect inspection reports including defect type, location, severity, and repair recommendations.

[0064] Preferably, the construction of the endoscopic sampling forceps defect ontology knowledge base specifically includes:

[0065] Collect samples of various defects that occur during the production, use and maintenance of endoscopic sampling forceps;

[0066] Invite at least three medical device quality inspection experts with more than five years of experience to mark the defective samples;

[0067] Create a structured description for each defect sample, including defect type, appearance characteristics, cause analysis, and impact assessment;

[0068] Establish defect similarity metrics for quick retrieval of similar defect cases;

[0069] The knowledge base content is updated regularly to include newly discovered defect types and characteristics.

[0070] Preferably, the data preprocessing step is also included:

[0071] Perform illumination normalization on the acquired endoscopic sampling forceps images to eliminate the influence of uneven illumination;

[0072] Adopt the specular reflection suppression algorithm based on reflection characteristics to reduce the interference of metal surface highlights on detection;

[0073] According to the geometric characteristics of the endoscope sampling forceps, the image is perspective corrected to ensure the proportional consistency of key parts;

[0074] Enhance image details through contrast-limited adaptive histogram equalization;

[0075] Multi-scale noise suppression is performed on the image while retaining the subtle features of the defects.

[0076] Preferably, before acquiring the image data of the endoscopic sampling forceps, an acquisition strategy optimization step is also included:

[0077] Based on the 3D CAD model of the endoscopic sampling forceps, determine the key detection viewing angle and optimal imaging distance;

[0078] Design specific lighting layout, including the position, angle and intensity of main and auxiliary light sources;

[0079] Develop corresponding image acquisition parameters for different types of potential defects, including exposure time, aperture size, and focus position;

[0080] The quality of the acquired images is evaluated. When the image quality is lower than the preset standard, the acquisition parameters are automatically adjusted and the image is re-acquired.

[0081] According to the described method, after completing the defect detection of the endoscopic sampling forceps, the quality closed-loop control step is also included:

[0082] Record the defect detection results of each batch of endoscopic sampling forceps, including defect type, location, size and distribution characteristics;

[0083] Analyze the temporal and spatial distribution patterns of defects to identify potential systemic problems;

[0084] Generate improvement suggestions for production processes, including material selection, processing parameters, assembly processes and quality inspection standards;

[0085] Correlate the test results with the patient safety risk level and establish a risk warning mechanism for endoscopic sampling forceps defects;

[0086] Regularly evaluate and optimize the performance of the detection system, including monitoring and improvement of detection rate, false alarm rate and processing speed.

[0087] The present invention forms a complete information processing pipeline through five progressive processing modules. The output of each module provides more refined features for the next module, thus realizing the step-by-step refinement of endoscopic sampling forceps defect detection. The method has the following beneficial effects:

[0088] 1. Strong adaptability: Through the multi-scale geometric perception feature extractor, it can adapt to the geometric characteristics of different parts of the endoscope sampling forceps and effectively extract defect features of various scales;

[0089] 2. Accurate positioning: The part-adaptive attention module can allocate attention weights according to the importance of different parts of the endoscope sampling forceps, so as to focus on high-risk areas;

[0090] 3. Defect sensitivity: Design a special feature enhancement mechanism to improve detection accuracy for typical defects such as surface scratches, jaw deformation, hinge loosening, coating peeling and cracks of endoscope sampling forceps;

[0091] 4. Medical-grade accuracy: Through precise boundary segmentation and multi-stage refinement, it meets the high-precision detection requirements of medical devices and can identify defects as small as 0.01mm²;

[0092] 5. Strong interpretability: Combined with the expert knowledge base and self-calibration mechanism, it provides interpretable test results and quality improvement suggestions to support medical device production decisions;

[0093] 6. Full-process adaptation: covering the entire life cycle of endoscopic sampling forceps from production, use to maintenance, to achieve closed-loop quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1It is an overall flow chart of the whole process defect detection method of endoscopic sampling forceps based on machine vision provided by the present invention;

[0095] Figure 2 It is a schematic diagram of the structure of the multi-scale geometric perception feature extractor provided by the present invention;

[0096] Figure 3 is a schematic diagram of the structure of the part-adaptive attention module provided by the present invention;

[0097] Figure 4 It is a schematic diagram of the structure of the defect-sensitive residual network provided by the present invention;

[0098] Figure 5 It is a structural schematic diagram of the medical-grade precision segmentation module provided by the present invention;

[0099] Figure 6 is a flow chart of the expert knowledge guided self-calibration mechanism provided by the present invention;

[0100] Figure 7 It is a structural schematic diagram of the endoscope sampling forceps defect ontology knowledge base provided by the present invention;

[0101] Figure 8 It is a comparison chart of defect detection effects in an application example of the present invention. DETAILED DESCRIPTION

[0102] Please refer to the attached Figure 1-8 The present invention will be described in detail below in conjunction with the accompanying drawings and embodiments. These embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0103] Example 1: Overall process of the whole process defect detection method of endoscopic sampling forceps based on machine vision

[0104] Reference Figure 1 The present invention provides a method for detecting defects in the whole process of endoscopic sampling forceps based on machine vision, which mainly includes the following steps:

[0105] First, obtain the image data of the endoscopic sampling forceps. Preferably, a high-resolution industrial camera is used to obtain the image of the endoscopic sampling forceps, with a resolution of not less than 2048×1536 pixels to ensure that the characteristics of tiny defects are captured. During the image acquisition process, a diffuse light source is used to avoid strong reflections on the metal surface, and a fixed fixture is used to ensure the consistency of the position and angle of the sampling forceps.

[0106] Next, the image data is processed by a multi-scale geometry-aware feature extractor to generate a multi-scale feature atlas. The feature extractor is designed with a dedicated geometry-aware convolution kernel that can adapt to the geometric characteristics of different parts of the endoscope sampling forceps, such as the jaws, hinges, and rods, and effectively extract defect features of various scales.

[0107] Then, based on the multi-scale feature atlas, the three key parts of the endoscope sampling forceps, namely the jaws, hinges and rod body, are identified through the part-adaptive attention module to generate a weighted attention feature map. This module assigns attention weights according to the defect risk level of different parts to achieve focus on high-risk areas.

[0108] Next, based on the weighted attention feature map, the defect-sensitive residual network is used to enhance the features related to the defect type to generate a defect-sensitive feature map. The network designs a special feature enhancement mechanism for typical defects such as surface scratches, jaw deformation, hinge loosening, coating peeling and cracks of endoscopic sampling forceps.

[0109] Subsequently, based on the defect sensitive feature map, the defect segmentation result of the endoscope sampling forceps is generated by the medical-grade precise segmentation module. This module meets the high-precision detection requirements of medical devices through precise boundary segmentation and multi-stage refinement, and can identify defects as small as 0.01mm².

[0110] Finally, based on the defect segmentation results, the uncertainty of the detection results is evaluated through an expert knowledge-guided self-calibration mechanism, and when the uncertainty exceeds a preset threshold, the detection results are calibrated based on the endoscopic sampling forceps defect ontology knowledge base. This mechanism combines the expert knowledge base and the self-calibration algorithm to provide explainable detection results and quality improvement suggestions.

[0111] Embodiment 2: Implementation method of multi-scale geometric perception feature extractor

[0112] Reference Figure 2 , this embodiment describes in detail the implementation method of the multi-scale geometry-aware feature extractor, which can effectively handle the feature extraction problem under the complex geometric structure of the endoscopic sampling forceps.

[0113] The multi-scale geometry-aware feature extractor first constructs a geometry-aware convolution kernel (GACK) consisting of three branches: a standard convolution branch (3×3), a depth-wise separable convolution branch (5×5), and a point convolution branch (1×1). The standard convolution branch mainly captures local structural features, the depth-wise separable convolution branch expands the receptive field to obtain a wider range of contextual information, and the point convolution branch achieves feature integration through channel dimension fusion.

[0114] Based on the geometric features of different components of the endoscopic sampling forceps, the feature extractor dynamically adjusts the importance of each branch through an adaptive weight adjustment mechanism. The weight coefficient calculation formula is:

[0115] ,

[0116] in, is the weight coefficient of the i-th branch, indicating the importance of the branch in feature extraction; is the output feature map of the i-th branch; GAP is the global average pooling operation, which is used to compress the feature map into a single feature vector; FC is the fully connected layer, which is used to map the feature vector to the weight coefficient; Softmax is the normalization function, which ensures that the sum of all branch weights is 1.

[0117] In order to better capture the directional features of the key parts of the textured endoscope sampling forceps, the feature extractor also introduces an angle-sensitive pooling layer (ASP). This layer enhances directional sensitivity by applying the maximum pooling operation at different angles. The formula is:

[0118] ,

[0119] in, It is an angle-sensitive pooling operation; is the input feature map; θ is the rotation angle, ranging from {0°, 45°, 90°, 135°}; is the rotation transformation matrix of angle θ; MaxPool is the maximum pooling operation; represents the matrix multiplication operation. This angle-sensitive design is particularly suitable for detecting directional defects on the sampling clamp, such as surface scratches and cracks.

[0120] The output of the feature extractor is a multi-scale feature atlas ,in Indicates the resolution is the original image The multi-scale feature design enables the system to focus on large-scale structures (such as overall deformation of the jaws) and small-scale details (such as tiny surface scratches) at the same time.

[0121] In practical applications, the feature extractor uses 64 standard convolution kernels, 48 ​​depth convolution kernels and 32 point convolution kernels, and the activation function uses the Swish function. sigmoid ,in The initial learning rate is set to 0.002 and decays by 10% every 25 training cycles to ensure training stability.

[0122] Embodiment 3: Implementation method of part-adaptive attention module

[0123] Reference Figure 3 , This embodiment describes in detail the implementation method of the site adaptive attention module, which can adaptively allocate attention resources according to the importance and risk level of different parts of the endoscopic sampling forceps.

[0124] The part-adaptive attention module first identifies the three key parts of the endoscope sampling forceps: the jaws, hinges, and rods. The jaws directly contact the tissue for sampling and are the most critical part of the function; the hinge controls the opening and closing of the jaws and is the core of the mechanical structure; the rod connects the operating end and the working end to transmit the operating force.

[0125] For these three key parts, the module calculates the channel-space joint attention map, the formula is:

[0126] ,

[0127] in, For the part The attention map indicates the degree of attention paid to the features of the part; Indicates the key parts of the endoscopic sampling forceps. Jaw, hinge, lever For the part feature map; ChannelPool is a channel pooling operation, which is used to aggregate channel dimension features; SpatialPool is a spatial pooling operation, which is used to aggregate spatial dimension features; and is a learnable weight parameter matrix; ReLU is the rectified linear unit activation function, defined as ReLU is the sigmoid activation function, defined as Conv is a convolution operation; · represents a matrix multiplication or an element-by-element multiplication operation.

[0128] In addition, since the defects of endoscopic sampling forceps are often related (for example, loose hinges may cause deformation of the jaws), the module designs a modeling unit for the relationship between parts, and the formula is:

[0129] ,

[0130] in, For the part and parts The relationship characteristics between the two parts indicate the degree of mutual influence between them; and Part and parts The feature representation of Represents the feature connection operation, which concatenates two feature vectors in the channel dimension. A matrix of weight parameters for modeling relationships; is the bias parameter vector; is the tanh activation function, defined as .

[0131] Based on risk assessment, different parts are assigned different initial weights: the jaw area has an initial weight of 1.5 (high risk area), the hinge area has an initial weight of 1.2 (medium risk area), and the rod area has an initial weight of 1.0 (low risk area). These initial weights will be automatically adjusted according to the actual sample distribution during training.

[0132] In practical applications, the attention channel compression rate r is set to 16 to balance computational efficiency and expression ability. The module finally outputs the weighted attention feature map , as the input of the defect-sensitive residual network in the next stage.

[0133] Example 4: Implementation method of defect-sensitive residual network

[0134] Reference Figure 4 ,This embodiment describes in detail the implementation method of a defect-sensitive residual network, which specifically enhances the typical defect features of the endoscope sampling forceps to improve the detection accuracy.

[0135] The defect-sensitive residual network designs specialized residual units for the five main defects of endoscope sampling forceps (surface scratches, jaw deformation, hinge loosening, coating peeling and cracks). The basic structure of the residual unit is:

[0136] ,

[0137] in, is the output feature map of the residual unit; is the input feature map of the residual unit; is the standard residual path, which means that through the weight parameter For input Functions that perform nonlinear transformations; is a defect-sensitive jump connection, which indicates a feature enhancement path designed for a specific defect type; + indicates an element-by-element addition operation. Different from the traditional residual network, this network adds a defect-sensitive jump connection. , the formula is:

[0138] ,

[0139] in, is the output of the defect-sensitive skip connection; is a defect type correlation function, which is used to enhance the feature channels related to a specific defect type; is the activation function; Conv is the convolution operation; · represents the element-by-element multiplication operation.

[0140] For different types of defects, the network designs specialized residual paths:

[0141] For surface scratches, a multi-directional Gabor filter is used to enhance the texture features. The Gabor filter parameters are set to 8 directions and 3 scales to effectively capture the scratch features with strong directionality.

[0142] For jaw deformation, a residual connection with shape prior constraints is introduced. By calculating the shape difference with the standard jaw template, the characteristic response of the deformation area is enhanced, making the network more sensitive to the deformation characteristics.

[0143] For hinge looseness, the time difference module is combined to capture dynamic features. By analyzing the motion characteristics of the hinge during opening and closing, abnormal motion patterns can be detected, which is crucial for detecting functional defects.

[0144] For coating peeling, the reflectivity analysis unit is introduced to enhance the material change characteristics. By analyzing the reflection characteristics at different incident angles, the coating state changes are identified and the normal surface and peeling area are effectively distinguished.

[0145] For cracks, direction-sensitive dilated convolution is used to amplify the subtle structure. The dilation rate sequence is set to {1, 2, 4, 8, 4, 2, 1}, which expands the receptive field without increasing the number of parameters and enhances the perception of subtle cracks.

[0146] The network contains 8 residual blocks, the defect type encoding dimension is 64, and the jump connection uses a combination of additive and multiplicative gating mechanisms to simultaneously retain the original features and enhance the defect features. Network output defect sensitive feature map , the feature dimension is 256, providing high-quality feature representation for subsequent accurate segmentation.

[0147] Example 5: Implementation method of medical-grade precise segmentation module

[0148] Reference Figure 5 This embodiment describes in detail the implementation method of the medical-grade precise segmentation module, which realizes pixel-level precise segmentation according to the high-precision requirements of medical devices.

[0149] The medical-grade precise segmentation module first designs a reverse attention-guided segmentation network to enhance the defect boundary extraction capability through foreground-background contrast. The network not only focuses on the defect area itself, but also emphasizes the boundary features between the defect and the normal area to improve segmentation accuracy.

[0150] In order to better capture defect boundaries, the module introduces a medical-specific boundary enhancement loss function:

[0151] ,

[0152] in: Enhance the loss function value for the boundary; is the pixel weight, which reflects the importance of the pixel in the segmentation task; is the defect category probability, which indicates the predicted probability that the pixel belongs to the defect category; is the distance from the pixel to the nearest defect boundary; is the boundary attention coefficient, set to 0.8; Indicates the sum operation of all pixels. This loss function design makes the network pay more attention to the boundary area during training and improves the accuracy of boundary positioning.

[0153] The segmentation process is divided into three stages: first, a rough segmentation is performed to obtain a preliminary defect area; then the defect boundary is iteratively optimized through boundary refinement, and the formula is:

[0154] ,

[0155] in, is the boundary of the tth iteration; t is the number of iterations; For the The boundary of the iteration; It is a defect sensitive feature map that provides the feature information required for boundary refinement. Refine is a boundary refinement function that uses wavelet transform to enhance boundary features. Boundary refinement is iterated 3 times, and each iteration further improves the accuracy of the boundary. Finally, through multi-scale fusion, the segmentation results at different resolutions are integrated to generate the final segmentation result.

[0156] In actual applications, the boundary judgment threshold is set to 0.65, and the wavelet transform adopts 4-level decomposition to balance detail retention and noise suppression. At the same time, according to the requirements of medical device standards, the minimum defect area that the system can identify is 0.01mm², which meets the medical-level detection accuracy requirements.

[0157] The segmentation module finally outputs a pixel-level defect segmentation map M, with a value range of {0, 1, 2, 3, 4, 5}, where 0 represents the background and 1 to 5 represent five types of defects: surface scratches, jaw deformation, hinge looseness, coating peeling, and cracks.

[0158] Embodiment 6: Implementation method of self-calibration mechanism guided by expert knowledge

[0159] Reference Figure 6,This embodiment describes in detail the implementation method of the ,expert knowledge guided self-calibration mechanism which combines expert ,knowledge and uncertainty assessment to improve the reliability and ,interpretability of the detection results.

[0160] The self-calibration mechanism guided by expert knowledge first constructs an endoscope sampling forceps defect ontology knowledge base, which contains defect cases annotated by experts. Each defect case in the knowledge base contains structured information such as defect type, appearance characteristics, cause analysis, and impact assessment, providing expert-level knowledge support for the system.

[0161] The mechanism calculates the uncertainty of the detection results through an uncertainty evaluation module based on the attention map:

[0162] ,

[0163] Among them, U is the uncertainty measurement value of the detection result; is the defect category probability, indicating the probability that the pixel belongs to the i-th defect; log is the natural logarithm function, λ is the balance parameter, which controls the weight of the attention ladder item, and λ is set to 0.3; For the attention map The gradient of reflects the stability of attention distribution; · represents the L1 norm, which is calculated as the sum of the absolute values ​​of all elements; ∑ represents the summation operation for all defect categories. This uncertainty measure comprehensively considers the entropy of the predicted probability and the stability of attention, and can more comprehensively evaluate the reliability of the detection results.

[0164] When the uncertainty U exceeds the preset threshold θ (set to 0.75), the system triggers the self-calibration process. Self-calibration is guided by expert knowledge, and the formula is:

[0165] ,

[0166] in, is the defect category probability after calibration; is the original predicted probability; is the dynamic weight coefficient, which controls the influence of expert knowledge. It is initially set to 0.6 and gradually decreases as the number of samples increases; Knowledge-based The expert prior probability is obtained by retrieving cases similar to the previous sample in the knowledge base and extracting the probability distribution of expert annotations.

[0167] Dynamic weight coefficient The initial value is set to 0.6, and it gradually decreases as the number of samples increases, reflecting the process of gradually reducing the reliance on expert knowledge as the system accumulates experience. The minimum confidence level set by the system is 0.85, and test results below this value will be marked as requiring manual review.

[0168] The self-calibration mechanism also includes a closed-loop feedback function, which extracts features from high uncertainty areas, queries the knowledge base to generate calibration suggestions, and updates the model. The knowledge base is updated monthly to include newly discovered defect types and features to keep the knowledge base current.

[0169] Example 7: Method for constructing an endoscope sampling forceps defect ontology knowledge base

[0170] Reference Figure 7 ,This embodiment describes in detail the method of constructing the endoscope sampling forceps defect ontology knowledge base which ,provides expert knowledge support for the self-calibration mechanism.

[0171] The construction of the knowledge base of endoscopic sampling forceps defect ontology first collects various defect samples that occur during the production, use and maintenance of endoscopic sampling forceps. The collection channels include multiple sources such as production line inspection, clinical use feedback, and maintenance records to ensure the diversity and representativeness of the samples.

[0172] The collected defective samples are annotated by at least three medical device quality inspection experts with more than five years of experience. The annotation adopts a double-blind method, with each sample independently annotated by two experts, and then confirmed and arbitrated by a third expert to ensure the quality of the annotation.

[0173] Create a structured description for each defect sample, including the following:

[0174] Defect type: clearly classified into surface scratches, jaw deformation, hinge loosening, plating peeling or cracks;

[0175] Appearance characteristics: describe in detail the shape, size, location, color and other visual characteristics of the defect;

[0176] Cause analysis: explain the possible causes of defects, such as material problems, processing problems, wear and tear, etc.

[0177] Impact assessment: Evaluate the impact of defects on functionality and security, divided into five levels: very low, low, medium, high, and very high;

[0178] Treatment suggestions: Provide treatment solutions for this type of defects, such as rework, scrapping, restricted use, etc.

[0179] In order to achieve fast retrieval, the knowledge base establishes a defect similarity metric. The similarity calculation comprehensively considers appearance features, location information and context, and adopts a weighted cosine similarity calculation method. Preferably, the weight of appearance features is 0.5, the weight of location information is 0.3, and the weight of context is 0.2.

[0180] The knowledge base adopts a regular update mechanism, and the expert team reviews and incorporates newly discovered defect types and features every month. When more than 50 new samples are accumulated or new defects are discovered, the knowledge base update process is triggered. The update process includes four steps: data cleaning, feature extraction, expert annotation, and data integration.

[0181] After long-term accumulation, the knowledge base has included more than 240 defect cases, covering all kinds of defects in the entire life cycle of endoscopic sampling forceps, providing rich expert knowledge support for the self-calibration mechanism.

[0182] Embodiment 8: Data preprocessing method

[0183] This embodiment describes in detail a data preprocessing method, which provides high-quality input data for subsequent feature extraction and defect detection.

[0184] Data preprocessing first normalizes the acquired endoscopic sampling forceps image to eliminate the influence of uneven illumination. The multi-scale Retinex algorithm is used to decompose the image into reflection component and illumination component. Illumination normalization is achieved by adjusting the illumination component, and the defect information in the reflection component is retained.

[0185] Next, a specular reflection suppression algorithm based on reflection characteristics is used to reduce the interference of metal surface highlights on detection. The algorithm identifies and suppresses specular reflection areas by analyzing polarization characteristics, and restores image details blocked by highlights. Preferably, a dual threshold method is used to identify specular reflection areas, and the thresholds are set to 1.5 times and 2 times the average brightness of the image, respectively.

[0186] Then, according to the geometric characteristics of the endoscope sampling forceps, the images are perspective corrected to ensure the consistency of the proportions of key parts. The correction process is based on the pre-defined standard contour template of the sampling forceps, and the image correction is achieved through affine transformation, so that each batch of images has a consistent perspective and proportion.

[0187] Subsequently, contrast-limited adaptive histogram equalization (CLAHE) is used to enhance image details. The CLAHE algorithm parameters are set as follows: block size 8×8, contrast limiting factor 3.0, and distribution type Rayleigh distribution to enhance details while suppressing noise amplification.

[0188] Finally, multi-scale noise suppression is performed on the image while retaining the subtle features of the defects. A combination of bilateral filtering and guided filtering is used to suppress noise while retaining edge and texture information. Preferably, the spatial domain standard deviation of the bilateral filter is set to 3.0, the range standard deviation is set to 0.1, the guided filter window radius is set to 5, and the regularization parameter is set to 0.01.

[0189] After the above preprocessing steps, the endoscopic sampling forceps image with improved quality is obtained, which provides a good data basis for subsequent feature extraction and defect detection.

[0190] Example 9: Collection strategy optimization method

[0191] This embodiment describes in detail an acquisition strategy optimization method, which improves the quality of raw data by optimizing image acquisition parameters, thereby providing better input for subsequent processing.

[0192] The acquisition strategy optimization first determines the key detection viewing angle and the optimal imaging distance based on the 3D CAD model of the endoscope sampling forceps. Through virtual imaging simulation, the coverage of key parts such as the jaws, hinges, and rods at different viewing angles is analyzed to determine the optimal viewing angle combination, which usually includes the front view, side view, and 45° tilt view. Figure 3 The optimal imaging distance is determined by the image resolution and depth of field requirements. For a camera with a resolution of 2048×1536, the preferred imaging distance is 150mm-200mm.

[0193] Next, design a specific light source layout, including the position, angle and intensity of the main light source and auxiliary light source. The main light source uses a ring-shaped LED light source to provide the main lighting; the auxiliary light source is a point light source, which provides supplementary lighting from a specific angle to enhance the visibility of defects. The core principle of light source layout is that the main light source provides uniform basic lighting, and the auxiliary light source enhances the defect contrast through specific angle incidence. For metal surface inspection, the intensity of the main light source is set to 2-3 times that of the auxiliary light source, and the angle between the main light source and the object is 15°-30°.

[0194] Then, for different types of potential defects, corresponding image acquisition parameters are formulated, including exposure time, aperture size and focus position. For example, for surface scratches and cracks, a shorter exposure time (10-15ms), a smaller aperture (F8-F11) and precise focus are used to enhance edge sharpness; for jaw deformation and hinge looseness, a medium exposure time (20-30ms), a medium aperture (F5.6-F8) and panoramic focus are used to ensure the overall shape is clear; for coating peeling, a longer exposure time (30-40ms), a larger aperture (F4-F5.6) and surface focus are used to enhance the visibility of material texture.

[0195] Finally, the quality of the acquired images is evaluated. When the image quality is lower than the preset standard, the acquisition parameters are automatically adjusted and the image is re-acquired. The image quality evaluation indicators include clarity, contrast, noise level and dynamic range, which are quantitatively evaluated through gradient amplitude distribution, histogram uniformity, signal-to-noise ratio and grayscale distribution standard deviation. When any indicator is lower than the threshold (clarity <0.6, contrast <0.5, signal-to-noise ratio <30dB, dynamic range <0.7), the system automatically adjusts the corresponding parameters and triggers the re-acquisition process.

[0196] Example 10: Quality closed-loop control method

[0197] This embodiment describes in detail a closed-loop quality control method, which feeds back defect detection results to production control and product improvement to achieve closed-loop quality management.

[0198] The quality closed-loop control first records the defect detection results of each batch of endoscope sampling forceps, including defect type, location, size and distribution characteristics. The detection results are aggregated by batch to generate a batch quality report, which contains statistical information such as defect rate, defect type distribution, and severity distribution, providing a data basis for quality trend analysis.

[0199] Next, analyze the temporal and spatial distribution patterns of defects to identify potential systemic problems. Through time series analysis, identify the fluctuation trend and periodic changes of defect rates; through spatial distribution analysis, identify high-incidence defect areas and defect clustering phenomena; through correlation analysis, identify the correlation and co-occurrence patterns between different defect types. These analysis results help to discover systemic problems in the production process, such as equipment wear, operating deviations, and material fluctuations.

[0200] Then, based on the analysis results, improvement suggestions for the production process are generated, including material selection, processing parameters, assembly process and quality inspection standards. The improvement suggestions are managed using the PDCA (Plan-Do-Check-Act) cycle, and each suggestion contains clear improvement goals, implementation steps, verification methods and expected results. For example, when more hinge loose defects are detected, the system may recommend adjusting the hinge assembly torque or replacing the assembly tool, and provide specific parameter adjustment ranges and verification plans.

[0201] Next, the test results are associated with the patient safety risk level to establish a risk warning mechanism for endoscopic sampling forceps defects. The risk level is divided into five levels: very low (only affects the appearance), low (slightly affects the function but does not affect the use), medium (affects the function but is acceptable), high (significantly affects the function and requires rework) and very high (may cause safety accidents and must be scrapped). The system automatically assesses the risk level based on the defect type, location and severity, and triggers automatic warnings when high and very high risks occur to ensure that problems are handled in a timely manner.

[0202] Finally, the performance of the detection system is regularly evaluated and optimized, including monitoring and improving the detection rate, false alarm rate, and processing speed. The system performance evaluation uses confusion matrix analysis to calculate indicators such as the detection rate, precision rate, and F1 score for each type of defect to identify the strengths and weaknesses of the system; at the same time, the real-time processing capability of the system is monitored to ensure that the production line speed requirements are met. Based on the evaluation results, the system regularly updates model parameters, optimizes algorithm structure, and adjusts detection strategies to maintain continuous improvement in detection performance.

[0203] Through the above-mentioned closed-loop quality control method, the present invention not only realizes the accurate detection of defects of endoscopic sampling forceps, but also establishes a complete closed loop from detection to improvement, providing a full-process solution for medical device quality management.

[0204] The following is a practical application case to demonstrate the technical effect of the present invention.

[0205] The defect detection system of the present invention was deployed on the production line of endoscope sampling forceps of a medical device manufacturer to compare the detection effects of the traditional manual detection method and the method of the present invention. The test was conducted on 500 samples of endoscope sampling forceps, which contained various defects, including surface scratches, jaw deformation, loose hinges, coating peeling and cracks.

[0206] like Figure 8 As shown, the comparison results between the method of the present invention and the manual detection method show that:

[0207] 1. Detection rate: The overall defect detection rate of the method of the present invention reaches 96.8%, while that of manual inspection is 82.3%. In particular, in the detection of tiny defects (area < 0.05mm²), the detection rate of the method of the present invention is 94.2%, while that of manual inspection is only 65.7%;

[0208] 2. Accuracy: The defect classification accuracy of the proposed method is 93.5%, while that of manual inspection is 89.1%;

[0209] 3. Consistency: The consistency of the test results of different operators using the method of the present invention reached 95.7%, while the consistency of manual testing by different operators was only 78.2%;

[0210] 4. Efficiency: The average detection time of the method of the present invention is 0.25 seconds per item, while manual detection takes an average of 15 seconds per item, which is 60 times more efficient;

[0211] 5. Explainability: The method of the present invention can provide defect location, type, severity and cause analysis to support production decision-making and quality improvement.

[0212] This application example fully demonstrates the remarkable technical effect of the present invention in the field of endoscopic sampling forceps defect detection and verifies its application value in improving the quality control level of medical devices.

[0213] In summary, the full-process defect detection method for endoscopic sampling forceps based on machine vision provided by the present invention realizes high-precision and fully automatic defect detection of endoscopic sampling forceps through innovative technologies such as multi-scale geometric perception feature extraction, part-adaptive attention allocation, defect-sensitive residual learning, medical-grade precise segmentation and expert knowledge-guided self-calibration, which meets the strict quality control requirements of medical devices and has significant practical value and promotion prospects.

[0214] The above description is only a preferred embodiment of the present invention, and does not limit the patent protection scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A full-process defect detection method for endoscopic sampling forceps based on machine vision, characterized in that: include: Acquire image data of an endoscopic sampling forceps; Processing the image data through a multi-scale geometry-aware feature extractor to generate a multi-scale feature atlas; Based on the multi-scale feature atlas, three key parts of the endoscope sampling forceps, namely the jaws, hinges and rod body, are identified by a part-adaptive attention module to generate a weighted attention feature map; Based on the weighted attention feature map, features related to the defect type are enhanced through a defect-sensitive residual network to generate a defect-sensitive feature map; Based on the defect sensitive feature map, generating an endoscopic sampling forceps defect segmentation result through a medical-grade precise segmentation module; Based on the defect segmentation result, the uncertainty of the detection result is evaluated through a self-calibration mechanism guided by expert knowledge, and when the uncertainty exceeds a preset threshold, the detection result is calibrated based on an endoscopic sampling forceps defect ontology knowledge base.

2. The method according to claim 1, characterized in that The multi-scale geometry-aware feature extractor processes the image data, specifically comprising: Construct a geometry-aware convolution kernel consisting of a standard convolution branch, a depth-wise separable convolution branch, and a point-wise convolution branch; Based on the geometric features of different components of the endoscopic sampling forceps, the weight coefficient of the geometry-aware convolution kernel is adaptively adjusted by the following formula: ， in, is the weight coefficient of the i-th branch; GAP is the global average pooling operation; FC is the fully connected layer; is the output feature map of the i-th branch; Softmax is the normalization function; Through the angle-sensitive pooling layer, the directional features of the three key parts of the jaw, hinge and rod are captured. The formula is: , Among them, ASP is the angle-sensitive pooling operation; F is the input feature map; θ is the rotation angle, and the value range is [0°, 45°, 90°, 135°]; is the rotation transformation matrix of the angle; MaxPool is the maximum pooling operation; Represents a matrix multiplication operation; Generating multi-scale feature atlas ,in: Indicates the resolution is the original image times the feature map; s is the scale factor.

3. The method according to claim 1, characterized in that The part-adaptive attention module identifies key parts of the endoscopic sampling forceps, specifically including: The channel-space attention map of the three key parts of the endoscope sampling forceps jaws, hinges and rods is calculated using the formula: ， in, For the part The attention map, Jaw, hinge, lever For the part feature map; ChannelPool is the channel pooling operation, and SpatialPool is the spatial pooling operation; and is the learnable weight parameter matrix, ReLU is the rectified linear unit activation function, is the sigmoid activation function, Conv is the convolution operation, and · represents the matrix multiplication or element-by-element multiplication operation; The inter-part relationship model is established to capture the synergistic relationship between the various parts of the sampling clamp. The formula is: ， in, For the part and The relationship characteristics between is the tanh activation function, and Part and parts The feature representation of represents the feature connection operation, A matrix of weight parameters for modeling relationships; is the bias parameter vector; is the tanh activation function; According to the defect risk level of the part, different weights are assigned to the feature map to generate a weighted attention feature map .

4. The method according to claim 1, characterized in that The defect-sensitive residual network specifically includes: Aiming at the unique defect types of endoscopic sampling forceps, including surface scratches, jaw deformation, hinge loosening, coating peeling and cracks, a special residual unit is designed, and the formula is: ， in, is the output feature map, is the input feature map, is the standard residual path, for defect-sensitive skip connections; Design specialized residual paths for different types of defects: For surface scratches, a multi-directional Gabor filter is used to enhance texture features; For jaw deformation, residual connections with shape prior constraints are used; For hinge looseness, the temporal difference module is combined to capture dynamic features; For coating peeling, a reflectivity analysis unit is introduced to enhance the material change characteristics; For cracks, direction-sensitive dilated convolution is used to amplify the fine structure; Generate defect-sensitive feature maps by cascading multiple residual units .

5. The method according to claim 1, characterized in that The medical-grade precise segmentation module specifically includes: Design a reverse attention-guided segmentation network to enhance defect boundary extraction capabilities through foreground-background contrast; The medical-specific boundary enhancement loss function is used, and the formula is: , in, is the pixel weight, is the defect category probability, Pixel The distance to the nearest defect boundary, is the boundary attention coefficient; The defect boundary is iteratively optimized through a multi-stage boundary refinement strategy, and the formula is: , in, is the boundary of the tth iteration; t is the number of iterations; For the The boundary of the iteration; is the defect sensitive feature map; Refine is the boundary refining function; The rough segmentation results, boundary refinement results and multi-scale fusion results are combined to generate the final endoscopic sampling forceps defect segmentation results.

6. The method according to claim 1, characterized in that The self-calibration mechanism guided by expert knowledge specifically includes: Construct an ontological knowledge base of endoscopic sampling forceps defects, including defect cases annotated by experts; The uncertainty based on the attention map is calculated by the following formula: , Among them, U is the uncertainty measurement value of the detection result; is the defect category probability; log is the natural logarithm function, λ is the balance parameter; For the attention map The gradient of · represents the L1 norm; ∑ represents the summation operation of all defect categories; When uncertainty Exceeding the preset threshold When , the expert knowledge-guided self-calibration is performed by the following formula: , in, is the defect category probability after calibration, is the original predicted probability, is the dynamic weight coefficient, Knowledge-based The expert prior probability of Generates calibrated endoscopic forceps defect inspection reports including defect type, location, severity, and repair recommendations.

7. The method according to claim 6, characterized in that The construction of the endoscopic sampling forceps defect ontology knowledge base specifically includes: Collect samples of various defects that occur during the production, use and maintenance of endoscopic sampling forceps; Invite at least three medical device quality inspection experts with more than five years of experience to mark the defective samples; Create a structured description for each defect sample, including defect type, appearance characteristics, cause analysis, and impact assessment; Establish defect similarity metrics for quick retrieval of similar defect cases; The knowledge base content is updated regularly to include newly discovered defect types and characteristics.

8. The method according to any one of claims 1 to 7, characterized in that: It also includes data preprocessing steps: Perform illumination normalization on the acquired endoscopic sampling forceps images to eliminate the influence of uneven illumination; Adopt the specular reflection suppression algorithm based on reflection characteristics to reduce the interference of metal surface highlights on detection; According to the geometric characteristics of the endoscopic sampling forceps, the image is perspective corrected to ensure the proportional consistency of key parts; Enhance image details through contrast-limited adaptive histogram equalization; Multi-scale noise suppression is performed on the image while retaining the subtle features of the defects.

9. The method according to claim 1, characterized in that: Before acquiring image data of the endoscopic sampling forceps, the acquisition strategy optimization step is also included: Based on the 3D CAD model of the endoscopic sampling forceps, determine the key detection viewing angle and optimal imaging distance; Design specific lighting layout, including the position, angle and intensity of main and auxiliary light sources; Develop corresponding image acquisition parameters for different types of potential defects, including exposure time, aperture size, and focus position; The quality of the acquired images is evaluated. When the image quality is lower than the preset standard, the acquisition parameters are automatically adjusted and the image is re-acquired.

10. The method according to claim 1, after completing the defect detection of the endoscopic sampling forceps, further comprises a quality closed-loop control step: Record the defect detection results of each batch of endoscopic sampling forceps, including defect type, location, size and distribution characteristics; Analyze the temporal and spatial distribution patterns of defects to identify potential systemic problems; Generate improvement suggestions for production processes, including material selection, processing parameters, assembly processes and quality inspection standards; Correlate the test results with the patient safety risk level and establish a risk warning mechanism for endoscopic sampling forceps defects; Regularly evaluate and optimize the performance of the detection system, including monitoring and improvement of detection rate, false alarm rate and processing speed.

Images