YOLOv11-based light-weight medical tibia tray surface defect detection method, system, equipment and medium
By using a lightweight YOLOv11 network and data augmentation technology, combined with a robotic arm and vision subsystem, automated and efficient detection of tibial tray surface defects is achieved, solving the problems of high missed detection rate and low efficiency of manual inspection.
Patent Information
- Application Number
- CN202510715731.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-30
- Publication Date
- 2025-09-12
AI Technical Summary
Existing tibial tray defect detection relies on manual inspection, which has a high missed detection rate, low detection efficiency, and high cost, making it difficult to meet the quality requirements of medical device production.
It uses a lightweight YOLOv11 network model, combined with data enhancement technology and dilated convolution, to optimize the prior frame and achieve automated detection through a robotic arm and visual subsystem.
The accuracy and efficiency of tibial tray surface defect detection are significantly improved, the amount of calculation and parameters are reduced, and the needs of industrial applications are met.
Smart Images
Figure CN120634992A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of surface defect detection, and in particular to a method, system, device and medium for detecting surface defects of a lightweight medical tibial tray based on YOLOv11. Background Art
[0002] The tibial tray is a key component in total knee replacement surgery, primarily used for the treatment of tibial fractures, postoperative rehabilitation, or soft tissue injuries. It is typically made of lightweight, breathable materials such as polymer plastic or carbon fiber to ensure a comfortable and secure fit. Its quality significantly impacts the success rate of surgery and the patient's postoperative life. High-quality tibial trays can reduce postoperative complications (such as loosening, infection, and wear), enhance the long-term stability of the prosthesis, and improve the patient's normal postoperative function and quality of life.
[0003] Existing methods for detecting defects in tibial trays usually rely mainly on manual naked-eye inspection, which is overly dependent on manual experience, has a high rate of missed detection, poor detection level, low detection efficiency, and high production cost. It is no longer suitable for surface defect detection in medical tibial trays. Summary of the Invention
[0004] In response to the deficiencies in the prior art, the present invention provides a lightweight medical tibial tray surface defect detection method, system, equipment and medium based on YOLOv11. The lightweight model is used to reduce the number of parameters and calculations, which can significantly improve the detection speed while maintaining high accuracy.
[0005] To achieve the above object, the present invention adopts the following technical solutions:
[0006] A method for detecting surface defects of a lightweight medical tibial tray based on YOLOv11 includes the following steps:
[0007] Take pictures of the tibial tray and build a dataset;
[0008] Use the dataset to train the YOLOv11 network;
[0009] K-fold cross validation is used to evaluate the trained YOLOv11 network to obtain the optimal YOLOv11 network;
[0010] Take an image of the tibial tray to be inspected and input it into the optimal YOLOv11 network for defect recognition to obtain the location and category information of the defect area.
[0011] To optimize the above technical solutions, specific measures taken also include:
[0012] Furthermore, the construction of the data set is specifically as follows:
[0013] Each tibial tray image was labeled and preprocessed to obtain a dataset for training the YOLOv11 network.
[0014] The preprocessing includes mosaic enhancement. The specific process of mosaic enhancement is: stretching the aspect ratio of the tibial tray image to be consistent, and then rotating the image 90 degrees, 180 degrees and 270 degrees respectively based on the image to obtain four images including the main body, arranging these four images in a 2×2 form to obtain a spliced image, performing the same operation on each tibial tray image to obtain a spliced image set, randomly selecting four different images from the spliced image set, cropping these four images, and splicing the cropped parts together to form a new composite image, and continuing to randomly select four different images from the remaining spliced images in the spliced image set, performing the same operation to obtain a mosaic-enhanced composite image set.
[0015] Furthermore, the YOLOv11 network includes a backbone network, a neck network, and a head network;
[0016] The structure of the backbone network includes two conv modules connected in series, and then the first C3K2 module, Conv module, second C3K2 module, Conv module, third C3K2 module, Conv module, SPPF module and C2PSA module are connected in series in sequence; the second C3K2 module outputs the shallow feature map of the backbone network, the third C3K2 module outputs the middle feature map of the backbone network, and the C2PSA module outputs the high-level feature map of the backbone network;
[0017] The shallow feature map of the backbone network, the middle feature map of the backbone network, and the high-level feature map of the backbone network are input into the neck network, specifically:
[0018] The high-level feature map of the backbone network is upsampled and spliced with the middle-level feature map of the backbone network. The spliced feature map is input into the first deep learning module C3K2_Faster. The feature map output by the first deep learning module C3K2_Faster is upsampled and spliced with the shallow-level feature map of the backbone network. The spliced feature map is input into the second deep learning module C3K2_Faster to obtain the shallow-level feature map of the neck network. The shallow-level feature map of the neck network passes through the Conv module and is spliced with the feature map output by the first deep learning module C3K2_Faster. The spliced feature map is input into the third deep learning module C3K2_Faster to obtain the middle-level feature map of the neck network. The middle-level feature map of the neck network passes through the Conv module and is spliced with the high-level feature map of the backbone network. The spliced feature map is input into the fourth deep learning module C3K2_Faster to obtain the high-level feature map of the neck network.
[0019] The shallow feature map of the neck network, the middle feature map of the neck network, and the high-level feature map of the neck network are input into the head network to detect defects in the tibial tray image.
[0020] Furthermore, the C3K2 module includes a FAST module, which adopts dilated convolution technology, specifically: using a 1×1 convolution kernel to reduce the number of input channels of each layer of the network, so that the number of subsequent convolution kernels is also reduced accordingly; applying a convolution with a dilated number of 2 to increase the receptive field of the subsequent layer, and also including a 3×3 dilated convolution.
[0021] Furthermore, the head network includes a full-field-of-view detection head, a medium-field-of-view detection head, and a small-field-of-view detection head;
[0022] The high-level feature map of the neck network is subjected to feature detection and extraction through the full-view prior frame before being input into the full-view detection head;
[0023] The feature map of the middle layer of the neck network is subjected to feature detection and extraction through the middle visual field prior frame before being input into the middle visual field detection head;
[0024] The shallow feature map of the neck network is subjected to feature detection and extraction through the small field of view prior frame before being input into the small field of view detection head.
[0025] Furthermore, the YOLOv11 network is trained using an intersection-over-union loss function, and network hyperparameters are dynamically adjusted during the training process, including a learning rate and a batch size.
[0026] The present invention also proposes a lightweight medical tibial tray surface defect detection system based on YOLOv11, which is characterized in that it includes: a robotic arm, a vision subsystem, a workpiece tray, a fixture and a control cabinet; the robotic arm, the vision subsystem, the workpiece tray and the fixture are all installed on the control cabinet, the robotic arm uses the fixture to clamp the tibial tray placed in the workpiece tray, moves it to the vision subsystem, and takes a picture of the tibial tray. The vision subsystem performs defect recognition on the tibial tray picture based on the YOLOv11 network to obtain the location and category information of the defect area.
[0027] To optimize the above technical solutions, specific measures taken also include:
[0028] Furthermore, the system also includes a monitoring device with a display screen installed above the control cabinet, which displays the working status and detection results of the robotic arm in real time.
[0029] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, it implements the lightweight medical tibial tray surface defect detection method based on YOLOv11 as described above.
[0030] The present invention also proposes a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the above-mentioned lightweight medical tibial tray surface defect detection method based on YOLOv11.
[0031] The beneficial effects of the present invention are as follows: in response to the current problems of low quality, high error detection and missed detection rates, and low production efficiency in medical device production, the lightweight model of YOLOv11 is improved, and a lightweight medical tibial tray surface defect detection method, system, equipment, and medium based on YOLOv11 are proposed, which have the following innovations: 1. Quantization processing, data pruning, void convolution, and prior frame are adopted; 2. In the YOLOv11 network, the C3k2 module is used to improve the speed and performance of the feature aggregation process, and two convolutions are used instead of one large convolution, thereby speeding up the feature extraction speed. 3. A FAST module is proposed, which is used as the core for lightweighting, and void convolution and other technologies are used to improve network performance, and is applied to an industrial robot integrated vision system. The surface quality of the tibial tray is automatically detected, and appearance defects are identified and marked. It meets the requirements of industrial applications. 4. Hyperparameter optimization: The number of network layers is reduced and smaller convolution kernels are used. These strategies significantly reduce the number of parameters and computational complexity of the model while maintaining model performance. 5. Optimization of prior boxes: In YOLOv11, a series of prior boxes of different sizes and aspect ratios are predefined to cover various target shapes that may appear in the image. The learning efficiency of the model can be improved by improving the design of the prior boxes. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 This is a flowchart of the YOLOv11-based lightweight medical tibial tray surface defect detection method proposed in this invention.
[0033] Figure 2 This is a structural diagram of the lightweight medical tibial tray surface defect detection system based on YOLOv11 proposed in this invention.
[0034] Figure 3 This is a schematic diagram of the structure of the YOLOv11 network proposed in this invention.
[0035] Figure 4 This is the FAST module structure diagram.
[0036] The reference numerals are: 1. Robotic arm, 2. Vision subsystem, 3. Workpiece plate, 4. Fixture, 5. Control cabinet. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0038] Example 1
[0039] This paper proposes a method for detecting surface defects of lightweight medical tibial tray based on YOLOv11. The overall process of the method is as follows: Figure 1 As shown, the following steps are included:
[0040] Take pictures of the tibial tray and build a dataset. The specific steps for building the dataset are:
[0041] Each tibial tray image was labeled and preprocessed to obtain a dataset for training the YOLOv11 network.
[0042] To address the diversity of tibial tray surface defects, limited by the limited number of training images (only a few hundred), data augmentation techniques were necessary. Preprocessing included mosaic enhancement, which involves cropping, splicing, and combining multiple images to generate new training samples. This process involves dividing the synthetic image into quadrants and randomly selecting patches from other images to fill these quadrants. This way, each quadrant contains visual information from a different image, creating a synthetic image with diverse backgrounds, textures, and object configurations, improving the utilization of the original images and the model's generalization ability. The specific process of mosaic enhancement is as follows: the aspect ratio of the tibial tray image (size 459×547) is stretched to be consistent (adjusted to 640×640), and then the image is used as a reference, rotated 90 degrees, 180 degrees and 270 degrees respectively to obtain four images including the main body, and these four images are arranged in a 2×2 form to obtain a spliced image with a size of 1280×1280. The same operation is performed on each tibial tray image to obtain a spliced image set, and four different images are randomly selected from the spliced image set. These four images are cropped and the cropped parts are spliced together to form a new composite image with a size of 2560×2560. Four different images are randomly selected from the remaining spliced images in the spliced image set and the same operation is performed to obtain a mosaic-enhanced composite image set.
[0043] The YOLOv11 network is trained using the dataset. The intersection-over-union loss function (IOU) is used when training the YOLOv11 network. Its advantages are: it requires high position confidence but not high accuracy of the detection box size. The network hyperparameters are dynamically adjusted during training. The network hyperparameters include learning rate and batch size. The structure of the YOLOv11 network is as follows: Figure 3 As shown, it includes the backbone network, the neck network and the head network;
[0044] The structure of the backbone network includes two conv modules connected in series, and then the first C3K2 module, Conv module, second C3K2 module, Conv module, third C3K2 module, Conv module, SPPF module and C2PSA module are connected in series in sequence; the second C3K2 module outputs the shallow feature map of the backbone network, the third C3K2 module outputs the middle feature map of the backbone network, and the C2PSA module outputs the high-level feature map of the backbone network; the use of the C3k2 module can improve the speed and performance of the feature aggregation process, and use two convolutions instead of one large convolution, thereby speeding up the feature extraction speed.
[0045] The C3K2 module in the backbone network uses a new FAST module, which employs techniques such as dilated convolution to improve network performance, including:
[0046] Use 1×1 convolution instead of the original 3×3 convolution in YOLOv11.
[0047] The 1×1 convolution kernel is used to reduce the number of input channels of each layer of the network, so that the number of subsequent convolution kernels is also reduced accordingly.
[0048] In order to reduce the downsampling rate, a 2x downsampling pooling layer is removed from the backbone network, and a convolution with a hole number of 2 is applied to increase the receptive field of the subsequent layer. After removing the pooling layer, a 3×3 convolution is converted into a hole convolution.
[0049] The C3K2 module includes the FAST module. The structure of the FAST module is as follows: Figure 4 As shown in the figure, the FAST module uses dilated convolution technology, specifically: using a 1×1 convolution kernel to reduce the number of input channels of each layer of the network, so that the number of subsequent convolution kernels is also reduced accordingly; applying a convolution with a dilation number of 2 to increase the receptive field of the subsequent layer, and also including a 3×3 dilated convolution.
[0050] The shallow feature map of the backbone network, the middle feature map of the backbone network, and the high-level feature map of the backbone network are input into the neck network, specifically:
[0051] The high-level feature map of the backbone network is upsampled and spliced with the middle-level feature map of the backbone network. The spliced feature map is input into the first deep learning module C3K2_Faster. The feature map output by the first deep learning module C3K2_Faster is upsampled and spliced with the shallow-level feature map of the backbone network. The spliced feature map is input into the second deep learning module C3K2_Faster to obtain the shallow-level feature map of the neck network. The shallow-level feature map of the neck network passes through the Conv module and is spliced with the feature map output by the first deep learning module C3K2_Faster. The spliced feature map is input into the third deep learning module C3K2_Faster to obtain the middle-level feature map of the neck network. The middle-level feature map of the neck network passes through the Conv module and is spliced with the high-level feature map of the backbone network. The spliced feature map is input into the fourth deep learning module C3K2_Faster to obtain the high-level feature map of the neck network.
[0052] The shallow feature map of the neck network, the middle feature map of the neck network, and the high-level feature map of the neck network are input into the head network to detect defects in the tibial tray image.
[0053] The head network includes a full-field detection head, a medium-field detection head, and a small-field detection head;
[0054] The high-level feature map of the neck network is subjected to feature detection and extraction through the full-view prior frame before being input into the full-view detection head;
[0055] The feature map of the middle layer of the neck network is subjected to feature detection and extraction through the middle visual field prior frame before being input into the middle visual field detection head;
[0056] The shallow feature map of the neck network is subjected to feature detection and extraction through the small field of view prior frame before being input into the small field of view detection head.
[0057] This invention optimizes the prior frames: it significantly reduces the number of prior frames for other fields of view, thereby reducing the computational effort. The invention uses five prior frames: one for the full field of view, one for the middle field of view, and three for the small field of view, reducing the computational effort by 44%. The full field of view prior frame is sized at 400×400, the middle field of view prior frame is sized at 200×200, and the three small field of view prior frames are sized at 100×100, 50×50, and 25×25, respectively.
[0058] K-fold cross-validation is used to evaluate the trained YOLOv11 network, which can fully utilize the dataset and reduce the risk of overfitting, thereby obtaining the optimal YOLOv11 network. This embodiment uses 10-fold cross-validation to fully utilize the data in the training and validation stages.
[0059] An image of the tibial tray to be inspected is captured and fed into the optimized YOLOv11 network for defect identification, obtaining the location and category of defective areas. NG areas are then marked on the image. Cracks are indicated in purple, pores in red, predicted shrinkage in yellow, and depressions in green. Test results demonstrate that the system accurately identifies and marks surface defects on the tibial tray, significantly improving recognition accuracy.
[0060] Example 2
[0061] The present invention proposes a lightweight medical tibial tray surface defect detection system based on YOLOv11 corresponding to the method of embodiment 1. The structure of the system is as follows: Figure 2 As shown, it includes: a robotic arm 1, a visual subsystem 2, a workpiece tray 3, a fixture 4 and a control cabinet 5; the robotic arm 1, the visual subsystem 2, the workpiece tray 3, and the fixture 4 are all installed on the control cabinet 5, and the robotic arm 1 uses the fixture 4 to clamp the tibial tray placed in the workpiece tray 3, moves to the visual subsystem 2, and takes a picture of the tibial tray. The visual subsystem performs defect recognition on the tibial tray picture based on the YOLOv11 network to obtain the location and category information of the defect area.
[0062] The visual subsystem 2 uses a high-resolution camera and a customized light source to capture multiple surfaces of the tibial tray and identify appearance defects through image processing algorithms; the workpiece tray 3 is responsible for placing the tibial tray parts to be inspected and is relatively stationary with respect to the base of the robotic arm 1; the tooling fixture 4 is used to fix the tibial tray to ensure stability and accuracy during the shooting process; a monitoring device with a display screen is installed above the control cabinet 5, which can display the working status and detection results of the robotic arm in real time.
[0063] The YOLOv11-based lightweight medical tibial tray surface defect detection system uses industrial-grade robotic arms and cameras to ensure shooting stability and accuracy; the fixture adopts a customized design to adapt to different models of tibial trays; the control system uses programmable controllers such as PLC to realize automated control processes.
[0064] Example 3
[0065] The present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting surface defects of a lightweight medical tibial tray based on YOLOv11 as described in Example 1 is implemented.
[0066] Example 4
[0067] The present invention proposes a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the lightweight medical tibial tray surface defect detection method based on YOLOv11 as described in Example 1.
[0068] In the embodiments disclosed herein, computer storage media can be tangible media that can contain or store programs for use by or in conjunction with an instruction execution system, device, or apparatus. Computer storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0069] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0070] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.
Claims
1. A method for detecting surface defects of lightweight medical tibial trays based on YOLOv11, characterized in that: The following steps are involved: Take pictures of the tibial tray and build a dataset; Use the dataset to train the YOLOv11 network; K-fold cross validation is used to evaluate the trained YOLOv11 network to obtain the optimal YOLOv11 network; Take an image of the tibial tray to be inspected and input it into the optimal YOLOv11 network for defect recognition to obtain the location and category information of the defect area.
2. The method for detecting surface defects of a lightweight medical tibial tray based on YOLOv11 according to claim 1, wherein: The construction of the data set is specifically as follows: Each tibial tray image was labeled and preprocessed to obtain a dataset for training the YOLOv11 network. The preprocessing includes mosaic enhancement. The specific process of mosaic enhancement is: stretching the aspect ratio of the tibial tray image to be consistent, and then rotating the image 90 degrees, 180 degrees and 270 degrees respectively based on the image to obtain four images including the main body, arranging these four images in a 2×2 form to obtain a spliced image, performing the same operation on each tibial tray image to obtain a spliced image set, randomly selecting four different images from the spliced image set, cropping these four images, and splicing the cropped parts together to form a new composite image, and continuing to randomly select four different images from the remaining spliced images in the spliced image set, performing the same operation to obtain a mosaic-enhanced composite image set.
3. The method for detecting surface defects of a lightweight medical tibial tray based on YOLOv11 according to claim 1, wherein: The YOLOv11 network includes a backbone network, a neck network, and a head network; The structure of the backbone network includes two conv modules connected in series, and then the first C3K2 module, Conv module, second C3K2 module, Conv module, third C3K2 module, Conv module, SPPF module and C2PSA module are connected in series in sequence; the second C3K2 module outputs the shallow feature map of the backbone network, the third C3K2 module outputs the middle feature map of the backbone network, and the C2PSA module outputs the high-level feature map of the backbone network; The shallow feature map of the backbone network, the middle feature map of the backbone network, and the high-level feature map of the backbone network are input into the neck network, specifically: The high-level feature map of the backbone network is upsampled and spliced with the middle-level feature map of the backbone network. The spliced feature map is input into the first deep learning module C3K2_Faster. The feature map output by the first deep learning module C3K2_Faster is upsampled and spliced with the shallow-level feature map of the backbone network. The spliced feature map is input into the second deep learning module C3K2_Faster to obtain the shallow-level feature map of the neck network. The shallow-level feature map of the neck network passes through the Conv module and is spliced with the feature map output by the first deep learning module C3K2_Faster. The spliced feature map is input into the third deep learning module C3K2_Faster to obtain the middle-level feature map of the neck network. The middle-level feature map of the neck network passes through the Conv module and is spliced with the high-level feature map of the backbone network. The spliced feature map is input into the fourth deep learning module C3K2_Faster to obtain the high-level feature map of the neck network. The shallow feature map of the neck network, the middle feature map of the neck network, and the high-level feature map of the neck network are input into the head network to detect defects in the tibial tray image.
4. The method for detecting surface defects of a lightweight medical tibial tray based on YOLOv11 according to claim 3, wherein: The C3K2 module includes a FAST module, which uses dilated convolution technology. Specifically, the FAST module uses a 1×1 convolution kernel to reduce the number of input channels of each layer of the network, thereby reducing the number of subsequent convolution kernels accordingly; a convolution with a dilated number of 2 is applied to increase the receptive field of the subsequent layer, and also includes a 3×3 dilated convolution.
5. The method for detecting surface defects of a lightweight medical tibial tray based on YOLOv11 according to claim 3, wherein: The head network includes a full-field detection head, a medium-field detection head and a small-field detection head; The high-level feature map of the neck network is subjected to feature detection and extraction through the full-view prior frame before being input into the full-view detection head; The feature map of the middle layer of the neck network is subjected to feature detection and extraction through the middle visual field prior frame before being input into the middle visual field detection head; The shallow feature map of the neck network is subjected to feature detection and extraction through the small field of view prior frame before being input into the small field of view detection head.
6. The method for detecting surface defects of a lightweight medical tibial tray based on YOLOv11 according to claim 1, wherein: The YOLOv11 network is trained using an intersection-over-union loss function, and network hyperparameters are dynamically adjusted during training. The network hyperparameters include a learning rate and a batch size.
7. A lightweight medical tibial tray surface defect detection system based on YOLOv11, characterized in that: include: Robotic arm, vision subsystem, workpiece tray, fixture and control cabinet; The robotic arm, vision subsystem, workpiece tray, and fixture are all installed on the control cabinet. The robotic arm uses the fixture to clamp the tibial tray placed in the workpiece tray, moves it to the vision subsystem, and takes a picture of the tibial tray. The vision subsystem identifies defects in the tibial tray image based on the YOLOv11 network to obtain the location and category information of the defective area.
8. The YOLOv11-based lightweight medical tibial tray surface defect detection system according to claim 7, characterized in that: The system further comprises a monitoring device with a display screen installed above the control cabinet, which displays the working status and detection results of the robotic arm in real time.
9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting surface defects of a lightweight medical tibial tray based on YOLOv11 as described in any one of claims 1 to 6 is implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables the computer to execute the lightweight medical tibial tray surface defect detection method based on YOLOv11 as described in any one of claims 1-6.