A lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3

Abstract

This invention provides a lightweight dental caries detection system and method based on the fusion of YOLOv5 and IGCV3, comprising: collecting dental caries data and constructing a sample dataset; preprocessing the sample dataset and dividing it into a training set, a validation set, and a test set; fusing the target detection algorithm YOLOv5 with the lightweight network algorithm IGCV3 to construct a dental caries detection model; training and validating the dental caries detection model using the training and validation sets; using the validated dental caries detection model to predict the test set to be identified; and outputting the dental caries detection model based on the prediction results and deploying it to a mobile platform for dental caries detection. Thus, this invention, by fusing the target detection algorithm YOLOv5 with the lightweight network algorithm IGCV3, better balances the detection speed and accuracy of the dental caries detection model, improving detection accuracy while being easier to deploy and possessing strong practicality.

Description

Technical Field

[0001] This invention relates to, but is not limited to, the field of oral disease prevention, and particularly to a lightweight caries detection system, method, device, terminal, and storage medium based on the fusion of YOLOv5 and IGCV3. Background Technology

[0002] Dental caries is a common disease threatening human health, and early detection and treatment of caries is an important research topic in the field of oral health prevention and treatment. Early caries detection mainly relies on visual inspection and palpation, depending on the patient's subjective feelings, which cannot accurately assess the condition of the teeth and is not conducive to accurate and timely diagnosis of early caries. Currently, X-ray transmission imaging and optical caries detection methods are commonly used. X-rays produce harmful ionizing radiation, which can have adverse effects on children and pregnant women. Conversely, optical caries detection methods do not produce ionizing radiation and can better detect tooth enamel, thus gaining wider application.

[0003] Furthermore, optical methods for predicting dental caries mainly include quantitative light-induced fluorescence (QLF), photothermal radiometry, and fiber optic transillumination. QLF works by irradiating teeth with light in the 385nm–425nm range. Teeth under different conditions will exhibit different colors of fluorescence; for example, healthy teeth show green fluorescence, while caries will produce dark spots due to demineralization, thus detecting plaque. However, its accuracy is lower than visual verification, and the method has a large number of model parameters, consuming significant memory. Photothermal radiometry (PTR) works based on the thermo-optic effect. Adjusting the laser irradiation of the teeth results in contrasting thermal signals obtained from damaged and healthy areas, leading to differences in the processed data and images. However, it suffers from low detection accuracy and sensitivity. Fiber-optic transillumination (FOTI) works on the same principle as traditional visual examination. When exposed to near-infrared light, healthy teeth scatter and absorb less light than decayed areas, thus appearing brown or orange-yellow. Decayed areas, due to demineralization, appear as dark gray bands. Although it does not involve ionizing radiation and can effectively detect the condition of adjacent teeth and the depth of cavities, its detection process is greatly affected by ambient light, and its accuracy is not high.

[0004] Therefore, improving the accuracy and sensitivity of caries detection while ensuring high universality and practicality is an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a lightweight caries detection system, method, device, terminal, and storage medium based on the fusion of YOLOv5 and IGCV3.

[0006] The technical solution of this invention is implemented as follows:

[0007] A lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3, the method comprising:

[0008] Collect dental caries data and construct a sample dataset;

[0009] The sample dataset is preprocessed and divided into a training set, a validation set, and a test set;

[0010] The target detection algorithm YOLOv5 is fused with the lightweight network algorithm IGCV3 to construct a caries detection model;

[0011] The caries detection model is trained and validated using the training and validation sets.

[0012] The validated caries detection model is used to predict the test set to be identified;

[0013] Based on the prediction results, the caries detection model is output and deployed to a mobile platform for caries detection.

[0014] In some embodiments, the method further comprises:

[0015] The sample dataset was labeled using a labeling tool;

[0016] The labeled sample dataset is divided into a training set, a validation set, and a test set according to a preset ratio.

[0017] In some embodiments, the method further comprises:

[0018] The YOLOv5s model is built within a deep learning framework to construct an object detection network; wherein the object detection model includes: an input module, a backbone module, a neck module, and a head module;

[0019] The IGCV3 module is integrated into the target detection network to construct the caries detection model.

[0020] In some embodiments, the method includes:

[0021] The IGCV3 module replaces some modules in the Backbone module, forming a new network structure for the Backbone module as follows:

[0022] First layer: Conv module, 32 convolutional kernels of size 6×6, stride of 2, resulting in data with features of 320×320×32;

[0023] The second layer: Conv module, with 64 convolutional kernels of size 3×3 and a stride of 1, yields data with features of 160×160×64;

[0024] The third layer: repeat the C3 module 3 times, with 64 convolutional kernels of size 3×3 and a stride of 1, to obtain data with features of 160×160×64;

[0025] The fourth layer: Conv module, with 128 convolutional kernels of size 3×3 and a stride of 2, yields data with features of 80×80×128;

[0026] Fifth layer: Repeat 6 C3 modules, 128 convolutional kernels of size 3×3, stride 1, to obtain data with features of 80×80×128;

[0027] The sixth layer: IGCV3 module, 256 convolutional kernels, stride of 2, and expansion coefficient of 2, yielding data with features of 40×40×256;

[0028] Layer 7: Repeat 9 IGCV3 modules, 256 convolutional kernels, stride of 1, and expansion coefficient of 2 to obtain data with features of 40×40×256;

[0029] The eighth layer: IGCV3 module, 512 convolutional kernels, stride of 2, and expansion coefficient of 2, yielding data with features of 20×20×512;

[0030] Ninth layer: Repeat 3 IGCV3 modules, 512 convolutional kernels, stride of 1, and expansion coefficient of 2 to obtain data with features of 20×20×512;

[0031] Layer 10: SPPF module, 3×3 pooling window, stride of 1, yields data with features of 20×20×512.

[0032] In some embodiments, the method includes:

[0033] The IGCV3 module replaces the CSP module in the Neck module, forming a new network structure for the Neck module as follows:

[0034] First layer: Conv module, 256 convolutional kernels of size 1×1, stride of 1, resulting in data with features of 20×20×256;

[0035] Second layer: 2x upsampling, resulting in data with features of 40×40×256;

[0036] Third layer: Connect the output of the second layer of the new Neck module network structure to the output of the seventh layer of the new Backbone module network structure, and output data with a feature of 40×40×512;

[0037] The fourth layer: IGCV3 module, with 256 convolutional kernels, a stride of 1, and an amplification factor of 2, yields data with features of 40×40×256;

[0038] Fifth layer: Conv module, 128 convolutional kernels of size 1×1, stride of 1, and expansion coefficient of 2, resulting in data with features of 40×40×128;

[0039] Sixth layer: 2x upsampling, resulting in data with features of 80×80×128;

[0040] Seventh layer: Connect the output of the sixth layer of the new Neck module's network structure to the output of the fifth layer of the new Backbone module's network structure, and output data with characteristics of 80×80×256;

[0041] The eighth layer: IGCV3 module, with 128 convolutional kernels, stride of 1, and expansion coefficient of 2, yields data with features of 80×80×128;

[0042] Ninth layer: Conv module, 128 convolutional kernels of size 3×3, stride of 2, resulting in data with features of 40×40×128;

[0043] Layer 10: Connects the outputs of the ninth and fifth layers of the new Neck module's network structure, outputting data with a feature of 40×40×256;

[0044] The eleventh layer: IGCV3 module, 256 convolutional kernels, stride of 1, and expansion coefficient of 2, yielding data with features of 40×40×256;

[0045] The twelfth layer: Cnov module, with 256 convolutional kernels of size 3×3 and a stride of 2, yields data with features of 20×20×256;

[0046] Thirteenth layer: Connect the outputs of the twelfth and first layers of the new Neck module's network structure, and output data with a feature of 20×20×512;

[0047] The fourteenth layer: IGCV3 module, 512 convolutional kernels, stride of 1, expansion coefficient of 2, resulting in data with features of 20×20×512;

[0048] The fifteenth layer: the Detect module, which obtains data with features of 80×80, 40×40 and 20×20 respectively, for detecting targets of different sizes.

[0049] In some embodiments, the method further comprises:

[0050] The caries detection model is trained using the training set.

[0051] The trained caries detection model was validated using the validation set.

[0052] Determine whether the loss curve of the validated caries detection model has converged; if so, determine that the caries detection model has stabilized and stop training the caries detection model; and / or,

[0053] If not, the caries detection model is determined to be unstable, and training of the caries detection model continues.

[0054] In some embodiments, the method includes:

[0055] The test set to be identified is objectively judged by a professional dentist, and the results obtained by the dentist are compared with the prediction results of the model to determine the accuracy of the prediction results; if the accuracy is less than a preset accuracy threshold, training is stopped and the caries detection model is output; and / or,

[0056] If the accuracy is not less than the preset accuracy threshold, then continue training the caries detection model.

[0057] This invention also provides a device for a lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3, comprising:

[0058] The acquisition module is used to collect dental caries data and build a sample dataset;

[0059] The processing module is used to preprocess the sample dataset and divide it into a training set, a validation set, and a test set;

[0060] The processing module is also used to fuse the target detection algorithm YOLOv5 with the lightweight network algorithm IGCV3 to construct a caries detection model;

[0061] The processing module is also used to train and validate the caries detection model using the training set and validation set;

[0062] The processing module is also used to make predictions on the test set to be identified using the validated caries detection model;

[0063] The output module is used to output the caries detection model based on the prediction results and deploy it to a mobile platform for caries detection.

[0064] This invention also provides a terminal, which includes a processor and a memory for storing computer programs that can run on the processor; wherein, when the processor runs the computer program, it implements the lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3 described in any embodiment of this invention.

[0065] This invention also provides a computer-readable storage medium containing computer-executable instructions, characterized in that the computer-executable instructions are executed by a processor to implement the lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3 described in any embodiment of this invention.

[0066] This invention provides a lightweight dental caries detection system and method based on the fusion of YOLOv5 and IGCV3. The method involves collecting dental caries data to construct a sample dataset; preprocessing the sample dataset and dividing it into a training set, a validation set, and a test set; fusing the target detection algorithm YOLOv5 with the lightweight network algorithm IGCV3 to construct a dental caries detection model; training the dental caries detection model using the training set; validating the trained dental caries detection model using the validation set; using the validated dental caries detection model to predict the test set to be identified, determining the accuracy of the validated dental caries detection model; and outputting the dental caries detection model based on the accuracy and deploying it to a mobile platform for intelligent dental caries detection. Thus, this invention can better balance the detection speed and accuracy of the model, improving detection accuracy while being easier to deploy and possessing strong practicality. Furthermore, by fusing the IGCV3 model into the original YOLOv5 model, this invention can determine a lightweight dental caries detection model, reducing model computational complexity while improving detection speed and accuracy, and thus making it easier to deploy on intelligent mobile platforms. Attached Figure Description

[0067] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0068] Figure 1 This is a flowchart illustrating a lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3 provided in an embodiment of the present invention.

[0069] Figure 2 This is a flowchart of a caries detection network before the fusion of YOLOv5 and IGCV3 provided in an embodiment of the present invention;

[0070] Figure 3This is a flowchart of a lightweight caries detection network based on the fusion of YOLOv5 and IGCV3 provided in an embodiment of the present invention.

[0071] Figure 4 This is a schematic diagram of a lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3 provided in an embodiment of the present invention;

[0072] Figure 5 This is a schematic diagram of the terminal hardware structure of a lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3 provided in an embodiment of the present invention. Detailed Implementation

[0073] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0074] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0075] like Figure 1 As shown, this embodiment of the invention provides a lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3, the method comprising:

[0076] Step S101: Collect dental caries data and construct a sample dataset;

[0077] Step S102: Preprocess the sample dataset and divide it into a training set, a validation set, and a test set;

[0078] Step S103: Merge the target detection algorithm YOLOv5 with the lightweight network algorithm IGCV3 to construct a caries detection model;

[0079] Step S104: Use the training set and validation set to train and validate the caries detection model;

[0080] Step S105: Use the validated caries detection model to predict the test set to be identified;

[0081] Step S106: Based on the prediction results, output the caries detection model and deploy it to a mobile platform for intelligent caries detection.

[0082] The material feeding method described in this embodiment of the invention is executed by a terminal. The terminal can be any type of terminal; for example, the terminal can be, but is not limited to, at least one of the following: a server, a computer, a tablet computer, or other electronic devices.

[0083] In some embodiments, the sample dataset in step S101 includes positive samples and negative samples.

[0084] Here, the sample dataset includes a certain amount of dental caries data; the dental caries data can be historical records of dental caries examinations in hospitals or professional literature in the field of dental science.

[0085] For example, based on 3,588 images provided by the hospital, images without cavities are used as positive samples, and images with cavities are used as negative samples, forming a dataset containing 3,588 images.

[0086] In some embodiments, step S102 includes: labeling the sample dataset using a labeling tool; and dividing the labeled sample dataset into a training set, a validation set, and a test set according to a preset ratio.

[0087] Here, the annotation tool can be, but is not limited to, an image annotation tool, such as the labelImg software.

[0088] Here, the preset ratio is a given value; for example, 8:1:1.

[0089] In one embodiment, step S102 includes: using a annotation tool to generate a corresponding annotation file, the annotation file including: file format and file content.

[0090] Here, the file format can be XML.

[0091] Here, the file content includes, but is not limited to: the target category and the target location information, wherein the target location information includes the X-axis of the real target. min Y min X max With Y max Coordinate information.

[0092] It is understood that the target can be a tooth in the caries image.

[0093] For example, the collected caries images are divided into two categories according to the disease type: suspected permanent teeth (hyys) and caries in permanent teeth (hyqc). The image annotation tool labelImg is used to annotate the location and type of suspected permanent teeth or caries in the target image, and an XML format annotation file is output. The caries images and the corresponding annotation files are divided into the training set, validation set and test set according to an 8:1:1 ratio.

[0094] In some embodiments, step S103 includes: building the YOLOv5 model in a deep learning framework to construct an object detection network; wherein the object detection model includes: an input module, a backbone module, a neck module, and a head module; and fusing the IGCV3 model into the object detection network to construct the caries detection model.

[0095] Here, the deep learning framework may be, but is not limited to, the PyTorch framework.

[0096] For example, a YOLOv5 model is built in the PyTorch framework, constructing something like... Figure 2 The flowchart of the target detection network is shown; the target detection network includes four main models: input module, backbone module, neck module, and head module; the IGCV3 model is fused into the target detection network to construct the following... Figure 3 The caries detection model with the network flow structure shown.

[0097] Here, the input module is used to preprocess the images; specifically, the input module scales each image according to a preset grid size and then normalizes it. Simultaneously, data augmentation techniques, such as the Mosaic method, are employed to improve the model's accuracy and training speed.

[0098] In one embodiment, the step of fusing the IGCV3 model into the target detection network to construct the caries detection model includes: replacing some modules in the Backbone module with the IGCV3 module to form a new Backbone module.

[0099] For example, the IGCV3 model replaces some of the CSP modules and some of the CBS modules in the Backbone module to form a new ten-layer network structure for the Backbone module, as follows:

[0100] First layer: Conv module, 32 convolutional kernels of size 6×6, stride of 2, resulting in data with features of 320×320×32;

[0101] Here, the data input to the first layer can be a three-channel RGB image with a size of 640×640.

[0102] The second layer: Conv module, with 64 convolutional kernels of size 3×3 and a stride of 1, yields data with features of 160×160×64;

[0103] The third layer: repeat the C3 module 3 times, with 64 convolutional kernels of size 3×3 and a stride of 1, to obtain data with features of 160×160×64;

[0104] The fourth layer: Conv module, with 128 convolutional kernels of size 3×3 and a stride of 2, yields data with features of 80×80×128;

[0105] Fifth layer: Repeat 6 C3 modules, 128 convolutional kernels of size 3×3, stride 1, to obtain data with features of 80×80×128;

[0106] The sixth layer: IGCV3 module, with 256 1×1 convolutional kernels, stride of 2, and expansion coefficient of 2, resulting in data with features of 40×40×256;

[0107] Layer 7: Repeat 9 IGCV3 modules, 256 1×1 convolutional kernels, stride 1, and expansion coefficient 2 to obtain data with features of 40×40×256;

[0108] The eighth layer: IGCV3 module, with 512 1×1 convolutional kernels, stride of 2, and expansion coefficient of 2, resulting in data with features of 20×20×512;

[0109] Ninth layer: Repeat 3 IGCV3 modules, 512 1×1 convolution kernels, stride 1, and expansion coefficient 2 to obtain data with features of 20×20×512;

[0110] Layer 10: SPPF module, 3×3 pooling window, stride of 1, yields data with features of 20×20×512.

[0111] Here, the SPPF module is spatial pyramid pooling, which solves the problem of inconsistent input image sizes.

[0112] It is understood that the Backbone module in the object detection network is mainly used for feature extraction, and it consists of convolutional Conv and C3 modules. The IGCV3 module replaces some modules in the Backbone module. The structure of the IGCV3 module is as follows: first, a 1×1 point convolution is performed, divided into two groups, for dimensionality increase; then, a 3×3 depth group convolution is performed, divided into several groups, for feature extraction; finally, a 1×1 point convolution is performed, divided into two groups, for dimensionality reduction.

[0113] Thus, the present invention can perform low-rank decomposition of the target detection network by replacing some modules of the Backbone module in the target detection network with the IGCV3 module, thereby reducing the network weight while ensuring accuracy, having fewer computational parameters, and thus improving detection efficiency and sensitivity.

[0114] In one embodiment, the step of fusing the IGCV3 model into the target detection network to construct the caries detection model includes: replacing part of the CSP module in the Neck module with the IGCV3 module to form a new Neck module.

[0115] Here, the Neck module follows the Backbone module, meaning that the final output of the Backbone module serves as the input of the Neck module.

[0116] For example, the IGCV3 model replaces all CSP modules in the Neck module to form a new fifteen-layer network structure for the Neck module, as follows:

[0117] First layer: Conv module, 256 convolutional kernels of size 1×1, stride of 1, resulting in data with features of 20×20×256;

[0118] Second layer: 2x upsampling, resulting in data with features of 40×40×256;

[0119] Third layer: Connect the output of the second layer of the new Neck module network structure to the output of the seventh layer of the new Backbone module network structure, and output data with a feature of 40×40×512;

[0120] The fourth layer: IGCV3 module, with 256 convolutional kernels, a stride of 1, and an amplification factor of 2, yields data with features of 40×40×256;

[0121] Fifth layer: Conv module, 128 convolutional kernels of size 1×1, stride of 1, and expansion coefficient of 2, resulting in data with features of 40×40×128;

[0122] Sixth layer: 2x upsampling, resulting in data with features of 80×80×128;

[0123] Seventh layer: Connect the output of the sixth layer of the new Neck module's network structure to the output of the fifth layer of the new Backbone module's network structure, and output data with characteristics of 80×80×256;

[0124] The eighth layer: IGCV3 module, with 128 convolutional kernels, stride of 1, and expansion coefficient of 2, yields data with features of 80×80×128;

[0125] Ninth layer: Conv module, 128 convolutional kernels of size 3×3, stride of 2, resulting in data with features of 40×40×128;

[0126] Layer 10: Connects the outputs of the ninth and fifth layers of the new Neck module's network structure, outputting data with a feature of 40×40×256;

[0127] The eleventh layer: IGCV3 module, 256 convolutional kernels, stride of 1, and expansion coefficient of 2, yielding data with features of 40×40×256;

[0128] The twelfth layer: Cnov module, with 256 convolutional kernels of size 3×3 and a stride of 2, yields data with features of 20×20×256;

[0129] Thirteenth layer: Connect the outputs of the twelfth and first layers of the new Neck module's network structure, and output data with a feature of 20×20×512;

[0130] The fourteenth layer: IGCV3 module, 512 convolutional kernels, stride of 1, expansion coefficient of 2, resulting in data with features of 20×20×512;

[0131] The fifteenth layer: the Detect module, which obtains data with features of 80×80, 40×40 and 20×20 respectively, for detecting targets of different sizes.

[0132] Understandably, through multi-layer convolution to extract features from the target, the Neck module ultimately outputs feature data with sizes of 80×80, 40×40, and 20×20. These grid feature data of different sizes are used to detect targets of different sizes. Specifically, the 20×20 feature map predicts large targets, the 20×20 and 40×40 feature maps together predict medium-sized targets, and the 20×20, 40×40, and 80×80 feature maps predict small targets.

[0133] It is understood that the Neck module in the target detection network is used to improve feature extraction capabilities, and the target detection network includes SPPF and PANet structures; the IGCV3 module replaces all C3 modules in the Neck module of the target detection network. Thus, this invention can reduce computational load while maintaining model accuracy by replacing the C3 modules in the Neck module of the target detection network with the IGCV3 module; thereby improving detection speed, meeting real-time requirements, and making it more suitable for deployment on intelligent mobile platforms for routine caries detection.

[0134] In one embodiment, in step S103: the Head module includes a classification loss function, a localization loss function, and a confidence loss function; wherein, the formula for the localization loss function is as follows:

[0135]

[0136] Furthermore, b is the center point of the predicted bounding box. gt Let ρ be the center point of the target bounding box, and let b be the distance between b and b'. gt The Euclidean distance between the predicted bounding box and the target bounding box is c, where c is the length of the diagonal of the minimum bounding rectangle of the predicted bounding box and the target bounding box.

[0137] Here, the Head module is used to measure the quality of the prediction results of the caries detection model; the localization loss function can determine the mean average precision (mAP) of the caries detection model.

[0138] Here, the classification loss function is used to calculate the classification loss between the anchor box and the target box; the localization loss function is used to calculate the error between the predicted box and the target box; and the confidence loss function is used to calculate the confidence of the caries detection model.

[0139] Understandably, the localization loss function is used to measure the correctness of the predictions made by the caries detection model. Here, the main consideration is adding a component to the loss function to minimize the center distance between the predicted bounding box and the target bounding box. Thus, this invention uses the localization loss function DIOU to minimize the center distance between the predicted bounding box and the target bounding box, thereby accelerating the convergence speed and improving the detection accuracy of the caries detection module, ultimately enhancing the target detection accuracy.

[0140] In some embodiments, step S104 includes: training the caries detection model using the training set; validating the trained caries detection model using the validation set; and determining whether the loss curve of the validated caries detection model has converged.

[0141] It is understood that after the caries detection model is trained using the training set, the training process will automatically initiate the verification process, and verification will not continue if training stops; the caries detection model is verified using the verification set to obtain the verification result of the caries detection model.

[0142] Here, the validation results include: loss curves and mean accuracy; the mean accuracy is used to evaluate the performance index of the caries detection model.

[0143] In one embodiment, step S104 includes: if the loss curve of the verified caries detection model converges, then the caries detection model is determined to be stable, and training of the caries detection model is stopped.

[0144] For example, a preset loss threshold is set. If the change in the loss curve of the verified caries detection model is less than the preset loss threshold, the loss curve is determined to have converged, the caries detection model tends to be stable, and training of the caries detection model is stopped.

[0145] Here, the preset threshold is a given value; for example, 0.001.

[0146] It is understandable that by judging that the error change of the loss curve is within 0.001, the detection result is considered to be correct, that is, it is considered to be a correct detection, and training is stopped.

[0147] In another embodiment, step S104 further includes: if the loss curve of the verified caries detection model does not converge, then it is determined that the caries detection model has not stabilized, and the caries detection model continues to be trained.

[0148] For example, if the change in the error of the loss curve of the verified caries detection model is not less than the preset loss threshold, it is determined that the loss curve has not converged, the caries detection model has not stabilized, and the caries detection model continues to be trained.

[0149] It is understandable that if the error change of the loss curve exceeds 0.001, the detection result is considered incorrect, i.e., it is considered a false detection, and the caries detection model needs to be trained again.

[0150] In one embodiment, step S104 includes: setting a preset number of rounds, and training and validating the caries detection model based on the training set and the validation set.

[0151] Generally, setting 300 or 600 rounds can achieve convergence.

[0152] It is understood that after each round of training and validation, the current mAP of the caries detection model will be obtained so that the final mAP of the caries detection model can be obtained after the training ends; it is understood that the final mAP is the best mAP of the caries detection model on the validation set.

[0153] In one embodiment, step S105 includes: using the validated caries detection model to predict the test set to be identified, and obtaining prediction results on the test set.

[0154] Here, the prediction result can be either a suspected permanent tooth based on the caries image, or a caries diagnosis of a permanent tooth based on the caries image.

[0155] In one embodiment, step S106 includes: having a professional dentist objectively judge the test set to be identified, comparing the results obtained by the dentist with the prediction results of the model, and determining the accuracy of the prediction results; if the accuracy is less than a preset accuracy threshold, then stopping training and outputting the caries detection model.

[0156] Here, the preset accuracy threshold is a given value, such as 80%.

[0157] In another embodiment, step S106 further includes: if the accuracy is not less than the preset accuracy threshold, then continue training the caries detection model.

[0158] For example, a professional dentist objectively judges the test set to be identified, compares the results obtained by the dentist with the prediction results of the model, and if the calculation accuracy does not reach 80%, then the caries detection model still needs to be further optimized.

[0159] In some embodiments, step S106 includes: deploying the output caries detection model to a mobile platform; acquiring new caries data; predicting the new caries data based on the deployed caries detection model; and outputting the prediction result of the caries detection model.

[0160] For example, a user can use a smartphone to take a picture of the inside of their mouth, upload the picture to the mobile platform, and obtain a prediction result through the caries detection model to directly determine the health condition of the user's mouth.

[0161] In this embodiment of the invention, dental caries data is collected to construct a sample dataset; the sample dataset is preprocessed and divided into a training set, a validation set, and a test set; the target detection algorithm YOLOv5 is fused with the lightweight network algorithm IGCV3 to construct a caries detection model; the caries detection model is trained and validated using the training and validation sets; the validated caries detection model is used to predict the test set to be identified; based on the prediction results, the caries detection model is output and deployed to a mobile platform for caries detection. Thus, this invention, by fusing the target detection algorithm YOLOv5 with the lightweight network algorithm IGCV3, better balances the detection speed and accuracy of the caries detection model, has fewer computational parameters, thereby improving detection efficiency and sensitivity, exhibiting stronger robustness, and is more suitable for deployment on intelligent mobile platforms for daily caries detection.

[0162] like Figure 4 As shown, this embodiment of the invention also provides a device for a lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3. The device includes: an acquisition module 201, a processing module 202, and an output module 203; wherein,

[0163] The acquisition module 201 is used to collect dental caries data and construct a sample dataset;

[0164] The processing module 202 is used to preprocess the sample dataset and divide it into a training set, a validation set, and a test set;

[0165] The processing module 202 is also used to fuse the target detection algorithm YOLOv5 with the lightweight network algorithm IGCV3 to construct a caries detection model;

[0166] The processing module 202 is also used to train and validate the caries detection model using the training set and validation set;

[0167] The processing module 202 is further configured to use the validated caries detection model to predict the test set to be identified;

[0168] The output module 203 is used to output the caries detection model based on the prediction results and deploy it to a mobile platform for caries detection.

[0169] In some embodiments, the method further comprises:

[0170] The processing module is used to annotate the sample dataset using an annotation tool;

[0171] The processing module is used to divide the labeled sample dataset into a training set, a validation set, and a test set according to a preset ratio.

[0172] In some embodiments, the method further comprises:

[0173] The processing module is used to build the YOLOv5s model in the deep learning framework and construct an object detection network; wherein, the object detection model includes: an input module, a backbone module, a neck module and a head module;

[0174] The processing module is used to integrate the IGCV3 module into the target detection network to construct the caries detection model.

[0175] In some embodiments, the method further comprises:

[0176] The processing module is used to replace some modules in the Backbone module with the IGCV3 module to form a new network structure for the Backbone module as follows:

[0177] First layer: Conv module, 32 convolutional kernels of size 6×6, stride of 2, resulting in data with features of 320×320×32;

[0178] The second layer: Conv module, with 64 convolutional kernels of size 3×3 and a stride of 1, yields data with features of 160×160×64;

[0179] The third layer: repeat the C3 module 3 times, with 64 convolutional kernels of size 3×3 and a stride of 1, to obtain data with features of 160×160×64;

[0180] The fourth layer: Conv module, with 128 convolutional kernels of size 3×3 and a stride of 2, yields data with features of 80×80×128;

[0181] Fifth layer: Repeat 6 C3 modules, 128 convolutional kernels of size 3×3, stride 1, to obtain data with features of 80×80×128;

[0182] The sixth layer: IGCV3 module, 256 convolutional kernels, stride of 2, and expansion coefficient of 2, yielding data with features of 40×40×256;

[0183] Layer 7: Repeat 9 IGCV3 modules, 256 convolutional kernels, stride of 1, and expansion coefficient of 2 to obtain data with features of 40×40×256;

[0184] The eighth layer: IGCV3 module, 512 convolutional kernels, stride of 2, and expansion coefficient of 2, yielding data with features of 20×20×512;

[0185] Ninth layer: Repeat 3 IGCV3 modules, 512 convolutional kernels, stride of 1, and expansion coefficient of 2 to obtain data with features of 20×20×512;

[0186] Layer 10: SPPF module, 3×3 pooling window, stride of 1, yields data with features of 20×20×512.

[0187] In some embodiments, the method further comprises:

[0188] The processing module is used to replace the CSP module in the Neck module with the IGCV3 module to form a new network structure for the Neck module as follows:

[0189] First layer: Conv module, 256 convolutional kernels of size 1×1, stride of 1, resulting in data with features of 20×20×256;

[0190] Second layer: 2x upsampling, resulting in data with features of 40×40×256;

[0191] Third layer: Connect the output of the second layer of the new Neck module network structure to the output of the seventh layer of the new Backbone module network structure, and output data with a feature of 40×40×512;

[0192] The fourth layer: IGCV3 module, with 256 convolutional kernels, a stride of 1, and an amplification factor of 2, yields data with features of 40×40×256;

[0193] Fifth layer: Conv module, 128 convolutional kernels of size 1×1, stride of 1, and expansion coefficient of 2, resulting in data with features of 40×40×128;

[0194] Sixth layer: 2x upsampling, resulting in data with features of 80×80×128;

[0195] Seventh layer: Connect the output of the sixth layer of the new Neck module's network structure to the output of the fifth layer of the new Backbone module's network structure, and output data with characteristics of 80×80×256;

[0196] The eighth layer: IGCV3 module, with 128 convolutional kernels, stride of 1, and expansion coefficient of 2, yields data with features of 80×80×128;

[0197] Ninth layer: Conv module, 128 convolutional kernels of size 3×3, stride of 2, resulting in data with features of 40×40×128;

[0198] Layer 10: Connects the outputs of the ninth and fifth layers of the new Neck module's network structure, outputting data with a feature of 40×40×256;

[0199] The eleventh layer: IGCV3 module, 256 convolutional kernels, stride of 1, and expansion coefficient of 2, yielding data with features of 40×40×256;

[0200] The twelfth layer: Cnov module, with 256 convolutional kernels of size 3×3 and a stride of 2, yields data with features of 20×20×256;

[0201] Thirteenth layer: Connect the outputs of the twelfth and first layers of the new Neck module's network structure, and output data with a feature of 20×20×512;

[0202] The fourteenth layer: IGCV3 module, 512 convolutional kernels, stride of 1, expansion coefficient of 2, resulting in data with features of 20×20×512;

[0203] The fifteenth layer: the Detect module, which obtains data with features of 80×80, 40×40 and 20×20 respectively, for detecting targets of different sizes.

[0204] In some embodiments, the method further comprises:

[0205] The processing module is used to train the caries detection model using the training set;

[0206] The processing module is used to validate the trained caries detection model using the validation set;

[0207] The processing module is used to determine whether the loss curve of the validated caries detection model has converged; if so, it determines that the caries detection model has stabilized and stops training the caries detection model; and / or,

[0208] The processing module is configured to determine, if not, that the caries detection model has not stabilized and to continue training the caries detection model.

[0209] In some embodiments, the method further comprises:

[0210] The output module is used for a professional dentist to objectively judge the test set to be identified, compare the results obtained by the dentist with the prediction results of the model, and determine the accuracy of the prediction results; if the accuracy is less than a preset accuracy threshold, then training is stopped and the caries detection model is output; and / or,

[0211] The processing module is configured to continue training the caries detection model if the accuracy is not less than the preset accuracy threshold.

[0212] like Figure 5 As shown, this embodiment of the invention also provides a terminal, which includes a processor 301 and a memory 302 for computer programs; wherein, when the processor 301 is used to run the computer program, it implements the lightweight caries detection system and method based on the fusion of YOLOv5 and IGCV3 described in any embodiment of the invention, and the memory 302 stores instructions and data.

[0213] In some embodiments of the present invention, the memory 302 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 302 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0214] The processor 301 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 301 or by instructions in software form. The processor 301 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 302, and the processor 301 reads the information in memory 302 and, in conjunction with its hardware, completes the steps of the above method.

[0215] In some embodiments, the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0216] For software implementation, the techniques described herein can be achieved through modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or externally.

[0217] Another embodiment of the present invention provides a computer storage medium storing an executable program. When the executable program is executed by a processor 301, it can implement the steps of an information processing method applied to the terminal. For example, such as... Figure 1 The method shown.

[0218] In some embodiments, the computer storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0219] It should be noted that the technical solutions described in the embodiments of the present invention can be combined arbitrarily without conflict.

[0220] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A lightweight caries detection method based on the fusion of YOLOv5 and IGCV3, characterized in that, The method includes: Collect dental caries data and construct a sample dataset; The sample dataset is preprocessed and divided into a training set, a validation set, and a test set; The target detection algorithm YOLOv5 is fused with the lightweight network algorithm IGCV3 to construct a caries detection model; the caries detection model is trained and validated using the training set and validation set. The validated caries detection model is used to predict the test set to be identified; Based on the predicted results, the caries detection model is output and deployed to a mobile platform for caries detection; the process of fusing the target detection algorithm YOLOv5 with the lightweight network algorithm IGCV3 to construct the caries detection model includes: A YOLOv5 model is built within a deep learning framework to construct an object detection network; wherein the object detection network includes: an input module, a backbone module, a neck module, and a head module; The IGCV3 module is integrated into the target detection network to construct the caries detection model; The IGCV3 module replaces some modules in the Backbone module to form a new Backbone module; The IGCV3 module replaces the C3 module in the Neck module to form a new Neck module.

2. The method according to claim 1, characterized in that, The preprocessing of the sample dataset and its division into training, validation, and test sets includes: The sample dataset was labeled using a labeling tool; The labeled sample dataset is divided into a training set, a validation set, and a test set according to a preset ratio.

3. The method according to claim 1, characterized in that: The new network structure of the Backbone module is as follows: First layer: Conv module, 32 convolutional kernels of size 6×6, stride of 2, resulting in data with features of 320×320×32; The second layer: Conv module, with 64 convolutional kernels of size 3×3 and a stride of 1, yields data with features of 160×160×64; The third layer: repeat the C3 module 3 times, with 64 convolutional kernels of size 3×3 and a stride of 1, to obtain data with features of 160×160×64; The fourth layer: Conv module, with 128 convolutional kernels of size 3×3 and a stride of 2, yields data with features of 80×80×128; Fifth layer: Repeat 6 C3 modules, 128 convolutional kernels of size 3×3, stride 1, to obtain data with features of 80×80×128; The sixth layer: IGCV3 module, 256 convolutional kernels, stride of 2, and expansion coefficient of 2, yielding data with features of 40×40×256; Layer 7: Repeat 9 IGCV3 modules, 256 convolutional kernels, stride of 1, and expansion coefficient of 2 to obtain data with features of 40×40×256; The eighth layer: IGCV3 module, 512 convolutional kernels, stride of 2, and expansion coefficient of 2, yielding data with features of 20×20×512; Ninth layer: Repeat 3 IGCV3 modules, 512 convolutional kernels, stride of 1, and expansion coefficient of 2 to obtain data with features of 20×20×512; Layer 10: SPPF module, 3×3 pooling window, stride of 1, yields data with features of 20×20×512.

4. The method according to claim 3, characterized in that: The new network structure of the Neck module is as follows: First layer: Conv module, 256 convolutional kernels of size 1×1, stride of 1, resulting in data with features of 20×20×256; Second layer: 2x upsampling, resulting in data with features of 40×40×256; Third layer: Connect the output of the second layer of the new Neck module network structure to the output of the seventh layer of the new Backbone module network structure, and output data with a feature of 40×40×512; The fourth layer: IGCV3 module, with 256 convolutional kernels, a stride of 1, and an amplification factor of 2, yields data with features of 40×40×256; Fifth layer: Conv module, 128 convolutional kernels of size 1×1, stride of 1, and expansion coefficient of 2, resulting in data with features of 40×40×128; Sixth layer: 2x upsampling, resulting in data with features of 80×80×128; Seventh layer: Connect the output of the sixth layer of the new Neck module's network structure to the output of the fifth layer of the new Backbone module's network structure, and output data with characteristics of 80×80×256; The eighth layer: IGCV3 module, with 128 convolutional kernels, stride of 1, and expansion coefficient of 2, yields data with features of 80×80×128; Ninth layer: Conv module, 128 convolutional kernels of size 3×3, stride of 2, resulting in data with features of 40×40×128; Layer 10: Connects the outputs of the ninth and fifth layers of the new Neck module's network structure, outputting data with a feature of 40×40×256; The eleventh layer: IGCV3 module, 256 convolutional kernels, stride of 1, and expansion coefficient of 2, yielding data with features of 40×40×256; The twelfth layer: Cnov module, with 256 convolutional kernels of size 3×3 and a stride of 2, yields data with features of 20×20×256; Thirteenth layer: Connect the outputs of the twelfth and first layers of the new Neck module's network structure, and output data with a feature of 20×20×512; The fourteenth layer: IGCV3 module, 512 convolutional kernels, stride of 1, expansion coefficient of 2, resulting in data with features of 20×20×512; The fifteenth layer: the Detect module, which obtains data with features of 80×80, 40×40 and 20×20 respectively, for detecting targets of different sizes.

5. The method according to claim 1, characterized in that, The step of training and validating the caries detection model using the training and validation sets includes: The caries detection model is trained using the training set. The trained caries detection model was validated using the validation set. Determine whether the loss curve of the validated caries detection model has converged; if so, determine that the caries detection model has stabilized and stop training the caries detection model; and / or, If not, the caries detection model is determined to be unstable, and training of the caries detection model continues.

6. The method according to claim 1, characterized in that, The method further includes: The test set to be identified is objectively judged by a professional dentist, and the results obtained by the dentist are compared with the prediction results of the model to determine the accuracy of the prediction results; if the accuracy is less than a preset accuracy threshold, training is stopped and the caries detection model is output; and / or, If the accuracy is not less than the preset accuracy threshold, then continue training the caries detection model.

7. A device for a lightweight caries detection method based on the fusion of YOLOv5 and IGCV3, characterized in that, The device includes: The acquisition module is used to collect dental caries data and build a sample dataset; The processing module is used to preprocess the sample dataset and divide it into training set, validation set and test set; the processing module is also used to fuse the target detection algorithm YOLOv5 with the lightweight network algorithm IGCV3 to construct a caries detection model; The processing module is also used to train and validate the caries detection model using the training set and validation set; The processing module is also used to make predictions on the test set to be identified using the validated caries detection model; The output module is used to output the caries detection model based on the prediction results and deploy it to a mobile platform for caries detection; The method of fusing the target detection algorithm YOLOv5 with the lightweight network algorithm IGCV3 to construct a caries detection model includes: A YOLOv5 model is built within a deep learning framework to construct an object detection network; wherein the object detection network includes: an input module, a backbone module, a neck module, and a head module; The IGCV3 module is integrated into the target detection network to construct the caries detection model; The IGCV3 module replaces some modules in the Backbone module to form a new Backbone module; The IGCV3 module replaces the C3 module in the Neck module to form a new Neck module.

8. A terminal, characterized in that, The terminal includes a processor and a memory for storing a computer program that can run on the processor; wherein, when the processor runs the computer program, it implements the lightweight caries detection method based on the fusion of YOLOv5 and IGCV3 as described in any one of claims 1-7.

9. A computer-readable storage medium having computer-executable instructions thereon, characterized in that, The computer-executable instructions are executed by the processor to implement the lightweight caries detection method based on the fusion of YOLOv5 and IGCV3 as described in any one of claims 1-7.

Images