Intelligent tongue coating detection system and detection method based on YOLOv5 deep learning algorithm

Abstract

The present invention relates to the technical field of computer application in TCM constitution identification, and in particular to an intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm. The system comprises an acquisition device and a processing system for acquiring tongue images, wherein the processing system comprises: a standard scene sample data set; a target detection module, which is used to judge the integrity and standardization of the tongue image acquired by the acquisition device in real time, and output a qualified tongue image; the target detection module constructs a detection network based on the YOLOv5s framework, the detection network performs multi-scale feature extraction and fusion on the qualified tongue image, compares the image with the standard scene sample data set, and outputs the qualified tongue image after quality inspection; a pathology analysis module is used to perform deep feature extraction on the qualified tongue image that passes the quality inspection of the target detection module, perform feature comparison with the standard scene sample data set, and output the detection result. The present invention achieves millisecond-level response through an event-driven architecture, ensuring acquisition efficiency and data reliability.

Description

Technical Field

[0001] The present invention relates to the technical field of computer application in TCM constitution identification, and specifically to an intelligent tongue coating detection system and a detection method thereof based on the YOLOv5 deep learning algorithm. Background Art

[0002] Tongue diagnosis, as an important component of the four diagnostic methods in Traditional Chinese Medicine, plays an irreplaceable role in disease diagnosis and constitution identification. By observing the patient's tongue shape, tongue color, tongue coating thickness and distribution, and combining clinical experience, the physician can judge the strength and weakness of the body's internal organs, the nature of the pathogenic factors, and the severity of the disease. Traditional tongue diagnosis relies on the physician's visual observation, which is limited by factors such as differences in subjective experience, interference from lighting conditions, and dynamic changes in the patient's tongue. There are problems such as inconsistent diagnostic standards and difficulty in quantifying features. Especially in modern medical scenarios, facing the growing demand for primary medical care, how to achieve standardized collection and intelligent analysis of tongue images has become a key technical bottleneck in improving the efficiency of TCM diagnosis and treatment.

[0003] Traditional tongue coating detection equipment mostly uses a fixed optical acquisition device combined with a threshold segmentation algorithm. This type of method is easily interfered by factors such as ambient lighting, saliva reflections, and tongue motion blur, resulting in insufficient contour extraction accuracy. More seriously, the existing system lacks the ability to judge the quality of the collected images in real time. Unqualified images such as tongue deviation and abnormal mouth opening angle caused by insufficient patient cooperation often enter the analysis process, seriously affecting the reliability of subsequent pathological feature extraction. Therefore, the development of an intelligent tongue coating analysis system with real-time quality detection capabilities has important clinical value in promoting the intelligent upgrade of traditional Chinese medicine diagnosis and treatment equipment.

[0004] At the same time, traditional tongue coating detection equipment has the problems of large size and high price, which makes it difficult to widely popularize. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide an intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm, comprising an acquisition device for acquiring tongue images and a processing system, wherein the processing system comprises:

[0007] Standard scene sample data set, which includes common abnormal tongue coating morphologies and typical unqualified samples;

[0008] An object detection module is used to determine the integrity and standardization of the tongue image collected by the acquisition device in real time and output a qualified tongue image. The object detection module also constructs a detection network based on the YOLOv5s framework. The detection network is used to extract and fuse multi-scale features of the qualified tongue image, compare it with the standard scene sample data set, and output a tongue image that has passed the quality inspection.

[0009] The pathological analysis module is used to perform deep feature extraction on tongue image graphics that have passed quality inspection, perform feature comparison with the standard scene sample data set, and output the detection results, which are the pathological analysis results.

[0010] The standard scene sample dataset is simulated and trained through a basic enhancement configuration module, a mosaic enhancement module, a hybrid enhancement module, and a random perspective transformation module. The basic enhancement configuration module is used to simulate the image color deviation caused by shooting at different distances. The mosaic enhancement module, the hybrid enhancement module, and the random perspective transformation module are used to simulate the tongue deformation characteristics under different viewing angles, thereby enhancing the adaptability of the target detection module to non-standard shooting angles.

[0011] In some embodiments, the basic enhancement configuration module is provided with a spatial joint adjustment parameter of hue-saturation-lightness, and the mosaic enhancement module constructs a composite training sample containing multi-scale tongue targets by randomly selecting multiple original images for splicing and reassembly;

[0012] The hybrid enhancement module uses a linear interpolation algorithm to fuse multiple input images and their annotation information, and sets a preset activation probability to balance the dialectical relationship between improving data diversity and introducing label noise;

[0013] The random perspective transformation module is used to simulate the three-dimensional deformation characteristics of the tongue under different viewing angles, thereby enhancing the adaptability of the target detection module to non-standard shooting angles.

[0014] In some embodiments, the detection network includes a backbone network, a neck network, and a detection head; wherein:

[0015] The backbone network consists of a Focus module, a multi-level C3 module, and an SPPF pooling layer. The Focus module achieves channel expansion and spatial downsampling by splicing image slices. The multi-level C3 module adopts a cross-stage partial connection strategy and reduces the number of parameters by 20%-30% through bottleneck design. The SPPF pooling layer replaces traditional multi-scale parallel pooling with serial maximum pooling.

[0016] The neck network integrates an FPN+PAN bidirectional feature pyramid structure, achieving multi-scale feature fusion through upsampling, downsampling, and lateral connections. The top-down FPN path transmits high-level semantic information, while the bottom-up PAN path optimizes the underlying positioning accuracy, ultimately forming a three-level feature map corresponding to tongue detection tasks at small, medium, and large scales.

[0017] The detection head introduces a dynamic anchor box mechanism, predefines multiple sets of initial anchor box sizes based on the K-means++ clustering algorithm, optimizes the aspect ratio matching through an adaptive adjustment strategy, and the output dimensions of each detection head include bounding box coordinates, confidence level, and classification probability.

[0018] In some embodiments, the module configuration file of the target detection module adopts a modular definition system, and each layer structure declares the input source, number of repetitions, module type and configuration parameters through a four-tuple parameter.

[0019] In some embodiments, the division of the standard scene sample data set uses an automated script tool to divide the original data into training set and validation set according to a preset ratio based on the stratified sampling principle; a random seed is set during the division process to ensure the reproducibility of the experiment, and the output directory structure strictly follows the standard format of the COCO data set.

[0020] In some embodiments, the standard scenario sample data set verification process includes a data quality check system, and the data quality check system includes a triple verification mechanism:

[0021] The mapping relationship between label files and images is detected through the annotation integrity verification script to eliminate matching errors caused by inconsistent file name capitalization.

[0022] Perform statistical information analysis, generate category distribution histograms to reveal sample balance, draw target size distribution heatmaps to assist in identifying abnormally labeled samples, and visualize the spatial distribution patterns of the tongue area through position density maps;

[0023] Interactive visualization tools are used to manually review single samples, while batch verification scripts are used to automatically detect typical problems such as bounding box overflow and label omission.

[0024] In some embodiments, the intelligent tongue coating detection system further comprises an embedded intelligent terminal in communication with the acquisition device, and the processing system is loaded into the embedded terminal;

[0025] Preferably, the embedded intelligent terminal is an embedded intelligent terminal built on the RV1126 heterogeneous computing platform;

[0026] Preferably, the acquisition device includes a CMOS sensor and a fill light group, and both the CMOS sensor and the fill light group are connected to the embedded intelligent terminal.

[0027] To achieve the above objectives, the present invention also provides the following technical solution: an intelligent tongue coating detection method based on the YOLOv5 deep learning algorithm, using the intelligent tongue coating detection system for detection, and the detection method includes:

[0028] (1) collecting tongue image through a collection device;

[0029] (2) Real-time determination of the integrity and standardization of the collected tongue image, and output of qualified tongue image graphics;

[0030] (3) Using a detection network built based on the YOLOv5s framework to perform multi-scale feature extraction and fusion on the qualified tongue image graph, and comparing it with a standard scene sample dataset, outputting a tongue image graph that has passed the quality inspection;

[0031] (4) Performing deep feature extraction on the tongue image that has passed the quality inspection, performing feature comparison with the standard scene sample data set, and outputting the pathological analysis results.

[0032] In some embodiments, the method for collecting qualified tongue image patterns includes:

[0033] (1) Start the detection system. The system automatically loads the camera driver of the acquisition device and initializes the hardware module. It also renders the camera image to the interactive interface of the detection system in real time and starts the fill light component at the same time.

[0034] (2) When the patient fully extends the tongue into the detection cavity of the collection device, the detection system dynamically tracks the tongue coating area and draws a high-precision detection frame in real time on the interactive interface;

[0035] (3) If the detection frame is stable and meets the preset tongue ratio threshold, the acquisition instruction is triggered, and the acquired image is white-balanced. The optimized image is then input into the target detection module for double verification;

[0036] (4) First, locate the minimum bounding rectangle of the tongue area in the image, and then evaluate the image quality.

[0037] If the tongue coating is missing or the image quality does not meet the standards after verification, the system will immediately play a directional voice prompt, lock the camera function and clear the cache data, and re-capture the qualified tongue image through the acquisition device;

[0038] If the verification is successful, the ROI area in the image is extracted, and the device fingerprint and millisecond timestamp are attached, and transmitted to the pathology analysis module through an encrypted channel;

[0039] (5) All qualified images verified by the target detection module are stored in the local database in a standardized format.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] Introducing target detection technology into Traditional Chinese Medicine tongue diagnosis breaks away from the traditional approach of prioritizing analysis over quality control. By designing a hierarchical processing architecture, we leverage the edge computing advantages of the embedded platform while retaining sufficient computing power for subsequent detailed analysis. At the algorithmic level, we propose an attention-guided feature enhancement mechanism to effectively improve the detection accuracy of small tongue targets in complex backgrounds. In terms of engineering implementation, we develop a mixed-precision quantization strategy and a dynamic power management module to optimize the energy efficiency of medical-grade equipment while maintaining manageable accuracy loss.

[0042] This system has three core advantages: first, it establishes standardized tongue image collection specifications to improve the reliability of diagnostic data through real-time quality feedback; second, it adopts an explainable detection module, and the output results are accompanied by heat maps of key areas to facilitate manual review by physicians; third, it builds an open analysis interface to support seamless integration of third-party pathology modules, providing a basic platform for the construction of an intelligent diagnosis and treatment ecosystem in Traditional Chinese Medicine.

[0043] Details of one or more embodiments of the present application are presented in the following drawings and descriptions to make other features, purposes and advantages of the present application more concise and easy to understand, and the present application is fully described and understood through the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of the system structure principle of the present invention;

[0045] Figure 2 This is a schematic diagram of the structure principle of the standard scene sample data set of the present invention;

[0046] Figure 3 Schematic diagram of the structure principle of the target detection module of the present invention;

[0047] Figure 4 Flowchart of the data acquisition method of the present invention;

[0048] Figure 5 Flow chart of the detection method of the present invention;

[0049] Figure 6 Schematic diagram of the application scenario of the detection system. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] See also Figure 1 The present invention provides an intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm, comprising an acquisition device for acquiring tongue images and a processing system, wherein the processing system comprises:

[0052] Standard scene sample data set, which includes common abnormal tongue coating morphologies and typical unqualified samples;

[0053] An object detection module is used to determine the integrity and standardization of the tongue image collected by the acquisition device in real time and output a qualified tongue image. The object detection module also constructs a detection network based on the YOLOv5s framework. The detection network is used to extract and fuse multi-scale features of the qualified tongue image, compare it with the standard scene sample data set, and output a tongue image that has passed the quality inspection.

[0054] The pathological analysis module is used to perform deep feature extraction on the tongue image that has passed the quality inspection, perform feature comparison with the standard scene sample data set, and output the detection result. The detection result is the pathological analysis result or recommended pathological analysis result output after comparing the same or similar confirmed samples in the standard scene sample data set.

[0055] This system utilizes a two-stage processing architecture to decouple tongue image quality detection from pathological analysis. The first stage deploys a lightweight object detection module, responsible for real-time determination of the integrity and standardization of the tongue region in images captured by the acquisition device. The second stage integrates with a specialized pathological analysis module, performing in-depth feature extraction only on tongue images that pass quality inspection. This architectural design ensures the real-time responsiveness of front-end equipment while providing independent computational resource allocation for business modules with varying precision requirements.

[0056] Target detection module implementation steps:

[0057] Multi-scale feature extraction and fusion: This is accomplished using the backbone network, neck network, and detection head within the YOLOv5s framework. The backbone network's Focus module slices and splices the input image to achieve channel expansion and spatial downsampling. The multi-level C3 module employs a cross-stage partial connection strategy, reducing the number of parameters through bottleneck design. The SPPF pooling layer replaces traditional multi-scale parallel pooling with serial maximum pooling. The neck network integrates an FPN+PAN bidirectional feature pyramid structure, achieving multi-scale feature fusion through upsampling, downsampling, and lateral connections. The detection head introduces a dynamic anchor frame mechanism, pre-defines multiple sets of initial anchor frame sizes based on the K-means++ clustering algorithm, and optimizes the aspect ratio matching through an adaptive adjustment strategy.

[0058] Real-time determination of tongue integrity and normality: An object detection algorithm dynamically tracks the tongue coating, drawing a high-precision detection frame in real time on the interactive interface. If the detection frame is stable and meets the preset tongue ratio threshold, a photo is triggered. The photo is then white-balanced and fed into the object detection module for dual verification, including locating the minimum bounding rectangle of the tongue area and assessing image quality.

[0059] Pathology analysis module implementation steps:

[0060] Deep feature extraction: For qualified tongue images that have passed the quality inspection of the target detection module, deep learning algorithms are used to extract features. For example, models such as convolutional neural networks (CNNs) are used to automatically learn deep features in tongue images, including tongue color, tongue coating thickness, distribution, and other information.

[0061] Feature comparison and output of test results: The extracted deep features are compared with the features in the standard scene sample data set. By calculating indicators such as similarity and distance, the pathological state corresponding to the tongue image is determined, and the test results are output, such as the type of abnormal tongue coating and the severity of the disease.

[0062] The standard scene sample dataset is simulated and trained through a basic enhancement configuration module, a mosaic enhancement module, a hybrid enhancement module, and a random perspective transformation module. The basic enhancement configuration module is used to simulate the image color deviation caused by shooting at different distances. The mosaic enhancement module, the hybrid enhancement module, and the random perspective transformation module are used to simulate the tongue deformation characteristics under different viewing angles, thereby enhancing the adaptability of the target detection module to non-standard shooting angles.

[0063] During the training process of deep learning modules using standard scene sample datasets, data augmentation strategies are a key technical path to improve the generalization capabilities of the modules. By integrating a multi-level data augmentation pipeline through the YOLOv5 framework and flexibly adjusting optical and geometric transformation parameters through configuration files, we can effectively address practical issues such as uneven illumination and body position deviation in medical image acquisition scenarios.

[0064] The basic enhancement configuration module utilizes a joint hue-saturation-value (HSV) space adjustment scheme. The hue enhancement amplitude parameter is set to 0.015 to control physiological color deviations, and the saturation and value adjustment coefficients are set to 0.7 and 0.4, respectively, to compensate for ambient light attenuation while preserving the texture details of the tongue coating. The geometric transformation module uses a ±5-degree random rotation tolerance to match the natural tongue swaying angle range during clinical acquisition. A 0.1-ratio translation parameter and a 0.5-magnification zoom parameter are also configured to simulate the field of view differences resulting from shooting at different distances.

[0065] The mosaic enhancement module constructs a composite training sample containing multi-scale tongue targets by splicing and reconstructing four randomly selected original images. This technology significantly improves the module's detection sensitivity for small tongue coating patches while also enhancing robustness to partial occlusions such as teeth. The parameter configuration maintains a 100% activation probability to ensure that each batch of training data undergoes spatial reconstructing.

[0066] The hybrid enhancement module uses a linear interpolation algorithm to fuse two input images and their annotation information, and sets the activation probability to 0.1 to balance the dialectical relationship between improving data diversity and introducing label noise.

[0067] The random perspective transformation module simulates the three-dimensional deformation characteristics of the tongue under different viewing angles through the parameter configuration of the 0.0005 coefficient, enhancing the module's adaptability to non-standard shooting angles. The angle range of the perspective transformation is ±15 degrees.

[0068] In actual applications, the standard scene sample dataset contains 12,000 high-resolution tongue images, covering 6 common types of abnormal tongue coating forms and 5 typical unqualified samples (including tongue deviation, teeth occlusion, excessive tongue extension, etc.).

[0069] The division of the standard scene sample dataset uses an officially provided automated script tool, which divides the original data into a training set and a validation set in an 8:2 ratio based on the principle of stratified sampling. A random seed is set during the division process to ensure the reproducibility of the experiment, and the output directory structure strictly follows the standard format of the COCO dataset. The data quality inspection system includes a triple verification mechanism: first, the mapping relationship between the label file and the image is detected through the annotation integrity verification script to eliminate matching errors caused by inconsistent file name capitalization; second, statistical information analysis is performed to generate a category distribution histogram to reveal the sample balance, draw a target size distribution heat map to assist in identifying abnormally labeled samples, and visualize the spatial distribution pattern of the tongue area through a position density map; finally, an interactive visualization tool is used to manually review a single sample, and the batch verification script can automatically detect typical problems such as bounding box overflow and label omission.

[0070] In practical applications, the object detection module builds a detection network based on the YOLOv5s framework. This module employs a three-stage detection framework to construct a complete feature pyramid: After the input image is normalized to 640×640 resolution, it undergoes multi-scale feature extraction and fusion through the backbone network, the neck network, and the detection head. The backbone network consists of a Focus module, a multi-level C3 module, and an SPPF pooling layer. The Focus module implements channel expansion and spatial downsampling through slice splicing, converting the input feature dimensions from 3×640×640 to 64×320×320. This architecture has evolved to a 6×6 convolutional layer since version v6.0 to improve ONNX export compatibility. The C3 module employs a cross-stage partial connection strategy, reducing the number of parameters by 20%-30% through bottleneck design, while also alleviating the vanishing gradient problem in deep networks. The SPPF module replaces traditional multi-scale parallel pooling with serial max pooling, reducing computational effort by 70% while maintaining the receptive field, significantly improving the efficiency of embedded deployments.

[0071] The neck network integrates an FPN+PAN bidirectional feature pyramid structure, achieving multi-scale feature fusion through upsampling, downsampling, and lateral connections. The top-down FPN pathway conveys high-level semantic information, while the bottom-up PAN pathway optimizes low-level localization accuracy. Ultimately, three feature maps of 80×80, 40×40, and 20×20 are generated, corresponding to small, medium, and large tongue detection tasks, respectively. The feature fusion process uses bilinear interpolation to align spatial dimensions, combined with 1×1 convolution for channel compression, to ensure smooth integration of features at different levels. The detection head incorporates a dynamic anchor box mechanism, pre-defining nine initial anchor box sizes based on the K-means++ clustering algorithm. An adaptive adjustment strategy is used to optimize aspect ratio matching. Each detection head output dimension includes bounding box coordinates, confidence score, and classification probability. The center point offset range is extended to -0.5-1.5 to improve localization accuracy for large tongues. An exponential correction strategy is used for the aspect ratio scaling factors to enhance training stability.

[0072] The module configuration file uses a modular definition system, with each layer structure declaring the input source, number of repetitions, module type, and configuration parameters through a four-tuple parameter. For example, in the backbone network, [-1,3,C3,

[256] ] indicates that a C3 module with three bottleneck layers is constructed from the output of the previous layer, with 256 output channels. The depth scaling factor and width scaling factor control the number of network layers and channels, respectively. YOLOv5s uses a depth factor of 0.33 and a width factor of 0.5, compressing the parameter size to 7.5MB while ensuring detection accuracy. This layered, configurable architecture enables the module to flexibly adapt to different hardware platforms, and by adjusting the scaling factor, a lightweight version that meets the memory constraints of the RV1126 chip can be quickly generated.

[0073] A detection network built on the YOLOv5s framework. Its full-process analysis and tuning strategy for target detection network training includes YOLOv5 detection network training, hyperparameter configuration, and training optimization.

[0074] 1. The YOLOv5 detection network training process includes the following core steps:

[0075] (1) Backbone network configuration and feature extraction

[0076] CSPDarknet53 is constructed as the backbone feature extractor, with the input size fixed to {Xin∈R3×640×640}. The network consists of five stages of downsampling operations:

[0077] Stage 0: The Focus structure reorganizes the input slice into 12×320×320, and outputs {C0∈R64×320×320} through the CBL module (Conv+BN+LeakyReLU);

[0078] Stages 1-4: Each stage contains a CSP module and stride 2 convolution downsampling, and finally outputs high-level semantic features {F5∈R1024×20×20};

[0079] Feature Pyramid Network (FPN) fuses multi-scale features and constructs a three-level detection head {P3∈R256×80×80, P4∈R512×40×40, P5∈R024×20×20} through upsampling and Concat operations;

[0080] (2) Dynamic detection head optimization mechanism

[0081] Introduce the adaptive spatial feature fusion module (ASFF) before each detection head:

[0082] Perform 1×1 convolution on the P3-P5 features to align the channels to 256 dimensions and generate {Q3, Q4, Q5};

[0083] Calculate the cross-scale weight matrix W∈R3×H×W, the formula is Wijk=σ(Conv([Qi,Qj,Qk]));

[0084] Weighted fusion feature Ffusion = ∑Wi⊙Qi, output optimized detection features;

[0085] (3) Intelligent training process control system

[0086] Early stopping mechanism: Monitor validation set fitness = 0.9 × mAP@0.5 + 0.1 × mAP@0.5: 0.95, and terminate training when the improvement Δ is less than 0.001 for 100 consecutive epochs;

[0087] (4) Multi-dimensional training monitoring and tuning

[0088] Visual Analysis: TensorBoard displays the training loss surface and detection box quality in real time, including:

[0089] Loss / box reflects the changing trend of positioning error;

[0090] The Precision-Recall curve shows the detection performance under different confidence thresholds;

[0091] Memory optimization: When GPU memory usage is > 90%, enable --half mixed precision training and adjust the weight update formula to: Where s is the gradient scaling factor, the default value is 65536;

[0092] After the program is executed, the extracted features are generated into a module file in .pt format for subsequent operations.

[0093] 2. Hyperparameter configuration and training optimization process

[0094] The hyperparameter configuration file serves as the core control center for module training, and its design directly impacts the convergence speed and ultimate performance of the detection network. YOLOv5 utilizes a hierarchical parameter management system, defining three major parameter groups in the default configuration file: optimizer parameters, loss function weights, and data augmentation coefficients. The initial learning rate (lr0) is set to 0.01 with a cosine annealing schedule, which is smoothly decreased to 0.002 over 300 training cycles, ensuring rapid initial convergence while avoiding late-stage oscillation. The momentum factor (momentum) is set to 0.937, using an exponentially weighted average mechanism to suppress high-frequency noise in the parameter update direction. The weight decay coefficient (weight_decay) is configured to 0.0005 to prevent overfitting caused by complex background features under the L2 regularization constraint. The data augmentation module uses a progressive adjustment strategy, enabling 100% mosaic enhancement probability and 0.5 horizontal flip probability at the beginning of training, significantly improving the detection robustness of small tongue objects. In the middle and late stages, the hue disturbance amplitude was gradually reduced to 0.015, and the brightness adjustment coefficient was maintained at 0.4 to balance the risk of color distortion and the need for lighting generalization.

[0095] In terms of optimizer configuration, for small-batch training scenarios (batch_size = 8) on the RV1126 embedded platform, the Adam optimizer with settings of β1 = 0.9 and β2 = 0.999 effectively mitigates bias in gradient direction estimation. A three-stage composite learning rate scheduling mechanism is employed: a linear warmup is implemented in the first three epochs, with the bias learning rate gradually increasing from 0.1 to the baseline value to eliminate parameter oscillation during network initialization. The main training phase utilizes a cosine annealing algorithm, with the learning rate curve decaying smoothly according to the rule lr = lr0 × (1 + cos(π × epoch / epochs)) / 2. Early stopping detection is enabled for the final 10 epochs, automatically terminating training if the validation set mAP fails to break the record for 20 consecutive times to prevent ineffective iterations from wasting computing resources.

[0096] The input image size is dynamically adjusted based on the characteristics of the detection target, maintaining a standard 640×640 resolution for the microscopic features of tongue coating texture, and maintaining edge sharpness through bicubic interpolation. The module save mechanism uses a golden copy strategy, saving a complete checkpoint (including optimizer state and hyperparameter configuration) every 10 cycles. The weighted version with the highest F1 score on the validation set is selected through cross-validation as the final output.

[0097] Deployment and optimization of detection network modules based on YOLOv5s framework

[0098] 1. ONNX-RKNN conversion process

[0099] The module conversion phase utilizes a three-stage conversion architecture: PyTorch → ONNX → RKNN. First, the trained .pth module is exported as an ONNX intermediate representation. During this process, special attention must be paid to dynamic axis settings, specifying the input dimensions as "None × 3 × 320 × 320" to support variable batch inference. Next, quantization conversion is performed using RKNN-Toolkit2, with mean = [123.675, 116.28, 103.53] and std = [58.395, 57.12, 57.375] for normalization parameter calibration. The uint8 quantization mode is selected to match the characteristics of the NPU computing module.

[0100] During quantization accuracy tuning, a calibration dataset consisting of 500 representative samples was constructed, and the KL divergence minimization strategy was used to determine the quantization parameters for each layer. A mixed-precision quantization strategy was implemented for the sensitive position prediction branch of the detection task: 8-bit quantization was maintained in the classification head, while 16-bit floating-point quantization was used in the bounding box regression head. The resulting .rknn module achieved a real-time inference speed of 15 FPS in testing, meeting the interactive requirements for instant feedback in clinical detection scenarios.

[0101] 2. Embedded Inference Engine Design

[0102] The embedded intelligent terminal is built on the RV1126 heterogeneous computing platform. The hardware platform uses the Rockchip RV1126 chipset. The processor integrates a quad-core ARM Cortex-A7 CPU and an NPU neural network acceleration module, and has a fixed-point computing capability of 2.0TOPS. Compared with traditional GPU solutions, its ultra-low power consumption is controlled within 1.5W, making it very suitable for deployment in portable medical detection equipment. At the software level, the RKNN tool chain officially provided by Rockchip is used to realize the quantitative migration of the PyTorch training module to the embedded platform through the ONNX intermediate representation, effectively solving the problems of precision loss and computing resource limitations in mobile terminal deployment.

[0103] A multi-threaded inference pipeline was built on the RV1126 platform, fully leveraging the heterogeneous CPU and NPU computing resources. The main thread is responsible for image acquisition and preprocessing, acquiring the YUV422 video stream through the V4L2 interface and calling the libyuv library to perform color space conversion and resizing. The preprocessed RGB image is fed into a dedicated NPU inference thread, using a double buffering mechanism to avoid memory copy overhead. For post-processing of detection results, a non-maximum suppression algorithm based on NMS was designed, with a confidence threshold of 0.6 and an IoU threshold of 0.45 to balance false detection and missed detection rates.

[0104] The data acquisition device includes a CMOS sensor and a fill light assembly, both of which are connected to an embedded smart terminal. The device uses an industrial-grade CMOS sensor and is equipped with a ring-shaped LED fill light assembly. During data acquisition, the subject is instructed to naturally extend their tongue, with the jaw secured using a mechanical bracket to ensure the tongue is within the standard observation area.

[0105] Through this technical solution, the system workflow is as follows:

[0106] The user connects the computing platform embedded with the detection system to a computer or mobile phone via a USB-Type-C port and activates the detection system. The system automatically loads the camera driver and initializes the hardware module, rendering the camera image to the interactive interface in real time and activating the LED fill light. When the user fully inserts the tongue into the detection cavity of the light-shielding mask, the system dynamically tracks the tongue coating area and draws a high-precision detection frame on the screen in real time.

[0107] If the detection frame is stable and meets the preset tongue coverage threshold (35% ± 5% of the frame), the user can trigger the photo command. The photo is then white-balanced and fed into the lightweight YOLOv5-Tiny module for double verification: first, the minimum bounding rectangle of the tongue area is located (IoU ≥ 0.85), and then the image quality is evaluated (including 12 indicators such as clarity, unobstructed, and non-reflective).

[0108] Verified tongue images automatically extract the ROI region, attach a device fingerprint and millisecond timestamp, and transmit it to the pathology analysis module via an encrypted channel. If the tongue coating is missing or substandard, the system immediately plays a directional voice prompt (e.g., "No valid tongue coating detected, please readjust"), locks the camera function, and clears the cached data. All qualified images are stored in a local SQLite database in a standardized format (640×640 pixels), separating the raw data and quality assessment reports. This provides structured input for subsequent TCM syndrome differentiation analysis. The entire process achieves millisecond response times through an event-driven architecture, ensuring efficient acquisition and data reliability.

[0109] Based on the above technical solution, the data collection method process is as follows:

[0110] Step 1: Start the detection system. The system automatically loads the camera driver of the acquisition device and initializes the hardware module. It also renders the camera image to the detection system's interactive interface in real time and starts the fill light component.

[0111] Step 2: When the patient fully extends his tongue into the detection cavity of the collection device, the detection system dynamically tracks the tongue coating area and draws a high-precision detection frame in real time on the interactive interface;

[0112] Step 3: If the detection frame is stable and meets the preset tongue ratio threshold, the acquisition command is triggered and the acquired image is white-balanced. The optimized image is then input into the object detection module for double verification.

[0113] Step 4: First locate the minimum bounding rectangle of the tongue area in the image, and then evaluate the image quality.

[0114] If the tongue coating is missing or the image quality does not meet the standards after verification, the system will immediately play a directional voice prompt, lock the camera function and clear the cache data, and re-capture the qualified tongue image through the acquisition device;

[0115] If the verification is successful, the preset ROI area in the qualified tongue image will be extracted, and the device fingerprint and millisecond timestamp will be attached. The area will be transmitted to the pathology analysis module through an encrypted channel for subsequent automated medical record analysis.

[0116] Step 5: All qualified images verified by the target detection module are stored in a standardized format in the local database, with the original data and quality assessment report saved separately to provide structured input for subsequent TCM syndrome differentiation analysis.

[0117] The detection device of the present invention was tested for clinical effectiveness in the Department of Traditional Chinese Medicine of a Class A tertiary hospital, with a total of 200 outpatients included in the double-blind test. Using the results jointly interpreted by three deputy chief physicians as the gold standard, the system achieved an accuracy rate of 92.3% in tongue coating qualification testing, significantly higher than the 76.8% of the traditional threshold method. In terms of specificity, the recognition accuracy rate for 20 common interference scenarios (such as the edge of a mask entering the camera, finger accidental touch, etc.) reached 89.5%, demonstrating the system's good robustness.

[0118] Performance test data showed that under standard lighting conditions, the system's average response time was 136ms. In strong backlight conditions, the latency increased to 203ms, still meeting real-time requirements. Power consumption tests showed that after eight hours of continuous operation, the system consumed only 742mAh. Combined with a 6000mAh lithium battery, this system enables all-day mobile diagnosis and treatment. In long-term stability testing, the device operated continuously for 72 hours without experiencing memory leaks or freezes, and the fluctuation of core indicators was controlled within ±2%.

[0119] With the increasing demand for intelligent tongue diagnosis equipment in primary healthcare institutions, this system is expected to be applied in scenarios such as community TCM constitution identification stations and mobile diagnosis and treatment vehicles, significantly improving the efficiency and quality of family doctor contract services. Future research and development will focus on multimodal data fusion, combining tongue characteristics with pulse waves, facial diagnosis, and other information to construct comprehensive syndrome differentiation modules. At the same time, we will explore the module evolution mechanism within the federated learning framework, enabling cross-institutional knowledge sharing while protecting patient privacy, and promoting the development of TCM artificial intelligence diagnostic technology to a higher level.

[0120] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

[0121] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm, characterized by: The invention comprises an acquisition device and a processing system for acquiring tongue images, wherein the processing system comprises: Standard scene sample data set, which includes common abnormal tongue coating morphologies and typical unqualified samples; An object detection module is used to determine the integrity and standardization of the tongue image collected by the acquisition device in real time and output a qualified tongue image. The object detection module also constructs a detection network based on the YOLOv5s framework. The detection network is used to extract and fuse multi-scale features of the qualified tongue image, compare it with the standard scene sample data set, and output a tongue image that has passed the quality inspection. The pathological analysis module is used to perform deep feature extraction on the tongue image that has passed the quality inspection, perform feature comparison with the standard scene sample data set, and output the detection result, which is the pathological analysis result. The standard scene sample dataset is simulated and trained using a basic enhancement configuration module, a mosaic enhancement module, a hybrid enhancement module, and a random perspective transformation module. The basic enhancement configuration module is used to simulate image color deviations resulting from shooting at different distances. The mosaic enhancement module, the hybrid enhancement module, and the random perspective transformation module are used to simulate tongue deformation characteristics under different viewing angles, thereby enhancing the adaptability of the target detection module to non-standard shooting angles. The detection network includes a backbone network, a neck network and a detection head; wherein: The backbone network consists of a Focus module, a multi-level C3 module, and an SPPF pooling layer. The Focus module achieves channel expansion and spatial downsampling by splicing image slices. The multi-level C3 module adopts a cross-stage partial connection strategy and reduces the number of parameters by 20%-30% through bottleneck design. The SPPF pooling layer replaces traditional multi-scale parallel pooling with serial maximum pooling. The neck network integrates an FPN+PAN bidirectional feature pyramid structure, achieving multi-scale feature fusion through upsampling, downsampling, and lateral connections. The top-down FPN path transmits high-level semantic information, while the bottom-up PAN path optimizes the underlying positioning accuracy, ultimately forming a three-level feature map corresponding to tongue detection tasks at small, medium, and large scales. The detection head introduces a dynamic anchor box mechanism, predefines multiple sets of initial anchor box sizes based on the K-means++ clustering algorithm, optimizes the aspect ratio matching through an adaptive adjustment strategy, and the output dimensions of each detection head include bounding box coordinates, confidence level, and classification probability.

2. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 1 is characterized in that: The basic enhancement configuration module is provided with spatial joint adjustment parameters of hue-saturation-lightness; The mosaic enhancement module randomly selects multiple original images for splicing and reassembly to construct a composite training sample containing multi-scale tongue targets; The hybrid enhancement module uses a linear interpolation algorithm to fuse multiple input images and their annotation information, and sets a preset activation probability to balance the dialectical relationship between improving data diversity and introducing label noise; The random perspective transformation module is used to simulate the three-dimensional deformation characteristics of the tongue under different viewing angles, thereby enhancing the adaptability of the target detection module to non-standard shooting angles.

3. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 1 is characterized in that: The module configuration file of the target detection module adopts a modular definition system, and each layer structure declares the input source, number of repetitions, module type and configuration parameters through a four-tuple parameter.

4. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 1, characterized in that: The standard scene sample dataset is divided into training and validation sets using an automated script tool based on the stratified sampling principle. The original data is divided into training and validation sets according to a preset ratio. A random seed is set during the division process to ensure the reproducibility of the experiment, and the output directory structure strictly follows the standard format of the COCO dataset.

5. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 4 is characterized in that: The standard scenario sample data set verification process includes a data quality inspection system, and the data quality inspection system includes a three-fold verification mechanism: The mapping relationship between label files and images is detected through the annotation integrity verification script to eliminate matching errors caused by inconsistent file name capitalization. Perform statistical information analysis, generate category distribution histograms to reveal sample balance, draw target size distribution heatmaps to assist in identifying abnormally labeled samples, and visualize the spatial distribution patterns of the tongue area through position density maps; Interactive visualization tools are used to manually review single samples, while batch verification scripts are used to automatically detect typical problems such as bounding box overflow and label omission.

6. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 1, characterized in that: The intelligent tongue coating detection system further comprises an embedded intelligent terminal in communication with the acquisition device, and the processing system is loaded in the embedded intelligent terminal; The embedded intelligent terminal is an embedded intelligent terminal built on the RV1126 heterogeneous computing platform; The acquisition device includes a CMOS sensor and a fill light assembly, and both the CMOS sensor and the fill light assembly are connected to the embedded intelligent terminal.

7. An intelligent tongue coating detection method based on the YOLOv5 deep learning algorithm, characterized by: The intelligent tongue coating detection system according to any one of claims 1 to 6 is used for detection, and the detection method comprises: (1) Collecting tongue image through a collection device; (2) Real-time determination of the integrity and standardization of the collected tongue images, and output of qualified tongue image graphics; (3) Using a detection network built based on the YOLOv5s framework to extract and fuse multi-scale features of the qualified tongue image, and comparing it with a standard scene sample dataset, output a tongue image that has passed the quality inspection; (4) Performing deep feature extraction on the tongue image that has passed the quality inspection, performing feature comparison with the standard scene sample data set, and outputting the pathological analysis results.

8. The detection method according to claim 7, wherein: Methods for collecting qualified tongue images include: (1) Start the detection system. The system automatically loads the camera driver of the acquisition device and initializes the hardware module. It also renders the camera image to the interactive interface of the detection system in real time and starts the fill light component at the same time. (2) When the patient fully extends the tongue into the detection cavity of the collection device, the detection system dynamically tracks the tongue coating area and draws a high-precision detection frame in real time on the interactive interface; (3) If the detection frame is stable and meets the preset tongue ratio threshold, the acquisition command is triggered, and the acquired image is white-balanced. The optimized image is then input into the target detection module for double verification. (4) First, locate the minimum bounding rectangle of the tongue area in the image, and then evaluate the image quality. If the tongue coating is missing or the image quality does not meet the standards after verification, the system will immediately play a directional voice prompt, lock the camera function and clear the cache data, and re-capture the qualified tongue image through the acquisition device; If the verification is successful, the ROI area in the image is extracted, and the device fingerprint and millisecond timestamp are attached, and transmitted to the pathology analysis module through an encrypted channel; (5) All qualified images verified by the target detection module are stored in a local database in a standardized format.

Images