Intelligent tongue coating detection system based on YOLOv5 deep learning algorithm and detection method thereof

Abstract

The invention relates to the technical field of computer application in traditional Chinese medicine constitution recognition, in particular to an intelligent tongue coating detection system based on a YOLOv5 deep learning algorithm, which comprises an acquisition device for acquiring tongue images and a processing system, the target detection module is used for judging the integrity and normalization of the tongue picture image acquired by the acquisition device in real time and outputting a qualified tongue picture graph; the target detection module constructs a detection network based on a YOLOv5s framework, and the detection network performs multi-scale feature extraction and fusion on the qualified tongue picture graph, compares the qualified tongue picture graph with a standard scene sample data set, and outputs the qualified tongue picture graph after quality detection; and the pathological analysis module is used for carrying out depth feature extraction on the qualified tongue picture graph subjected to quality detection by the target detection module, carrying out feature comparison with a standard scene sample data set and outputting a detection result, millisecond-level response is realized through an event-driven architecture, and the acquisition efficiency and the data reliability are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of computer applications in traditional Chinese medicine constitution recognition, and particularly relates to an intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm and its detection method. Background Art

[0002] Tongue diagnosis, as an important part of the four diagnostic methods in traditional Chinese medicine, plays an irreplaceable role in disease diagnosis and constitution identification. Physicians observe the characteristics of the patient's tongue body shape, tongue color, tongue coating thickness and distribution, etc., and combine clinical experience to judge the prosperity and decline of the viscera qi and blood of the human body, the nature of pathogenic factors, and the depth of the disease condition. Traditional tongue diagnosis relies on visual observation by physicians and is limited by factors such as subjective experience differences, light condition interference, and dynamic changes of the patient's tongue body, resulting in problems such as inconsistent diagnostic criteria and difficulty in quantifying features. Especially in the modern medical scenario, in the face of the increasing demand for primary medical care, how to achieve standardized acquisition and intelligent analysis of tongue images has become a key technical bottleneck for improving the efficiency of traditional Chinese medicine diagnosis and treatment.

[0003] Traditional tongue coating detection devices mostly adopt fixed optical acquisition devices combined with threshold segmentation algorithms. Such methods are easily interfered by factors such as environmental light, saliva reflection, and tongue body motion blur, resulting in insufficient accuracy of contour extraction. More seriously, the existing systems lack the ability to judge the quality of the acquired images in real time, and often unqualified images such as tongue body deviation and abnormal opening angle caused by insufficient patient cooperation enter the analysis process, seriously affecting the reliability of subsequent pathological feature extraction. Therefore, developing an intelligent tongue coating analysis system with real-time quality detection function has important clinical value for promoting the intelligent upgrade of traditional Chinese medicine diagnosis and treatment equipment.

[0004] At the same time, traditional tongue coating detection devices also have problems such as large volume and high price, which are not convenient for widespread popularization. Summary of the Invention

[0005] Aiming at the deficiencies of the existing technology, the purpose of the present invention is to provide an intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm.

[0006] To achieve the above purpose, the present invention provides the following technical solution: An intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm, including an acquisition device for acquiring tongue image and a processing system, wherein the processing system includes:

[0007] A standard scene sample data set, the standard scene sample data set includes common abnormal tongue coating morphologies and typical unqualified samples;

[0008] The object detection module is used to determine the integrity and standardization of the tongue image collected by the collection device in real time and output a qualified tongue image; the object detection module also constructs a detection network based on the YOLOv5s framework, and the detection network is used to perform multi-scale feature extraction and fusion on the qualified tongue image, and compare it with the standard scene sample data set to output a tongue image with qualified quality detection.

[0009] The pathological analysis module is used to extract deep features from the tongue images with qualified quality detection, compare the features with the standard scene sample data set, and output a detection result, and the detection result is a pathological analysis result.

[0010] The standard scene sample data set is simulated and trained through a basic enhancement configuration module, a mosaic enhancement module, a hybrid enhancement module and a random perspective transformation module. The basic enhancement configuration module is used to simulate the image color deviation generated by shooting at different distances, and the mosaic enhancement module, the hybrid enhancement module and the random perspective transformation module are used to simulate the tongue deformation characteristics under different perspectives to enhance the adaptability of the object detection module to non-standard shooting angles.

[0011] In some embodiments, the basic enhancement configuration module sets the spatial joint adjustment parameters of hue-saturation-brightness, and the mosaic enhancement module constructs a composite training sample containing multi-scale tongue targets by randomly selecting and splicing multiple original images.

[0012] The hybrid enhancement module uses a linear interpolation algorithm to fuse multiple input images and their annotation information, and sets the enabling probability of a preset value to balance the dialectical relationship between data diversity improvement and label noise introduction.

[0013] The random perspective transformation module is used to simulate the three-dimensional deformation characteristics of the tongue under different perspectives to enhance the adaptability of the object detection module to non-standard shooting angles.

[0014] In some embodiments, the detection network includes a backbone network, a neck network and a detection head; where:

[0015] The backbone network is composed of a Focus module, a multi-stage C3 module and an SPPF pooling layer. The Focus module realizes channel expansion and spatial downsampling by splicing image slices. The multi-stage C3 module adopts a cross-stage partial connection strategy and reduces the number of parameters by 20%-30% through bottleneck design. The SPPF pooling layer replaces the traditional multi-scale parallel pooling with serial maximum pooling.

[0016] The neck network integrates an FPN+PAN bidirectional feature pyramid structure, which realizes multi-scale feature fusion through upsampling, downsampling and lateral connection. The top-down FPN path transmits high-level semantic information, and the bottom-up PAN path optimizes the bottom-layer positioning accuracy, finally forming three-level feature maps, corresponding to tongue detection tasks of small, medium and large scales respectively;

[0017] The detection head introduces a dynamic anchor box mechanism, predefines multiple groups of initial anchor box sizes based on the K-means++ clustering algorithm, optimizes the width-height ratio matching degree through an adaptive adjustment strategy, and the output dimension of each detection head includes bounding box coordinates, confidence and classification probability.

[0018] In some embodiments, the module configuration file of the object detection module adopts a modular definition system, and each layer structure declares the input source, repetition times, module type and configuration parameters through a quadruple parameter.

[0019] In some embodiments, the division link of the standard scene sample data set adopts an automated script tool, and divides the original data into a training set and a validation set according to a preset ratio based on the principle of stratified sampling; a random seed is set during the division process to ensure the reproducibility of the experiment, and the output directory structure strictly follows the COCO data set standard format.

[0020] In some embodiments, the verification link of the standard scene sample data set includes a data quality inspection system, and the data quality inspection system includes a triple verification mechanism:

[0021] Detect the mapping relationship between the label file and the image through the annotation integrity verification script to eliminate matching errors caused by inconsistent file name case;

[0022] Perform statistical information analysis, generate a category distribution histogram to reveal the sample balance, draw a heat map of the target size distribution to assist in identifying abnormally annotated samples, and visualize the spatial distribution law of the tongue region through a position density map;

[0023] Use an interactive visualization tool to manually review a single sample, and at the same time sample a batch verification script to automatically detect typical problems such as bounding box overflow and label omission.

[0024] In some embodiments, the intelligent tongue coating detection system further includes an embedded intelligent terminal communicatively connected to the acquisition device, and the processing system is loaded in the embedded terminal;

[0025] Preferably, the embedded intelligent terminal is an embedded intelligent terminal constructed based on the RV1126 heterogeneous computing platform;

[0026] Preferably, the acquisition device includes a CMOS sensor and a fill light group, and both the CMOS sensor and the fill light group are connected to the embedded intelligent terminal.

[0027] To achieve the above object, the present invention also provides the following technical solution: An intelligent tongue coating detection method based on the YOLOv5 deep learning algorithm, which uses the intelligent tongue coating detection system for detection, and the detection method includes:

[0028] (1) Collecting tongue image graphics through an acquisition device;

[0029] (2) Real-time judging the integrity and normality of the collected tongue image, and outputting a qualified tongue image graphic;

[0030] (3) Using a detection network constructed based on the YOLOv5s framework to perform multi-scale feature extraction and fusion on the qualified tongue image graphic, and comparing it with the standard scene sample data set, and outputting a tongue image graphic with qualified quality detection;

[0031] (4) Performing deep feature extraction on the tongue image graphic with qualified quality detection, comparing features with the standard scene sample data set, and outputting a pathological analysis result.

[0032] In some of the embodiments, the method for collecting a qualified tongue image includes:

[0033] (1) Starting the detection system, the system automatically loads the camera driver of the acquisition device and initializes the hardware module, and renders the camera screen to the interaction interface of the detection system in real time, and at the same time starts the fill light component;

[0034] (2) When the patient fully extends the tongue into the detection cavity of the acquisition device, the detection system dynamically tracks the tongue coating area of the tongue and draws a high-precision detection frame on the interaction interface in real time;

[0035] (3) If the detection frame is stable and meets the preset tongue body ratio threshold, trigger the acquisition instruction, perform white balance correction on the acquired image, and then input the optimized image into the target detection module for double verification;

[0036] (4) First, locate the minimum circumscribed rectangle frame of the tongue body area in the image, and secondly, evaluate the image quality.

[0037] If the tongue coating of the tongue is missing or the image quality does not meet the standard after verification, the system immediately plays a directional voice prompt, locks the photographing function and clears the cache data, and re-collects a qualified tongue image through the acquisition device;

[0038] If the verification is passed, extract the ROI area in the image, attach the device fingerprint and millisecond-level time stamp, and transmit it to the pathological analysis module through an encrypted channel.

[0039] (5) All qualified images verified by the target detection module are stored in the local database in a standardized format.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] By introducing target detection technology into traditional Chinese medicine tongue diagnosis, the technical route of "emphasizing analysis and neglecting quality inspection" in the traditional scheme is broken through. Through the design of a hierarchical processing architecture, both the edge computing advantages of the embedded platform are fully utilized, and sufficient computing power resources are reserved for subsequent fine analysis. At the algorithm level, a feature enhancement mechanism based on attention guidance is proposed, effectively improving the detection accuracy of the small target tongue body under complex backgrounds. In terms of engineering implementation, a mixed-precision quantization strategy and a dynamic power consumption management module are developed to optimize the energy efficiency of medical-grade devices on the premise of controllable accuracy loss.

[0042] This system has three core advantages: First, a standardized tongue image acquisition specification is established to improve the reliability of diagnostic data through real-time quality feedback; second, an interpretable detection module is adopted, and the output result is accompanied by a heat map of the key area, which is convenient for doctors to conduct manual review; third, an open analysis interface is constructed to support the seamless access of third-party pathology modules, providing a basic platform for the construction of the intelligent diagnosis and treatment ecosystem of traditional Chinese medicine.

[0043] The details of one or more embodiments of this application are presented in the following drawings and descriptions, so that other features, purposes, and advantages of this application are more concise and understandable. The application is described in detail and understood through the embodiments of this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the system structure principle of the present invention;

[0045] Figure 2 It is a schematic diagram of the structure principle of the standard scenario sample data set of the present invention;

[0046] Figure 3 It is a schematic diagram of the structure principle of the target detection module of the present invention;

[0047] Figure 4 It is a flowchart of the data acquisition method of the present invention;

[0048] Figure 5 It is a flowchart of the detection method of the present invention;

[0049] Figure 6 It is a schematic diagram of the application scenario of the detection system. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0051] Please refer to Figure 1 , the present invention provides an intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm, including a collection device for collecting tongue image and a processing system, wherein the processing system includes:

[0052] A standard scene sample data set, which includes common abnormal tongue coating morphologies and typical unqualified samples;

[0053] A target detection module, which is used to real-time judge the integrity and standardization of the tongue image collected by the collection device, and output a qualified tongue image; the target detection module also constructs a detection network based on the YOLOv5s framework, and the detection network is used to perform multi-scale feature extraction and fusion on the qualified tongue image, and compare it with the standard scene sample data set, and output a tongue image with qualified quality detection;

[0054] A pathological analysis module, which is used to perform deep feature extraction on the tongue image with qualified quality detection, and compare the features with the standard scene sample data set, and output a detection result, and the detection result is the pathological analysis result output after comparing with the same or similar determined samples in the standard scene sample data set or the recommended pathological analysis result.

[0055] This system uses a two-stage processing architecture to decouple the functions of tongue image quality detection and pathological analysis. In the first stage, a lightweight target detection module is deployed, which is responsible for real-time judging the integrity and standardization of the tongue body area in the image collected by the collection device; in the second stage, a professional pathological analysis module is docked, and only the qualified tongue images that pass the quality detection are subjected to deep feature extraction. This architecture design not only ensures the real-time response ability of the front-end device, but also provides an independent operation resource allocation scheme for business modules with different precision requirements.

[0056] Steps for implementing the target detection module:

[0057] Multi-scale Feature Extraction and Fusion: It is completed using the backbone network, neck network, and detection head in the YOLOv5s framework. The Focus module in the backbone network slices and splices the input image to achieve channel expansion and spatial downsampling; the multi-level C3 module adopts a cross-stage partial connection strategy to reduce the number of parameters through bottleneck design; the SPPF pooling layer replaces traditional multi-scale parallel pooling with serial maximum pooling. The neck network integrates the FPN+PAN bidirectional feature pyramid structure to achieve multi-scale feature fusion through upsampling, downsampling, and lateral connections. The detection head introduces a dynamic anchor box mechanism, predefines multiple groups of initial anchor box sizes based on the K-means++ clustering algorithm, and optimizes the width-height ratio matching degree through an adaptive adjustment strategy.

[0058] Real-time Discrimination of the Integrity and Normativity of the Tongue Body Region: Run the object detection algorithm to dynamically track the tongue coating region and draw a high-precision detection box on the interactive interface in real time. If the detection box is stable and meets the preset tongue body ratio threshold, trigger the photo-taking command. After the photo is corrected for white balance, it is input into the object detection module for double verification, including locating the minimum bounding rectangle of the tongue body region and evaluating the image quality.

[0059] Implementation Steps of the Pathological Analysis Module:

[0060] Deep Feature Extraction: For the qualified tongue image graphics that pass the quality detection of the object detection module, use deep learning algorithms for feature extraction, such as using models like convolutional neural networks (CNNs) to automatically learn the deep features in the tongue image, including information such as tongue body color, tongue coating thickness, and distribution.

[0061] Feature Comparison and Output of Detection Results: Compare the extracted deep features with the features in the standard scene sample dataset. By calculating metrics such as similarity and distance, judge the pathological state corresponding to the tongue image, and output the detection results, such as the type of abnormal tongue coating and the degree of illness.

[0062] The standard scene sample dataset is simulated and trained through the basic enhancement configuration module, mosaic enhancement module, hybrid enhancement module, and random perspective transformation module. The basic enhancement configuration module is used to simulate the color deviation of images generated by shooting at different distances, and the mosaic enhancement module, hybrid enhancement module, and random perspective transformation module are used to simulate the tongue body deformation characteristics under different perspectives to enhance the adaptability of the object detection module to non-standard shooting angles.

[0063] During the training process of the deep learning module for the standard scene sample dataset, the data augmentation strategy is the key technical path to improve the generalization ability of the module. Through the YOLOv5 framework, a multi-level data augmentation pipeline is integrated, and the optical transformation and geometric transformation parameters are flexibly adjusted through the configuration file to effectively address practical problems such as uneven lighting and body position offset in the medical image acquisition scenario.

[0064] The basic enhancement configuration module adopts a combined adjustment scheme in the Hue-Saturation-Value (HSV) space. The hue enhancement amplitude parameter is set to 0.015 to control the physiological color deviation. The saturation and value adjustment coefficients are set to 0.7 and 0.4 respectively, which can compensate for the environmental light attenuation while maintaining the details of the tongue coating texture. The geometric transformation part sets a random rotation tolerance of ±5 degrees to match the angular range of the natural swing of the patient's tongue during clinical acquisition. At the same time, a translation parameter with a ratio of 0.1 and a scaling parameter with a magnification of 0.5 are configured to simulate the field-of-view differences generated by shooting at different distances.

[0065] The mosaic enhancement module constructs a composite training sample containing multi-scale tongue targets by randomly selecting and splicing four original images. This technology significantly improves the detection sensitivity of the module to small-area tongue coating patches and enhances the robustness to partially occluded scenarios such as tooth occlusion. At the parameter configuration level, a 100% enabling probability is maintained to ensure that each batch of training data undergoes spatial recombination processing.

[0066] The hybrid enhancement module uses a linear interpolation algorithm to fuse two input images and their annotation information, and sets an enabling probability of 0.1 to balance the dialectical relationship between data diversity improvement and label noise introduction.

[0067] The random perspective transformation module simulates the three-dimensional deformation characteristics of the tongue under different perspectives through a parameter configuration with a coefficient of 0.0005, enhancing the adaptability of the module to non-standard shooting angles. The angular range of the perspective transformation is ±15 degrees.

[0068] In practical applications, the standard scenario sample dataset contains 12,000 high-resolution tongue image pictures, covering 6 common abnormal tongue coating morphologies and 5 typical unqualified samples (including scenarios such as tongue body deviation, tooth occlusion, and excessive tongue extension).

[0069] For the division of the standard scenario sample dataset, an automated script tool provided by the official is adopted. Based on the principle of stratified sampling, the original data is divided into a training set and a validation set in a ratio of 8:2. A random seed is set during the division process to ensure the reproducibility of the experiment, and the output directory structure strictly follows the standard format of the COCO dataset. The data quality inspection system includes a triple verification mechanism: First, a script for checking annotation integrity is used to detect the mapping relationship between the label file and the image, eliminating matching errors caused by inconsistent file name cases. Secondly, statistical information analysis is performed to generate a category distribution histogram to reveal the sample balance, draw a heat map of the target size distribution to assist in identifying abnormally annotated samples, and visualize the spatial distribution law of the tongue body area through a position density map. Finally, an interactive visualization tool is used for manual review of single samples, and a batch verification script can automatically detect typical problems such as bounding box overflow and label omission.

[0070] In practical applications, the object detection module constructs a detection network based on the YOLOv5s framework. This module adopts a three-stage detection framework to build a complete feature pyramid: after the input image is normalized at a resolution of 640×640, it successively passes through the Backbone, Neck, and Head to complete multi-scale feature extraction and fusion. The Backbone consists of a Focus module, multiple C3 modules, and an SPPF pooling layer. Among them, the Focus module realizes channel expansion and spatial downsampling through slicing and splicing, converting the input feature dimension from 3×640×640 to 64×320×320. This structure evolves into a 6×6 convolutional layer after version v6.0 to improve the ONNX export compatibility. The C3 module adopts a cross-stage partial connection strategy, reducing the number of parameters by 20%-30% through bottleneck design, while alleviating the problem of gradient disappearance in deep networks. The SPPF module replaces traditional multi-scale parallel pooling with serial maximum pooling, reducing the computational amount by 70% while maintaining the receptive field, significantly improving the efficiency of embedded deployment.

[0071] The Neck network integrates the FPN+PAN bidirectional feature pyramid structure to achieve multi-scale feature fusion through upsampling, downsampling, and lateral connections. The top-down FPN path transmits high-level semantic information, and the bottom-up PAN path optimizes the bottom-layer positioning accuracy, finally forming three-level feature maps of 80×80, 40×40, and 20×20, corresponding to the tongue detection tasks of small, medium, and large scales respectively. The bilinear interpolation is used to align the spatial dimensions during the feature fusion process, and 1×1 convolution is used for channel compression to ensure the smooth splicing of features at different levels. The detection head part introduces a dynamic anchor box mechanism, predefining 9 groups of initial anchor box sizes based on the K-means++ clustering algorithm, and optimizing the width-height ratio matching degree through an adaptive adjustment strategy. The output dimension of each detection head includes the bounding box coordinates, confidence, and classification probability. Among them, the offset range of the center point is extended to -0.5-1.5 to improve the positioning accuracy of large-sized tongues, and the width-height scaling factor adopts an exponential correction strategy to enhance the training stability.

[0072] The module configuration file adopts a modular definition system. Each layer structure declares the input source, repetition times, module type, and configuration parameters through a quadruple parameter. For example, [-1,3,C3,

[256] ] in the Backbone network means constructing a C3 module with 3 bottleneck layers from the output of the previous layer, and the output channel number is 256. The depth scaling factor and width scaling factor respectively control the number of network layers and channels. YOLOv5s adopts a depth factor of 0.33 and a width factor of 0.5, compressing the number of parameters to 7.5MB while ensuring the detection accuracy. This hierarchical and configurable architecture enables the module to flexibly adapt to different hardware platforms and quickly generate a lightweight version that meets the memory constraints of the RV1126 chip by adjusting the scaling factors.

[0073] The detection network built based on the YOLOv5s framework, and the full process analysis and tuning strategy of its object detection network training include YOLOv5 detection network training, hyperparameter configuration, and training optimization;

[0074] 1. The training process of the YOLOv5 detection network includes the following core steps:

[0075] (1) Backbone network configuration and feature extraction

[0076] Build CSPDarknet53 as the backbone feature extractor, with the input size fixed as {Xin ∈ R3×640×640}. The network includes five stages of downsampling operations:

[0077] Stage 0: The Focus structure reorganizes the input slices into 12×320×320, and outputs {C0 ∈ R64×320×320} through the CBL module (Conv + BN + LeakyReLU);

[0078] Stages 1 - 4: Each stage includes a CSP module and a stride - 2 convolutional downsampling, and finally outputs high - level semantic features {F5 ∈ R1024×20×20};

[0079] The Feature Pyramid Network (FPN) fuses multi - scale features, and constructs three - level detection heads {P3 ∈ R256×80×80, P4 ∈ R512×40×40, P5 ∈ R024×20×20} through upsampling and Concat operations;

[0080] (2) Dynamic detection head optimization mechanism

[0081] Introduce an Adaptive Spatial Feature Fusion module (ASFF) before each detection head:

[0082] Perform 1×1 convolution on the P3 - P5 features to align the channels to 256 dimensions, generating {Q3, Q4, Q5};

[0083] Calculate the cross - scale weight matrix W ∈ R3×H×W, with the formula Wijk = σ(Conv([Qi, Qj, Qk]));

[0084] Fuse the weighted features Ffusion = ∑Wi ⊙ Qi, and output the optimized detection features;

[0085] (3) Intelligent training process control system

[0086] Early stopping mechanism: Monitor the fitness of the validation set fitness = 0.9×mAP@0.5 + 0.1×mAP@0.5:0.95, and terminate the training when the improvement amplitude Δ < 0.001 for 100 consecutive epochs;

[0087] (4) Multi - dimensional training monitoring and tuning

[0088] Visual analysis: TensorBoard displays the training loss surface and the quality of detection boxes in real time, where:

[0089] Loss / box reflects the changing trend of the localization error;

[0090] The Precision-Recall curve shows the detection performance at different confidence thresholds;

[0091] Video memory optimization: When the GPU memory occupancy > 90%, enable --half mixed-precision training, and the weight update formula is adjusted to: where s is the gradient scaling factor, and the default value is 65536;

[0092] After the program execution is completed, the extracted features are generated into a.pt format module file for subsequent operations.

[0093] 2. Hyperparameter Configuration and Training Optimization Process

[0094] The hyperparameter configuration file serves as the core control center for module training, and its design directly affects the convergence speed and final performance of the detection network. YOLOv5 adopts a hierarchical parameter management system and defines three major parameter groups in the default configuration file: optimizer parameters, loss function weights, and data augmentation coefficients. The initial learning rate (lr0) is set to 0.01 and combined with the cosine annealing scheduling strategy, which smoothly drops to 0.002 within 300 training epochs, ensuring both fast convergence in the initial stage and avoiding oscillations in the later stage. The momentum factor (momentum) is set to 0.937, which suppresses the high-frequency noise in the parameter update direction through the exponential weighted average mechanism. The weight decay coefficient (weight_decay) is configured to 0.0005 to prevent overfitting caused by complex background features under the L2 regularization constraint. The data augmentation module adopts a progressive adjustment strategy. At the initial stage of training, a 100% mosaic augmentation probability and a 0.5 horizontal flip probability are enabled, significantly improving the detection robustness of small-sized tongue targets. In the middle and later stages, the hue perturbation amplitude is gradually reduced to 0.01 , and the brightness adjustment coefficient is maintained at 0.4 to balance the risk of color distortion and the need for light generalization.

[0095] At the optimizer configuration level, for the small-batch training scenario (batch_size = 8) of the RV1126 embedded platform, the Adam optimizer is used and β1 = 0.9, β2 = 0.999 are set, which can effectively alleviate the gradient direction estimation bias. The learning rate scheduling adopts a three-stage composite mechanism: in the first 3 epochs, linear warm-up is implemented, and the learning rate of the bias term is gradually increased from 0.1 to the benchmark value to eliminate parameter oscillations in the network initialization stage. In the main training stage, the cosine annealing algorithm is applied, and the learning rate curve decays smoothly according to the rule of lr = lr0×(1 + cos(π×epoch / epochs)) / 2; in the final 10 epochs, early stopping detection is enabled, and when the validation set mAP fails to refresh the record for 20 consecutive times, the training is automatically terminated to prevent invalid iterations from consuming computing resources.

[0096] The input image size is dynamically adjusted according to the characteristics of the detection target. For the microscopic features of the tongue coating texture, the standard resolution of 640×640 is retained, and the bicubic interpolation algorithm is used to maintain the edge sharpness. The module saving mechanism adopts the golden copy strategy. A complete checkpoint (including optimizer state and hyperparameter configuration) is saved every 10 epochs, and the weight version with the highest F1 score in the validation set is selected as the final output through cross-validation.

[0097] Construction, deployment and optimization of the detection network module based on the YOLOv5s framework

[0098] 1. ONNX-RKNN conversion process

[0099] In the module conversion stage, a three-stage conversion architecture of PyTorch→ONNX→RKNN is adopted. First, the trained.pth module is exported as the ONNX intermediate representation. During this process, special attention should be paid to the dynamic axis setting, and the input dimension is specified as "None×3×320×320" to support variable batch inference. Secondly, RKNN-Toolkit2 is used for quantization conversion, and the normalization parameter calibration is performed by setting mean = [123.675, 116.28, 103.53] and std = [58.395, 57.12, 57.375], and the uint8 quantization mode is selected to match the characteristics of the NPU operation module.

[0100] In the quantization accuracy tuning link, a calibration dataset containing 500 representative samples is constructed, and the KL divergence minimization strategy is used to determine the quantization parameters of each layer. For the position prediction branch sensitive in the detection task, a mixed-precision quantization strategy is implemented: 8-bit quantization is maintained in the classification head, while the bounding box regression head uses 16-bit floating-point retention. The finally generated.rknn module reaches a real-time inference speed of 15 FPS after testing, meeting the interactive requirements of instant feedback in the clinical detection scenario.

[0101] 2. Embedded inference engine design

[0102] The embedded intelligent terminal is an embedded intelligent terminal built based on the RV1126 heterogeneous computing platform. The hardware platform selects the Rockchip RV1126 chipset. This processor integrates a quad-core ARM Cortex-A7 CPU and an NPU neural network acceleration module, with a fixed-point operation ability of 2.0 TOPS. Compared with the traditional GPU solution, its ultra-low power consumption characteristic of keeping the power consumption within 1.5W makes it very suitable for deployment in portable medical detection devices. At the software level, the RKNN toolchain provided by Rockchip official is adopted, and the quantization migration of the PyTorch training module to the embedded platform is realized through the ONNX intermediate representation, effectively solving the problems of accuracy loss and computing resource limitation in mobile deployment.

[0103] A multi-threaded inference pipeline is built on the RV1126 platform to make full use of the CPU+NPU heterogeneous computing resources. The main thread is responsible for image acquisition and preprocessing. The YUV422 format video stream is obtained through the V4L2 interface, and the libyuv library is called to perform color space conversion and size scaling. The preprocessed RGB image is sent to the dedicated NPU inference thread, and a double-buffer mechanism is adopted to avoid the memory copy overhead. In the post-processing stage of the detection results, a non-maximum suppression algorithm based on NMS is designed, and the confidence threshold is set to 0.6 and the IoU threshold is set to 0.45 to balance the false detection rate and the missed detection rate.

[0104] The acquisition device includes a CMOS sensor and a fill light group, and both the CMOS sensor and the fill light group are connected to the embedded intelligent terminal; the device uses an industrial-grade CMOS sensor to build the acquisition device and is equipped with a ring-shaped LED fill light group. During the acquisition process, the subject is required to naturally extend the tongue, and the mandibular position is fixed through a mechanical bracket to ensure that the tongue body is in the standard observation area.

[0105] Through this technical solution, its system working process is as follows:

[0106] After the user connects the computing platform device embedded with the detection system to a computer or mobile phone through the USB-TypeC interface, and then starts the detection system in the device, the system automatically loads the camera driver and initializes the hardware module, renders the camera screen to the interaction interface in real time, and at the same time starts the led fill light. When the user fully extends the tongue into the detection cavity of the light-shielding mask, the system dynamically tracks the tongue coating area and draws a high-precision detection frame on the screen in real time.

[0107] If the detection box is stable and meets the preset tongue body ratio threshold (covering 35% ± 5% of the screen), the user can trigger the photographing instruction. The photo will be subjected to white balance correction, and then the optimized image will be input into the lightweight YOLOv5-Tiny module for double verification: First, locate the minimum bounding rectangle of the tongue body area (IoU ≥ 0.85), and second, evaluate the image quality (including 12 indicators such as clarity, no occlusion, no reflection, etc.).

[0108] The tongue image graphics that pass the verification will automatically extract the ROI area, append the device fingerprint and millisecond-level timestamp, and transmit them to the pathological analysis module through an encrypted channel; if the tongue coating is detected to be missing or the quality does not meet the standard, the system will immediately play a targeted voice prompt (such as "No effective tongue coating is detected. Please readjust"), and at the same time lock the photographing function and clear the cache data. All qualified images are stored in the local SQLite database in a standardized format (640×640 pixels), separating and saving the original data and the quality assessment report, providing a structured input for subsequent traditional Chinese medicine syndrome differentiation analysis. The entire process realizes millisecond-level response through an event-driven architecture, ensuring the acquisition efficiency and data reliability.

[0109] Based on the above technical solution, the data acquisition method process is as follows:

[0110] Step 1: Start the detection system. The system automatically loads the camera driver of the acquisition device and initializes the hardware module, and renders the camera image to the interaction interface of the detection system in real time, and at the same time starts the fill light component;

[0111] Step 2: When the patient fully extends the tongue into the detection cavity of the acquisition device, the detection system dynamically tracks the tongue coating area of the tongue body and draws a high-precision detection box on the interaction interface in real time;

[0112] Step 3: If the detection box is stable and meets the preset tongue body ratio threshold, trigger the acquisition instruction, perform white balance correction on the acquired image, and then input the optimized image into the target detection module for double verification;

[0113] Step 4: First, locate the minimum bounding rectangle of the tongue body area in the image, and second, evaluate the image quality,

[0114] If the tongue coating of the tongue body is missing or the image quality does not meet the standard after verification, the system will immediately play a targeted voice prompt, and at the same time lock the photographing function and clear the cache data, and re-acquire the qualified tongue image graphics through the acquisition device;

[0115] If it passes the verification, extract the preset ROI area in the qualified tongue image graphics, append the device fingerprint and millisecond-level timestamp, and transmit them to the pathological analysis module through an encrypted channel for subsequent automated medical record analysis;

[0116] Step 5: All qualified images verified by the target detection module are stored in the local database in a standardized format, separating and saving the original data and quality assessment reports to provide structured input for subsequent traditional Chinese medicine syndrome differentiation analysis.

[0117] The detection device of the present invention conducts a clinical effectiveness test in the traditional Chinese medicine department of a Class III Grade A hospital, and a total of 200 outpatient patients are included for double-blind testing. Taking the results jointly judged by three deputy chief physicians as the gold standard, the accuracy rate of this system in the detection of qualified tongue coatings reaches 92.3%, which is significantly higher than 76.8% of the traditional threshold method. In terms of specificity indicators, the recognition accuracy rate for 20 common interference scenarios (such as the edge of the mask entering the frame, finger mis-touch, etc.) reaches 89.5%, proving that the system has good robustness.

[0118] The performance test data shows that under standard lighting conditions, the average response time of the system is 136ms, and the delay increases to 203ms in a strong backlight environment, still meeting the real-time requirements. The power consumption test shows that the power consumption for continuous operation for 8 hours is only 742mAh, and with a 6000mAh lithium battery, all-weather mobile diagnosis and treatment can be achieved. In the long-term stability test, the device runs continuously for 72 hours without memory leakage or crashing, and the fluctuation range of the core indicators is controlled within ±2%.

[0119] As the demand for the popularization of intelligent tongue diagnosis devices in primary medical institutions becomes increasingly urgent, this system is expected to be applied to scenarios such as community traditional Chinese medicine constitution identification stations and mobile diagnosis and treatment vehicles, significantly improving the efficiency and quality of the contracted services of family doctors. The future research and development direction will focus on multi-modal data fusion, constructing a comprehensive syndrome differentiation module by combining tongue image features with information such as pulse wave and face diagnosis. At the same time, explore the module evolution mechanism under the federated learning framework to achieve cross-institutional knowledge sharing while protecting patient privacy, and promote the development of traditional Chinese medicine artificial intelligence diagnosis technology to a higher level.

[0120] The above embodiments only represent several implementation manners of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

[0121] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principle and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm, characterized in that: It includes an acquisition device and a processing system for acquiring tongue image, wherein the processing system includes: A standard scenario sample data set, which includes common abnormal tongue coating morphologies and typical unqualified samples; A target detection module, which is used to determine the integrity and standardization of the tongue image acquired by the acquisition device in real time and output a qualified tongue image; the target detection module also constructs a detection network based on the YOLOv5s framework, and the detection network is used to perform multi-scale feature extraction and fusion on the qualified tongue image and compare it with the standard scenario sample data set, and output a tongue image with qualified quality detection; A pathological analysis module, which is used to perform deep feature extraction on the tongue image with qualified quality detection, compare the features with the standard scenario sample data set, and output a detection result, and the detection result is a pathological analysis result.

2. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 1, characterized in that: The standard scenario sample data set is simulated and trained through a basic enhancement configuration module, a mosaic enhancement module, a hybrid enhancement module and a random perspective transformation module. The basic enhancement configuration module is used to simulate the image color deviation generated by shooting at different distances, and the mosaic enhancement module, the hybrid enhancement module and the random perspective transformation module are used to simulate the tongue body deformation characteristics under different perspectives, so as to enhance the adaptability of the target detection module to non-standard shooting angles.

3. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 2, wherein: The basic enhancement configuration module sets the spatial joint adjustment parameters of hue-saturation-lightness; The mosaic enhancement module randomly selects multiple original images for splicing and recombination to construct a composite training sample containing multi-scale tongue body targets; The hybrid enhancement module uses a linear interpolation algorithm to fuse multiple input images and their annotation information, and sets the enabling probability of a preset value to balance the dialectical relationship between data diversity improvement and label noise introduction; The random perspective transformation module is used to simulate the three-dimensional deformation characteristics of the tongue body under different perspectives, so as to enhance the adaptability of the target detection module to non-standard shooting angles.

4. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 1, characterized in that: The detection network includes a backbone network, a neck network and a detection head; wherein: The backbone network is composed of a Focus module, a multi-stage C3 module and an SPPF pooling layer. The Focus module realizes channel expansion and spatial downsampling by splicing image slices. The multi-stage C3 module adopts a cross-stage partial connection strategy and reduces the parameter quantity by 20%-30% through a bottleneck design. The SPPF pooling layer replaces the traditional multi-scale parallel pooling with serial maximum pooling; The neck network integrates an FPN+PAN bidirectional feature pyramid structure, and realizes multi-scale feature fusion through upsampling, downsampling and horizontal connection. The top-down FPN path transmits high-level semantic information, and the bottom-up PAN path optimizes the bottom-layer positioning accuracy, and finally forms three-level feature maps, corresponding to the tongue body detection tasks of small, medium and large scales respectively; The detection head introduces a dynamic anchor box mechanism, predefines multiple groups of initial anchor box sizes based on the K-means++ clustering algorithm, optimizes the width-height ratio matching degree through an adaptive adjustment strategy, and the output dimension of each detection head includes bounding box coordinates, confidence and classification probability.

5. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 4, wherein: The module configuration file of the target detection module adopts a modular definition system, and each layer structure declares the input source, number of repetitions, module type and configuration parameters through a four-tuple parameter.

6. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 1, wherein: The standard scene sample dataset is divided into training and validation sets using an automated script tool based on the stratified sampling principle. The original data is divided into training and validation sets according to a preset ratio. A random seed is set during the division process to ensure the reproducibility of the experiment, and the output directory structure strictly follows the standard format of the COCO dataset.

7. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 6, characterized in that: The standard scenario sample data set verification process includes a data quality inspection system, and the data quality inspection system includes a three-fold verification mechanism: The mapping relationship between label files and images is detected through the annotation integrity verification script to eliminate matching errors caused by inconsistent file name capitalization. Perform statistical information analysis, generate category distribution histograms to reveal sample balance, draw target size distribution heatmaps to assist in identifying abnormally labeled samples, and visualize the spatial distribution patterns of the tongue area through position density maps; Interactive visualization tools are used to manually review single samples, while batch verification scripts are used to automatically detect typical problems such as bounding box overflow and label omission.

8. The intelligent tongue coating detection system based on the YOLOv5 deep learning algorithm according to claim 1, characterized in that: The intelligent tongue coating detection system further comprises an embedded intelligent terminal that is communicatively connected to the acquisition device, and the processing system is loaded into the embedded terminal; Preferably, the embedded intelligent terminal is an embedded intelligent terminal built on the RV1126 heterogeneous computing platform; Preferably, the acquisition device includes a CMOS sensor and a fill light assembly, and both the CMOS sensor and the fill light assembly are connected to the embedded intelligent terminal.

9. An intelligent tongue coating detection method based on the YOLOv5 deep learning algorithm, characterized in that: The intelligent tongue coating detection system according to any one of claims 1 to 8 is used for detection, and the detection method comprises: (1) collecting tongue image through a collection device; (2) Real-time determination of the integrity and standardization of the collected tongue image, and output of qualified tongue image graphics; (3) Using a detection network built based on the YOLOv5s framework to perform multi-scale feature extraction and fusion on the qualified tongue image graph, and comparing it with a standard scene sample dataset, outputting a tongue image graph that has passed the quality inspection; (4) Performing deep feature extraction on the tongue image that has passed the quality inspection, performing feature comparison with the standard scene sample data set, and outputting the pathological analysis results.

10. The detection method according to claim 9, characterized in that: Methods for collecting qualified tongue images include: (1) Start the detection system. The system automatically loads the camera driver of the acquisition device and initializes the hardware module. It also renders the camera image to the interactive interface of the detection system in real time and starts the fill light component at the same time. (2) When the patient fully extends the tongue into the detection cavity of the collection device, the detection system dynamically tracks the tongue coating area and draws a high-precision detection frame in real time on the interactive interface; (3) If the detection frame is stable and meets the preset tongue ratio threshold, the acquisition instruction is triggered, and the acquired image is white-balanced. The optimized image is then input into the target detection module for double verification; (4) First, locate the minimum bounding rectangle of the tongue area in the image, and then evaluate the image quality. If the tongue coating of the posterior tongue is missing or the image quality fails to meet the standard after verification, the system immediately plays a directional voice prompt, locks the camera function, clears the cache data, and re-collects a qualified tongue image through the acquisition device; If the verification is passed, the ROI area in the image is extracted, and the device fingerprint and millisecond-level timestamp are attached, and then transmitted to the pathological analysis module through an encrypted channel; (5) All qualified images verified by the target detection module are stored in the local database in a standardized format.

Images