Oral cavity sensitive area recognition-based oral irrigator intelligent regulation method and system

Abstract

The application discloses a kind of oral cavity sensitive area identification-based oral irrigator intelligent regulation method and system, the method includes: obtaining user oral cavity image and receiving user graphical marking to sensitive area;Based on image and marking, through the multiscale analysis of fusion user subjective marking and image objective features, generate sub-region oral cavity sensitivity atlas;Mapping relationship between atlas coordinate system and oral irrigator nozzle physical coordinate system is established;Real-time tracking nozzle pose, and it is mapped to atlas to determine the sensitivity level of current area;According to the level, automatically adjust oral irrigator operating parameter.The application also contains feedback-driven online calibration mechanism, can dynamically optimize model and atlas according to user manual operation.The application solves the problem that objective image recognition and user subjective perception are disconnected in prior art, realizes accurate, dynamic, personalized oral cleaning.

Description

Technical Field

[0001] This invention belongs to the field of oral care technology and relates to a method and system for intelligent control of graded cleaning of oral irrigators based on the identification of oral sensitive areas. Background Technology

[0002] Oral health is a crucial component of overall health. Oral diseases such as gingivitis and periodontitis not only affect local tissues but are also linked to various systemic diseases. With increasing health awareness and advancements in consumer electronics, personal oral care devices are rapidly evolving towards intelligence and personalization. Traditional oral cleaning tools, such as manual toothbrushes and ordinary water flossers, rely on user self-discipline and operating skills, resulting in inconsistent cleaning effectiveness and an inability to provide differentiated care based on the actual health status of different areas of the mouth. Therefore, intelligent water flossers that integrate computer vision, sensor technology, and artificial intelligence algorithms to achieve real-time monitoring and adaptive cleaning have become a clear technological trend in this field.

[0003] Under this trend, existing technologies are attempting to address personalized cleaning issues from different perspectives. Among them, using image recognition technology to assess oral health conditions and guide the actions of cleaning equipment has become a key technological branch.

[0004] Existing technology 1: Image recognition-based oral health assessment. Patent CN118105197A, "Techniques for gingivitis detection and periodontal pocket depth assessment," provides a representative solution. This technology utilizes oral scan data and analyzes color data and topological information at specific assessment locations using machine learning algorithms to determine the condition of gingivitis or the depth of periodontal pockets, generating a modified gingival index value. The core of this solution lies in using objective pathological features (such as color and morphology) for automated diagnosis, marking a step towards digitalization and objectivity in oral health assessment. However, the goal of such solutions is professional "disease diagnosis," and their model training relies on standardized, well-labeled clinical medical images. It aims to identify macroscopic pathological features that conform to medical standards but fails to incorporate and respond to the user's subjective, microscopic "discomfort sensations." For example, a user may experience significant pain due to minor ulcers, local abrasions, or friction from orthodontic appliances, but these areas may not show significant inflammatory color or morphological changes on macroscopic images and would therefore be ignored by such diagnostic models.

[0005] Related Existing Technology Two: Vision-Based Automatic Control of Water Flossers. Patent CN117357290A, "Automatic Flossing Control Method, Device, and Water Flosser for Water Flossers," represents another technical approach. This solution uses an image sensor on the water flosser to automatically activate and flush at the appropriate setting by acquiring images in real time and detecting the presence of "areas to be cleaned" within those images. This technology directly combines visual detection with device control, improving automation. However, the "areas to be cleaned" typically refer to areas with objective foreign objects such as food debris stuck between teeth; its detection logic is to find "things that shouldn't be there." It cannot identify or define "subjective discomfort areas" caused by the user's own sensitive, inflamed, or damaged gum tissue. When there are no foreign objects in the mouth but the user still experiences pain, this technology cannot provide differentiated care.

[0006] Based on the above closest existing technologies, it can be found that the current field of intelligent oral care faces a key bottleneck in achieving "precise and personalized cleaning": a gap exists between objective pathology identification models and users' subjective perception experiences, and existing systems lack a mechanism to effectively integrate microscopic, personalized subjective perceptions into macroscopic, general automated decision-making. Specifically:

[0007] First, diagnostic solutions, exemplified by CN118105197A, typically employ deep convolutional neural networks in their image recognition models to achieve universality and pathological accuracy, extracting high-level, global semantic features in the final layer. This architecture excels at perceiving macroscopic pathological patterns (such as large areas of redness and swelling), but its receptive field is large, and the spatial resolution of the feature map decreases continuously with each downsampling layer. When a user attempts to manually mark a tiny pain point (occupying only a few pixels) on an image taken with their mobile phone before using a system like CN117357290A, the location information of this "micro-marker" exhibits severe "scale mismatch bias" when aligned and fused with the macroscopic feature map of the diagnostic model. The user's precise point-like mark is severely diluted or obscured in the model's low-resolution feature map, preventing the model from assigning appropriate attention weights to that specific point. Ultimately, the model learns a mixed feature of a blurred area around the point, rather than the user's true pain point characteristics.

[0008] Secondly, existing solutions lack a closed-loop, learnable, and personalized adaptation process. Whether it's the automated diagnostics in CN118105197A or the foreign object detection in CN117357290A, their operational logic is unidirectional and pre-defined. They cannot receive real-time subjective feedback from users (such as manually lowering the water pressure when actually rinsing a certain area) and use this feedback as a supervisory signal to optimize the initial image recognition stage's judgment of that area. This results in the system's "personalization" being static and coarse, unable to continuously evolve with the user experience, and disconnected from the user's real, dynamically changing sensitive state.

[0009] The aforementioned technical issues can directly impact user experience and cleaning effectiveness in practical applications:

[0010] Over- or under-cleaning: Because the system cannot accurately identify the user's true microscopic sensitive areas, it may apply too high a water pressure to highly sensitive areas, causing pain or even tissue damage to the user; or use too low a water pressure to mildly inflamed areas that need more thorough cleaning, resulting in poor cleaning effect.

[0011] User experience disconnect: Users subjectively feel uncomfortable and mark it, but the device behavior does not accurately match it, leading to a decrease in users' trust in the intelligent system and their perception that it is "not intelligent".

[0012] Personalized functions are practically useless: Although the system has image recognition and gear adjustment capabilities, its claimed "personalized intelligent cleaning" effect is greatly reduced because the core perception-decision link fails to accurately match the user's individual physiological feelings.

[0013] Therefore, there is an urgent need in this field for an intelligent control method that can fundamentally bridge the gap between objective image recognition and subjective perception. This method requires a novel image recognition model architecture and processing flow that can accurately respond to the user's microscopic markings and integrate user interaction feedback into model optimization, thereby generating an oral sensitivity map that truly matches the user's individual real-time experience, and based on this, achieving precise, dynamic, and personalized control of the oral irrigator's cleaning level. Summary of the Invention

[0014] A brief overview of embodiments of the invention is provided below to provide a basic understanding of certain aspects of the invention. It should be understood that this overview is not an exhaustive summary of the invention. It is not intended to identify key or essential parts of the invention, nor is it intended to limit the scope of the invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to the more detailed description that follows.

[0015] According to a first aspect of this application, a method for intelligent control of a water flosser based on the identification of oral sensitive areas is provided, comprising:

[0016] S1. Acquire a digital image of the user's oral gingival region;

[0017] S2. Receive graphical mark input from the user on at least one sensitive area in the digital image, wherein the mark input contains location information of the sensitive area;

[0018] S3. Based on the digital image and the labeled input, generate a regional oral cavity sensitivity map, wherein the map divides the oral cavity region into multiple grids and assigns a level label representing the cleanliness sensitivity to each grid.

[0019] S4. Establish the projection transformation relationship between the two-dimensional image coordinate system of the sub-regional oral sensitivity map and the three-dimensional physical coordinate system of the water flosser nozzle moving in the oral cavity.

[0020] S5. The position and pose data of the nozzle in the three-dimensional physical coordinate system are acquired in real time through the sensor built into the nozzle of the water flosser.

[0021] S6. Based on the projection transformation relationship, the real-time acquired nozzle pose data is mapped to the regional oral cavity sensitivity map to determine the grid in which it is located and the corresponding sensitivity level identifier.

[0022] S7. Based on the sensitivity level indicator, automatically adjust the working parameters of the water flosser in the current position and issue corresponding water flosser control commands to control the water flosser to adjust the output water pressure or water flow mode.

[0023] Furthermore, step S3 specifically includes:

[0024] S31. Parse the user's marker input, calculate the area ratio of each marker region relative to the digital image, and classify the markers into micro markers, meso markers, or macro markers;

[0025] S32. Based on the digital image and the classified label information, feature extraction is performed through a parallel global feature extraction stream, a local high-resolution feature extraction stream, and a label guidance stream. Feature fusion is then performed via a scale-adaptive attention fusion mechanism to output a fused feature map. The global feature extraction stream is implemented using a deep convolutional neural network to extract global contextual features of the image. The local high-resolution feature extraction stream extracts features by cropping a local image centered on micro-labels and performing feature extraction to capture fine texture features. The label guidance stream generates a multi-scale heatmap based on the label's category and coordinates to provide spatial attention priors.

[0026] S33. Divide the fused feature map into grids, predict the sensitivity level of each grid using a classifier, and generate the regional oral sensitivity map.

[0027] As one implementation, further, S31 specifically includes:

[0028] Receive graphical marker input from step S2, which contains a sequence of boundary coordinates for each sensitive region defined by the user on the digital image;

[0029] The pixel area enclosed by each marked region is calculated based on the boundary coordinate sequence. Simultaneously calculate the total pixel area of ​​the digital image. ;

[0030] Calculate the area percentage of each marked region ; Area proportion Less than the first preset threshold The markers are classified as micro-markers, and the first preset threshold It can be set to 1%; the area percentage Greater than or equal to And less than the second preset threshold The markers are classified as mesoscopic markers, and the second preset threshold... It can be set to 5%; the area percentage Greater than or equal to The tags are classified as macro tags;

[0031] Extract the shape features of each marked region, including the roundness calculated based on the boundary coordinates and the aspect ratio of the minimum bounding rectangle; the roundness C is defined as: ;in This represents the perimeter of the region. This sub-step outputs structured data containing the region coordinates, area percentage classification results, and shape feature parameters for each label, which serves as the input for step S32.

[0032] As an implementation scheme, furthermore, in step S32, the global feature extraction stream is implemented using a deep convolutional neural network ResNet50 to extract global contextual features of the image. The ResNet50 deep convolutional neural network includes an initial convolutional layer, four sequentially connected stages (Stage 1 to Stage 4), and a global average pooling layer; Stage 1 to Stage 4 sequentially downsample the input image, and the final output spatial size is... Global feature map with 2048 channels The network was trained using a large-scale oral medicine image dataset, in which each image was labeled with a gum health status tag.

[0033] The training objective of this deep convolutional neural network, ResNet50, is to minimize the cross-entropy loss function. Lce :

[0034] ;

[0035] in For real labels, To predict probabilities for the model, This represents the total number of categories. During training, a stochastic gradient descent optimizer was used with an initial learning rate of 0.01 and weight decay regularization was applied. The trained network weights were then used to extract the global feature map. This feature map serves as the basis for subsequent scale-adaptive attention fusion.

[0036] As one implementation, further, in step S32, the local high-resolution feature extraction stream crops high-resolution local image patches centered on micro-markers, and extracts fine texture features through a shallow convolutional network to obtain high-resolution local features; the specific process includes: receiving the coordinate information of the micro-markers obtained from the classification in sub-step S31. For each micro-marker, its geometric center point... Centered on the original digital image obtained in step S1, a high-resolution local image patch of a fixed size is cropped out. The size of the local image patch is... Pixels. This local image patch serves as the input to the focus stream. The focus stream contains a shallow convolutional network, which consists of two convolutional cascades, each containing one pixel. Convolutional layers, batch normalized layers, and ReLU activation function. The network does not perform downsampling and introduces an inflation rate of 1 / 2 in the second cascade. Dilated convolutions are used to preserve the spatial resolution of the feature maps. The focusing stream processes local image patches from the input, outputting a spatial size of... High-resolution local feature map with 256 channels This local feature map It preserves the fine texture and structural information of the micro-regions.

[0037] As one implementation scheme, furthermore, the marker guidance flow generates a multi-scale Gaussian heatmap based on the coordinates of all markers, as a spatial attention prior, which specifically includes:

[0038] Receive the classification results and coordinate information of all tags output from substep S31. For each tag, apply different standard deviations based on its classification result. The two-dimensional Gaussian kernel function at its center coordinates A heatmap is generated at the location. For micro-markers, a heatmap with the same resolution as the original image is generated. Gaussian kernel standard deviation for Pixels. For mesoscopic markers, generate a quarter-resolution heatmap. Gaussian kernel standard deviation for Pixels. For macroscopic markings, generate a heatmap with a resolution of 1 / 16th. Gaussian kernel standard deviation The value is 20 pixels. All heatmap values ​​are normalized to [0,1]. The label guide stream further includes a lightweight coding network consisting of three parallel... Convolutional layers are constructed, respectively for , and The features are encoded and concatenated along the channel dimension, then fused through a fully connected layer to generate a multi-scale attention prior feature vector. This feature vector It encapsulates user-labeled multi-scale spatial prior information.

[0039] Furthermore, feature fusion is performed via a scale-adaptive attention fusion mechanism. Specifically, based on the label classification results, the scale-adaptive attention fusion mechanism is used to fuse the high-resolution local features or the spatial attention prior with the global context features, outputting a fused feature map. The scale-adaptive attention fusion mechanism is configured as follows: for micro-labels, local features are enhanced using the local feature map and corresponding heatmap and fused with the global features; for meso-labels, the corresponding regions in the global feature map are spatially reweighted using the heatmap; for macro-labels, the global feature map is channel-modulated using the heatmap; finally, the features processed by each branch are fused. Specifically, this includes:

[0040] Receive global feature maps from the global stream. Local feature map from the focused flow and multi-scale attention prior feature vectors from the label-guided flow ;

[0041] For micro-labeling, firstly, the local feature map is... Microscopic thermal maps of corresponding locations Feature enhancement is achieved through element-wise multiplication, resulting in... Use one Convolutional layers will Compressed into a single feature vector ; via space broadcast Add to global feature map The eigenvector at the spatial coordinates corresponding to the center position of the micro-marker;

[0042] For mesoscopic markers, mesoscopic heatmaps Bilinear upsampling to Same space size The spatial attention weight map is obtained. ;Will and Perform element-wise multiplication, for The corresponding regions in the data are reweighted to obtain the following results. ;

[0043] For macroscopic marking, macroscopic heatmap After global average pooling, the data is fed into a two-layer perceptron to generate channel attention weight vectors. The weight vector and The number of channels is the same; Each channel is multiplied by its corresponding weight to achieve channel modulation, resulting in... ;

[0044] The features processed by the micro-label, meso-label, and macro-label branches are summed to output the fused image feature map. .

[0045] As one implementation, further, S33 includes: receiving the fused image feature map output from sub-step S32. The feature map The space dimensions are This corresponds to dividing the original digital image into spatially uniform portions. There are a total of 49 grid sub-regions. (Targeting the feature map) The feature vector at each spatial location (i,j) A classifier is used to map it to a sensitivity level value. The classifier consists of a fully connected layer followed by a Softmax activation function. Sensitivity level value. The sensitivity level is selected from a predefined set of levels, which contains discrete levels representing different cleaning power requirements. Each grid sub-region and its corresponding sensitivity level value are then assigned. The correlation is used to generate a spatial index matrix, which is the regional oral cavity sensitivity map. This map fully describes the predicted sensitivity level information of each segmented region of the oral cavity image.

[0046] As one implementation, further, step S4 includes: receiving a regional oral sensitivity map generated from sub-step S33, wherein the map stores sensitivity level information in the form of image grid coordinates. Before the user uses the water flosser for the first time, an oral coordinate system calibration procedure is performed: guiding the user to sequentially touch multiple preset physical reference points in the oral cavity with the tip of the water flosser nozzle, wherein the reference points include at least the midline point of the maxilla, the buccal point of the left upper first molar, and the lingual point of the right lower first molar. The nozzle pose data at each reference point is recorded by the water flosser's built-in sensor. The pixel coordinates corresponding to the above physical reference points are manually or automatically marked on the digital image. Based on the correspondence between multiple pairs of three-dimensional physical coordinates and two-dimensional image coordinates, a projection transformation matrix is ​​calculated using a direct linear transformation algorithm. The transformation matrix A mapping relationship was established between the real-time three-dimensional position data of the water flosser nozzle and the two-dimensional grid coordinates of the sensitivity map. This mapping relationship is stored in the system and invoked during the position matching step.

[0047] As one implementation, further, in step S5, the sensor includes an inertial measurement unit and a magnetometer. A Kalman filter algorithm is used to fuse multi-source sensor data, and the three-dimensional position coordinates and three-dimensional orientation Euler angles of the nozzle are calculated in real time. Specifically, step S5 includes: during the operation of the water flosser, the position tracking step continuously collects raw sensor data through the inertial measurement unit and magnetometer integrated within the water flosser nozzle. This data includes triaxial acceleration... Triaxial angular velocity and triaxial magnetic field strength The Kalman filter algorithm is used to fuse data from multiple sensor sources, and the three-dimensional position coordinates of the nozzle relative to the calibrated oral cavity coordinate system at the current moment are calculated in real time. The three-dimensional orientation Euler angles are used. The position matching step receives the real-time three-dimensional position coordinates. And call the projection transformation matrix established in step S4. .

[0048] Through formula The three-dimensional position coordinates are projected onto the two-dimensional image coordinate system of the sensitivity map to obtain the corresponding grid index. According to the grid index Search within the segmented oral sensitivity map to retrieve the sensitivity level identifier stored at the index location. This identifier As a direct input for intelligent control steps.

[0049] As one implementation, step S7 further includes a feedback-driven online calibration mechanism, which includes:

[0050] Record user-manually overriding automatic control commands operation events, the events including manually set working parameters, nozzle position at the time of the event, and corresponding map grid index;

[0051] When the frequency of manual operation events at a specific grid index exceeds a preset threshold within a unit of time, the user's actual desired sensitivity level in that area is calculated in reverse based on the manually set working parameters.

[0052] Calculate the deviation between the actual expected level and the original predicted level in the map, and fine-tune the relevant parameters in the feature fusion model based on the deviation;

[0053] The sensitivity levels of the affected areas were recalculated using the fine-tuned model, and the sub-regional oral sensitivity maps were updated.

[0054] According to a second aspect of this application, a smart control system for a water flosser based on the identification of oral sensitive areas is provided, comprising:

[0055] The image acquisition module is used to acquire digital images of the user's oral cavity;

[0056] The user interaction module is used to receive and process graphical mark input from users on digital images;

[0057] The atlas generation module is used to generate regional oral sensitivity atlases based on images and labeling information;

[0058] The coordinate mapping module is used to establish the projection transformation relationship between the image coordinate system and the oral cavity physical coordinate system;

[0059] The pose tracking module is integrated into the water flosser nozzle and is used to track the nozzle's pose in the oral cavity in real time.

[0060] The processing and control module is used to match the sensitivity level in the pose data atlas and generate control instructions for the working parameters of the water flosser accordingly.

[0061] According to a third aspect of this application, a water flosser is provided, which includes the above-described intelligent control system for water flossers based on the identification of oral sensitive areas.

[0062] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the steps of the above-described intelligent control method for a water flosser based on the identification of oral sensitive areas.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] 1. By adopting an image recognition model that combines multi-scale feature extraction and adaptive fusion, the scale mismatch between the user's micro-level subjective markings and the model's macro-level feature perception is effectively solved, significantly improving the accuracy of oral sensitivity map generation. This enables the model to accurately reflect the subtle discomfort experienced by individual users, balancing medical objectivity with individual subjective perception.

[0065] 2. By introducing a closed-loop calibration mechanism based on online user feedback, the system can dynamically optimize model parameters and spectral data according to real-time manual operations by users, realizing a leap from static preset to dynamic evolution of cleaning strategies, thereby continuously improving the accuracy of personalized adaptation and user experience.

[0066] 3. By establishing a precise mapping relationship from two-dimensional image space to three-dimensional oral physical space, and combining it with high-precision real-time pose tracking, a complete technical closed loop of "image recognition - spatial positioning - dynamic control" is opened up, ensuring that personalized cleaning decisions can be accurately and in real time converted into physical actions for specific oral locations, thus enabling precise personalized cleaning to be reliably achieved.

[0067] Overall, this application proposes a novel intelligent control method for oral cleaning that integrates user subjective perception guidance with multimodal data fusion. By quantifying, learning, and responding to user subjective labels as key prior information, and driving the actuator to complete graded cleaning operations, a new paradigm of oral care integrating human-machine collaboration, perception, and execution is realized for the first time in a single system. Ultimately, this achieves a truly effective, comfortable, and safe personalized cleaning effect that is tailored to the individual's immediate condition. Attached Figure Description

[0068] The present invention can be better understood by referring to the description given below in conjunction with the accompanying drawings, in which the same or similar reference numerals are used throughout the drawings to denote the same or similar parts. These drawings, together with the following detailed description, are incorporated in and form part of this specification, and are used to further illustrate preferred embodiments of the invention and explain the principles and advantages of the invention. In the drawings:

[0069] Figure 1 This is a flowchart illustrating the overall process of intelligent control for oral sensitive area identification and graded cleaning in an embodiment of the present invention.

[0070] Figure 2 This is a detailed flowchart of the process for generating multi-scale oral sensitivity maps according to an embodiment of the present invention;

[0071] Figure 3 This is a block diagram illustrating the operation of the water flosser position tracking and map matching in an embodiment of the present invention.

[0072] Figure 4 This is a block diagram of online calibration and optimization of the feedback-driven model in an embodiment of the present invention. Detailed Implementation

[0073] Embodiments of the present invention will now be described with reference to the accompanying drawings. Elements and features described in one drawing or embodiment of the invention may be combined with elements and features shown in one or more other drawings or embodiments. It should be noted that, for clarity, representations and descriptions of components and processes unrelated to the present invention and known to those skilled in the art have been omitted from the drawings and description.

[0074] This invention provides an intelligent control method for oral irrigators based on the identification of oral sensitive areas. See [link to relevant documentation]. Figure 1 It includes the following steps:

[0075] S1. Image acquisition steps: Acquire a digital image of the user's oral gingival region;

[0076] S2, User Marking Step: Receive graphical marking input from a user on the digital image for at least one sensitive area, the marking input containing location information of the sensitive area;

[0077] S3. Sensitivity Map Generation Step: Based on the digital image and the location information of the user-marked areas, generate a regional oral sensitivity map containing different sensitivity level identifiers; see [link to relevant documentation]. Figure 2 This step specifically includes:

[0078] S31, Marker Scale Analysis and Classification Sub-step: Receive the location information of the user's marks, calculate the area ratio and shape features of each marked region relative to the digital image, and draw and classify them into micro-markers, meso-markers or macro-markers based on this.

[0079] S32, Multi-scale Feature Extraction and Fusion Sub-step: Receive the digital image and the classified label information, and process them through a multi-stream feature extraction network; the network includes a global stream for extracting macroscopic contextual features, a focusing stream for extracting high-resolution local features, and a label-guided stream for encoding multi-scale label priors; based on the label classification results, a scale-adaptive attention fusion mechanism is used to fuse the microscopic features of the focusing stream or the multi-scale attention priors of the label-guided stream with the macroscopic features of the global stream, and output the fused image features;

[0080] S33, Atlas Division and Assignment Sub-step: Receive the fused image features, divide the oral cavity region corresponding to the digital image into several sub-regions, and assign a sensitivity level identifier to each sub-region based on the image features corresponding to each sub-region, thereby generating the regional oral cavity sensitivity atlas;

[0081] S4. Mapping Step: Establish the mapping relationship between the regional oral sensitivity map and the user's actual oral physical region;

[0082] S5. Position tracking step: During the operation of the water flosser, the actual position data of the water flosser nozzle in the user's mouth is obtained in real time;

[0083] S6. Position matching step: Match the actual position data of the water flosser nozzle with the regional oral sensitivity map to determine the sub-region and its sensitivity level corresponding to the current nozzle position;

[0084] S7. Intelligent control steps: Based on the matched sensitivity level, automatically generate and issue corresponding oral irrigator working parameter control commands to control the oral irrigator to adjust the output water pressure or water flow mode.

[0085] Step S1 begins with the user using a mobile terminal equipped with a high-definition camera to capture images of the gingival region inside the oral cavity. The user needs to open their lips and adjust the phone angle under sufficient and uniform lighting conditions to ensure that the camera can capture the target tissues, including the gingival margins, gingival papillae, and attached gingiva, clearly and completely. The dedicated application on the mobile terminal will call the camera module and guide the user to complete the shooting. The raw image data obtained is immediately transmitted to the image preprocessing module built into the application. This module performs a series of standardized operations, including automatic cropping to focus on the oral cavity area, color correction to eliminate the influence of ambient light color cast, uniform scaling of resolution to a preset pixel size, and contrast enhancement to highlight the boundary between the gingival tissue and the teeth. The preprocessed standardized digital image is then stored in the temporary memory of the mobile terminal and serves as the sole visual data source for all subsequent image analysis processes of the entire system. The principle of this step is to eliminate the interference caused by hardware differences and shooting environment variables through standardized image acquisition and preprocessing processes, providing high-quality and consistent input data for subsequent machine learning-based image recognition models. This is the physical basis for ensuring the accuracy and stability of model recognition.

[0086] Step S2 is implemented based on the graphical user interface provided by the mobile terminal application. This interface loads and displays the standardized digital image generated in step one. The user interacts directly with the image via the touchscreen, using preset marking tools to precisely indicate subjectively perceived sensitive or uncomfortable areas on the displayed gingival image. The marking tool library includes dot marking tools for marking single sensitive locations, freehand drawing tools for outlining irregularly shaped sensitive areas, and filling tools for covering larger sensitive areas. Each touch interaction is recorded as a series of continuous screen coordinate points. The application's background logic converts these coordinate points into closed polygon vector graphic data relative to the pixel coordinate system of the standardized digital image, and simultaneously records... The tagging operation timestamps and generates a unique tag identifier. All the vector data, timestamps, and identifiers of the tags are integrated into a structured tag data set. This set is bound to the corresponding standardized digital image and prepared for transmission to the subsequent processing module. The principle of this step is to transform the user's subjective, qualitative oral discomfort into objective, quantifiable computer graphics data. This precise location information provides crucial supervision signals and spatial attention guidance for the subsequent image recognition model. In particular, it lays the data foundation for solving the scale mismatch problem between the user's micro-tags and the model's macro-feature perception, enabling the model's learning to closely match the user's actual physiological experience rather than relying solely on general pathological features.

[0087] The specific process of step S31, label scale analysis and classification sub-step, is as follows:

[0088] Receive graphical marker input from the user marking step S2, which contains a sequence of boundary coordinates for each sensitive region defined by the user on the digital image;

[0089] The pixel area enclosed by each marked region is calculated based on the boundary coordinate sequence. Simultaneously calculate the total pixel area of ​​the digital image. ;

[0090] Calculate the area percentage of each marked region ; Area proportion Less than the first preset threshold The markers are classified as micro-markers, and the first preset threshold It is 1%. The area percentage is... Greater than or equal to And less than the second preset threshold The markers are classified as mesoscopic markers, and the second preset threshold... It is 5%. The area percentage is... Greater than or equal to The markers are classified as macro-level markers. Simultaneously, the shape features of each marker region and the aspect ratio of its minimum bounding rectangle are extracted. Shape features include roundness calculated based on boundary coordinates. ;in The perimeter of the region is given. This sub-step outputs structured data containing the region coordinates, area percentage classification results, and shape feature parameters for each label, which serves as input for the multi-scale feature extraction and fusion sub-step.

[0091] The global flow in step S32, multi-scale feature extraction and fusion, is implemented using a deep convolutional neural network. The input to this network is the original digital image obtained in step S1, image acquisition, with the image size normalized to [value missing]. Pixels. The deep convolutional neural network uses the ResNet50 architecture, which includes an initial convolutional layer, four sequentially connected stages (Stage 1 to Stage 4), and a global average pooling layer. Stages 1 to 4 downsample the input image sequentially, resulting in a final output spatial size of [missing information]. Global feature map with 2048 channels The network was trained using a large-scale oral medicine image dataset, in which each image was labeled with a gum health status tag.

[0092] The training objective of the ResNet50 deep convolutional neural network architecture is to minimize the cross-entropy loss function:

[0093] ;

[0094] in For real labels, To predict probabilities for the model, This represents the total number of categories. During training, a stochastic gradient descent optimizer was used with an initial learning rate of 0.01 and weight decay regularization was applied. The trained network weights were then used to extract the global feature map. This feature map serves as the basis for subsequent scale-adaptive attention fusion.

[0095] The focusing flow processing in step S32, multi-scale feature extraction and fusion sub-step, is as follows:

[0096] Receive the coordinate information of the micro-markers obtained from the classification in sub-step S31. For each micro-marker, determine its geometric center point. Centered on the original digital image obtained in step S1, a high-resolution local image patch of a fixed size is cropped out. The size of the local image patch is... Pixels. This local image patch serves as the input to the focus stream;

[0097] Focused Stream contains a shallow convolutional network, which consists of two convolutional cascades, each containing a... Convolutional layers, batch normalized layers, and ReLU activation function. The network does not perform downsampling and introduces an inflation rate of 1 / 2 in the second cascade. Dilated convolutions are used to preserve the spatial resolution of the feature maps. The focusing stream processes local image patches from the input, outputting a spatial size of... High-resolution local feature map with 256 channels This local feature map It preserves the fine texture and structural information of the micro-regions.

[0098] The label-guided flow processing in step S32, multi-scale feature extraction and fusion sub-step, is as follows:

[0099] Receive the classification results and coordinate information of all tags output from sub-step S31;

[0100] For each label, different standard deviations are applied based on its classification result. The two-dimensional Gaussian kernel function at its center coordinates Heatmap generated at:

[0101] For micro-labels, a heatmap with the same resolution as the original image is generated. Gaussian kernel standard deviation for Pixel;

[0102] For mesoscopic markers, generate a quarter-resolution heatmap. Gaussian kernel standard deviation for Pixel;

[0103] For macroscopic markings, generate a heatmap with a 1 / 16th resolution. Gaussian kernel standard deviation 20 pixels;

[0104] The numerical range of all heatmaps is normalized to [0,1];

[0105] The marker-guided flow further includes a lightweight coding network consisting of three parallel... Convolutional layers are constructed, respectively for , and The features are encoded and concatenated along the channel dimension, then fused through a fully connected layer to generate a multi-scale attention prior feature vector. This feature vector It encapsulates user-labeled multi-scale spatial prior information.

[0106] The specific operation flow of the scale-adaptive attention fusion mechanism in step S32, multi-scale feature extraction and fusion sub-step, is as follows:

[0107] Receive global feature maps from the global stream. Local feature map from the focused flow and multi-scale attention prior feature vectors from the label-guided flow ;

[0108] The scale-adaptive attention fusion mechanism performs branching operations based on the classification results of the S31 sub-step. For micro-labels, the local feature map is first... Microscopic thermal maps of corresponding locations Feature enhancement is achieved through element-wise multiplication, resulting in... Then, using a Convolutional layers will Compressed into a single feature vector Finally, it will be broadcast via space. Add to global feature map The eigenvector at the spatial coordinates corresponding to the center position of the microscopic marker. For mesoscopic markers, the mesoscopic heatmap... Bilinear upsampling to Same space size The spatial attention weight map is obtained. .Will and Perform element-wise multiplication, for The corresponding regions in the data are reweighted to obtain the following results. For macroscopic markings, the macroscopic heatmap will be used. After global average pooling, the data is fed into a two-layer perceptron to generate channel attention weight vectors. The weight vector and The number of channels is the same. Each channel is multiplied by its corresponding weight to achieve channel modulation, resulting in... Finally, the features processed by all branches are summed to output the fused image feature map. .

[0109] The specific process of step S33, the map partitioning and assignment sub-step, is as follows:

[0110] Receive the fused image feature map output from sub-step S32 The feature map The space dimensions are This corresponds to dividing the original digital image into spatially uniform portions. There are a total of 49 grid sub-regions;

[0111] For feature maps The feature vector at each spatial location (i,j) A classifier is used to map it to a sensitivity level value. The classifier consists of a fully connected layer followed by a Softmax activation function. Sensitivity level value. The sensitivity level is selected from a predefined set of levels, which contains discrete levels representing different cleaning power requirements. Each grid sub-region and its corresponding sensitivity level value are then assigned. The correlation is used to generate a spatial index matrix, which is the regional oral cavity sensitivity map. This map fully describes the predicted sensitivity level information of each segmented region of the oral cavity image.

[0112] The specific process of step S4, the map mapping step, is as follows: The user receives a regional oral sensitivity map generated from sub-step S33. This map stores sensitivity level information in the form of image grid coordinates. Before the user uses the water flosser for the first time, an oral coordinate system calibration procedure is performed: the user is guided to sequentially touch multiple preset physical reference points in the oral cavity with the tip of the water flosser nozzle. These reference points include at least the midline point of the maxilla, the buccal point of the left upper first molar, and the lingual point of the right lower first molar. The nozzle pose data at each reference point is recorded using the water flosser's built-in sensor. The pixel coordinates corresponding to the aforementioned physical reference points are manually or automatically marked on the digital image. Based on the correspondence between multiple pairs of three-dimensional physical coordinates and two-dimensional image coordinates, a projection transformation matrix is ​​calculated using a direct linear transformation algorithm. The transformation matrix A mapping relationship was established between the real-time three-dimensional position data of the water flosser nozzle and the two-dimensional grid coordinates of the sensitivity map. This mapping relationship is stored in the system and invoked during the position matching step.

[0113] See Figure 3 The specific process of the S5 position tracking step and the S6 position matching step is as follows: During the operation of the water flosser, the position tracking step continuously collects raw sensor data through the inertial measurement unit and magnetometer integrated in the water flosser nozzle. The data includes triaxial acceleration. Triaxial angular velocity and triaxial magnetic field strength The Kalman filter algorithm is used to fuse data from multiple sensor sources, and the three-dimensional position coordinates of the nozzle relative to the calibrated oral cavity coordinate system at the current moment are calculated in real time. The three-dimensional orientation Euler angles are used. The position matching step receives the real-time three-dimensional position coordinates. And call the projection transformation matrix established in step S4. .

[0114] Through formula The three-dimensional position coordinates are projected onto the two-dimensional image coordinate system of the sensitivity map to obtain the corresponding grid index. According to the grid index Search within the segmented oral sensitivity map to retrieve the sensitivity level identifier stored at the index location. This identifier As a direct input for intelligent control steps.

[0115] See Figure 4 The S7 Smart Control Steps also include a feedback-driven online calibration mechanism. This mechanism continuously records user actions during operation, such as manually overriding automatic control commands via the physical buttons on the water flosser or the accompanying app. Each event includes the manually set operating parameters. Real-time position of the nozzle at the time of the incident and the corresponding mapping grid index A mapping table between the system's preset sensitivity levels and standard operating parameters. When in a specific grid index The system detected that the frequency of manual operation events within a unit of time exceeded a threshold. At that time, it was determined that there was a discrepancy between the automatic control of the area and the user's subjective experience.

[0116] At this point, the system initiates the calibration process: First, based on the manually set parameters... Reverse lookup of the mapping table This allows us to calculate the user's actual desired sensitivity level in that area. Next, the grade deviation will be... ,in This represents the original predicted rank of the map, associated with the user-labeled data upon which the initial predicted rank for this region was based. The system is based on bias. The gradient descent algorithm is used to fine-tune the fusion weight parameters related to the label category within the attention fusion mechanism in sub-step S32. The model's feature representation and sensitivity level are recalculated using fine-tuned parameters, and the corresponding grid level values ​​in the regional oral sensitivity map are updated, thus achieving personalized online optimization of the model.

[0117] To address the contradiction between the limitations of traditional oral hygiene devices in simultaneously considering the user's subjective microscopic sensations and objective medical pathological characteristics, this invention provides an innovative intelligent control scheme. This scheme constructs an image recognition model architecture that combines multi-scale feature extraction and adaptive fusion. It deploys a global feature stream, a local high-resolution feature stream, and a label-guided stream in parallel within a neural network. Based on the scale classification results actively labeled by the user, a differentiated attention fusion strategy is employed to seamlessly embed the user's microscopic subjective labels into a macroscopic medical feature atlas, thereby generating an oral sensitivity atlas that is both objective and individualized.

[0118] To achieve a precise closed loop from visual perception to physical execution, this invention proposes a full-link mapping and control method: by calibrating physical reference points, a projection transformation relationship between a two-dimensional sensitivity map and a three-dimensional oral cavity space is established; by using high-frequency inertial sensor data fusion, sub-centimeter-level real-time positioning of the water flosser nozzle is achieved; and the water pressure and spray mode are adjusted in real time according to the sensitivity level corresponding to the grid.

[0119] Furthermore, in response to the dynamic changes in the user's oral cavity status, this invention introduces a feedback-driven online calibration technique: real-time capture and analysis of the user's manual coverage operation of the device settings, correlation with the device pose and atlas prediction results to form a supervisory signal, driving the reverse fine-tuning of feature fusion weights and dynamic updates of the atlas.

[0120] The technical effects of this invention are significant: 1) It fundamentally solves the scale mismatch problem between the user's subjective feelings and the model's macroscopic feature perception, ensuring individualized and accurate response of cleaning intensity; 2) It realizes the transformation from static preset personalization to dynamic evolutionary personalization, continuously adapting to the user's real-time state; 3) It opens up a complete technical closed loop of "image recognition-spatial positioning-dynamic control", reliably transforming intelligent decisions into specific physical cleaning actions, creating a new paradigm of oral care that integrates human-machine collaboration, perception and execution.

[0121] The method of this invention is not limited to being executed in the chronological order described in the specification, but can also be executed in other chronological orders, in parallel, or independently. Therefore, the execution order of the method described in this specification does not constitute a limitation on the technical scope of this invention.

[0122] Although the invention has been disclosed above through the description of specific embodiments, it should be understood that all the embodiments and examples described above are exemplary and not restrictive. Those skilled in the art can design various modifications, improvements, or equivalents to the invention within the spirit and scope of the appended claims. These modifications, improvements, or equivalents should also be considered to be included within the protection scope of the invention.

Claims

1. A method for intelligent control of a water flosser based on the identification of oral sensitive areas, characterized in that, Includes the following steps: S1. Acquire a digital image of the user's oral gingival region; S2. Receive graphical mark input from the user on at least one sensitive area in the digital image, wherein the graphical mark input contains location information of the sensitive area; S3. Based on the digital image and the graphical marker input, a regional oral cavity sensitivity map is generated by multi-scale analysis that integrates user subjective marker information and objective image features; the regional oral cavity sensitivity map divides the oral cavity region into multiple grids and assigns a level label representing the cleanliness sensitivity to each grid. S4. Establish the projection transformation relationship between the two-dimensional image coordinate system of the sub-regional oral sensitivity map and the three-dimensional physical coordinate system of the water flosser nozzle moving in the oral cavity. S5. The position and pose data of the oral irrigator nozzle in the three-dimensional physical coordinate system are obtained in real time through the sensor built into the nozzle. S6. Based on the projection transformation relationship, the real-time acquired nozzle pose data is mapped to the regional oral cavity sensitivity map to determine the grid in which it is located and the corresponding sensitivity level identifier. S7. Based on the sensitivity level indicator, automatically adjust the working parameters of the water flosser in the current position.

2. The intelligent control method for a water flosser according to claim 1, characterized in that, Step S3 specifically includes: S31. Parse the user's marker input, calculate the area ratio of each marker region relative to the digital image, and classify the markers into micro markers, meso markers, or macro markers; S32. Based on the digital image and the classified label information, feature extraction is performed through parallel global feature extraction stream, local high-resolution feature extraction stream and label guidance stream, and feature fusion is performed through scale-adaptive attention fusion mechanism to output the fused feature map. S33. Divide the fused feature map into grids, and predict the sensitivity level of each grid using a classifier to determine the regional oral sensitivity map.

3. The intelligent control method for a water flosser according to claim 2, characterized in that, The global feature extraction stream in step S32 is implemented using a deep convolutional neural network to extract global contextual features of the image; the local high-resolution feature extraction stream extracts features by cropping a local image centered on micro-markers and performing feature extraction to capture fine texture features; the marker guidance stream generates a multi-scale heatmap based on the category and coordinates of the markers to provide spatial attention priors.

4. The intelligent control method for a water flosser according to claim 3, characterized in that, In step S32, the scale-adaptive attention fusion mechanism is configured to: for micro-labels, enhance local features using local feature maps and corresponding heatmaps and fuse them with global features; For mesoscopic labels, spatial reweighting of corresponding regions in the global feature map is performed using heatmaps; for macroscopic labels, channel modulation of the global feature map is performed using heatmaps; finally, the features processed by each branch are fused.

5. The intelligent control method for a water flosser according to claim 1, characterized in that, Step S4 includes: Receive a regional oral cavity sensitivity map, wherein the regional oral cavity sensitivity map stores sensitivity level information in the form of image grid coordinates; Before a user uses the water flosser for the first time, an oral coordinate system calibration procedure is performed: the user is guided to use the tip of the water flosser nozzle to touch multiple preset physical reference points in the oral cavity in sequence. The physical reference points include at least the midline point of the maxilla, the buccal point of the upper left first molar, and the lingual point of the lower right first molar. The nozzle pose data when each reference point is touched is recorded by the built-in sensor of the water flosser. Manually or automatically mark the pixel coordinates corresponding to the aforementioned physical reference points on the digital image; Based on the correspondence between multiple pairs of three-dimensional physical coordinates and two-dimensional image coordinates, a projection transformation matrix is ​​calculated using a direct linear transformation algorithm. This transformation matrix establishes a mapping relationship from the real-time three-dimensional position data of the water flosser nozzle to the two-dimensional grid coordinates of the sub-regional oral sensitivity map.

6. The intelligent control method for a water flosser according to claim 1, characterized in that, In step S5, the sensor includes an inertial measurement unit and a magnetometer. A Kalman filter algorithm is used to fuse data from multiple sensor sources and calculate the three-dimensional position coordinates and three-dimensional orientation Euler angles of the nozzle in real time.

7. The intelligent control method for a water flosser according to claim 1, characterized in that, Step S7 further includes a feedback-driven online calibration mechanism, which includes: Record user-manually overriding automatic control commands operation events, the operation events including manually set working parameters, nozzle position at the time of the event, and corresponding map grid index; When the frequency of manual operation events at a specific grid index exceeds a preset threshold within a unit of time, the user's actual desired sensitivity level in that area is calculated in reverse based on the manually set working parameters. Calculate the deviation between the sensitivity level and the original predicted level in the map, and fine-tune the relevant parameters in the feature fusion model based on the deviation; The sensitivity levels of the affected areas were recalculated using the fine-tuned model, and the sub-regional oral sensitivity maps were updated.

8. A smart control system for a water flosser based on the identification of oral sensitive areas, used to implement the method according to any one of claims 1-7, characterized in that, include: The image acquisition module is used to acquire digital images of the user's oral cavity; The user interaction module is used to receive and process graphical mark input from users on digital images; The atlas generation module is used to generate regional oral sensitivity atlases based on images and labeling information; The coordinate mapping module is used to establish the projection transformation relationship between the image coordinate system and the oral cavity physical coordinate system; The pose tracking module is integrated into the water flosser nozzle and is used to track the nozzle's pose in the oral cavity in real time. The processing and control module is used to match the sensitivity level in the pose data atlas and generate control instructions for the working parameters of the water flosser accordingly.

9. A dental flosser, characterized in that, It includes the intelligent control system for the oral irrigator as described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the intelligent control method for a water flosser as described in any one of claims 1-7.

Images