Intelligent regulation and control method and system for oral irrigator based on oral sensitive area recognition

Abstract

The invention discloses an oral cavity sensitive area identification-based intelligent regulation and control method and system for a water pick. The method comprises the following steps of: acquiring an oral cavity image of a user and receiving a graphical mark of the user on a sensitive area; based on the image and the mark, generating a regional oral cavity sensitivity map through multi-scale analysis of fusing the subjective mark of the user and the objective feature of the image; establishing a mapping relation between the map coordinate system and a physical coordinate system of the water pick nozzle; tracking the pose of the nozzle in real time, and mapping the pose to a map to determine the sensitivity level of the current area; and automatically adjusting the working parameters of the oral irrigator according to the grade. The method further comprises a feedback-driven online calibration mechanism, and the model and the atlas can be dynamically optimized according to manual operation of a user. The problem that objective image recognition and subjective perception of a user are disjointed in the prior art is solved, and accurate, dynamic and personalized oral cavity cleaning is achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of oral care, and relates to a waterpik grading cleaning intelligent control method and system based on identification of sensitive areas in the oral cavity. BACKGROUND

[0002] Oral health is an important part of overall health, and oral diseases such as gingivitis and periodontitis not only affect local tissues, but also are associated with various systemic diseases. With the improvement of people's health awareness and the progress of consumer electronics technology, personal oral care devices are rapidly developing towards intelligence and individualization. Traditional oral cleaning tools such as manual toothbrushes and ordinary waterpiks rely on the user's self-consciousness and operation skills, and the cleaning effect is uneven, and they cannot provide differential care according to the actual health status of different areas in the oral cavity. Therefore, intelligent waterpiks that realize real-time monitoring and adaptive cleaning by integrating computer vision, sensor technology and artificial intelligence algorithms have become a clear technology evolution trend in this field.

[0003] Under this trend, existing technologies attempt to solve the problem of individualized cleaning from different angles. Among them, using image recognition technology to assess oral health status and guide the action of cleaning equipment has become a key technology branch.

[0004] Related Prior Art One: Oral health assessment based on image recognition. The patent "Techniques for gingivitis detection and periodontal pocket depth assessment" with publication number CN118105197A provides a representative solution. This technology uses oral scanning data to analyze color data and topological information at specific evaluation positions through machine learning algorithms to determine gingivitis conditions or periodontal pocket depths and generate modified gingival index values. The core of this solution is to use objective pathological features (such as color and shape) for automated diagnosis, marking the progress of oral health assessment towards digitization and objectivity. However, the goal of such a solution is to make professional "disease diagnosis", and its model training relies on standardized and clearly labeled clinical medical images. It aims to identify macroscopic pathological manifestations that meet medical standards, but fails to incorporate and respond to users' subjective and microscopic "discomfort feelings". For example, users may experience significant pain due to minor ulcers, local abrasions or orthodontic appliance friction, but these areas may not show significant inflammatory color or shape changes in macroscopic images, so they will be ignored by such diagnostic models.

[0005] Related Prior Art Two: Vision-based Oral Irrigator Automatic Control. Patent CN117357290A, "Oral Irrigator Automatic Irrigation Control Method, Device and Oral Irrigator", represents another technical route. This scheme sets up an image sensor on the oral irrigator, automatically starts the oral irrigator and irrigates at the corresponding gear by real-time image acquisition and detection of whether there is a "cleaning area" in the image. This technology directly combines visual detection and device control, improving the degree of automation. However, the "cleaning area" it refers to usually refers to the area where objective foreign matter such as food residue is embedded in the tooth gap, and its detection logic is to find "things that should not exist". It cannot identify and define "subjective discomfort areas" formed due to user's own sensitive, inflamed or damaged gum tissue. When there is no foreign matter in the oral cavity but the user still feels pain, this technology cannot provide differentiated care.

[0006] In summary, the closest prior art can be found that the current intelligent oral care field faces a key bottleneck in achieving "precise personalized cleaning": there is a gap between objective pathological recognition models and user subjective perception experience, and existing systems lack a mechanism to effectively integrate micro, personalized subjective perception into macro, universal automated decision-making. Specifically: First, the diagnostic scheme represented by CN118105197A, its image recognition model pursues universality and pathological accuracy, usually using deep convolutional neural networks, and extracting high-level, global semantic features in the final layer. This architecture has an advantage in perceiving macro-pathological patterns (such as large redness), but its receptive field is large, and the spatial resolution of the feature map is constantly reduced during layer-by-layer downsampling. When the user tries to manually mark a tiny point of pain that only occupies a few pixels on the image taken by the phone before using the system described in CN117357290A, the location information of this "micro-mark" will produce a serious "scale mismatch bias" when it is aligned and fused with the macro-feature map of the above diagnostic model. The user's precise point marking is severely diluted or lost in the low-resolution feature map of the model, resulting in the model being unable to allocate appropriate attention weights for the specific point, and ultimately learning the mixed features of the fuzzy area around the point, rather than the user's real pain point features.

[0007] Secondly, the existing solutions lack a closed-loop, learnable personalized adaptation process. Whether it is the automatic diagnosis of CN118105197A or the foreign matter detection of CN117357290A, the running logic is one-way and preset. They cannot receive real-time subjective feedback from users (such as manually adjusting the water pressure low when actually flushing a certain area), and use this feedback as a supervisory signal to optimize the initial image recognition stage's judgment for that area. This results in the "personalization" provided by the system being static and rough, and unable to evolve continuously with user experience, disconnecting from the user's real, dynamically changing sensitivity state.

[0008] The above technical problems will have a direct impact on user experience and cleaning effect in practical application: Over or under cleaning: the system cannot accurately identify the user's real micro-sensitive areas, and may apply too high water pressure to highly sensitive areas, causing user pain and even tissue damage; or use too low water pressure on mildly inflamed areas that need to be cleaned, resulting in poor cleaning effect.

[0009] Split user experience: the user feels uncomfortable and marks it, but the device behavior does not accurately match it, leading to a decrease in user trust in the intelligent system, considering it "unintelligent".

[0010] Personalization function is virtual: although the system has image recognition and gear adjustment capabilities, due to the failure of the core perception-decision link to accurately connect to the user's individual physiological feelings, the claimed "personalized intelligent cleaning" effect is greatly discounted.

[0011] Therefore, there is an urgent need in the art for an intelligent control method that can fundamentally bridge the gap between objective image recognition and subjective perception. This method requires a novel image recognition model architecture and processing flow that can accurately respond to user micro-markings and incorporate user interaction feedback into model optimization, thereby generating a truly user-individual real-time sensitive oral map and achieving precise, dynamic, and personalized control of oral irrigator cleaning levels. SUMMARY

[0012] In the following, a brief summary of embodiments of the present application is given in order to provide a basic understanding of some aspects of the present application. It should be understood that the following summary is not an exhaustive overview of the present application. It is not intended to identify key or important parts of the present application nor to delineate the scope of the present application. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.

[0013] According to a first aspect of the present application, a method for intelligent control of an oral irrigator based on identification of sensitive oral areas is provided, comprising: S1, obtaining a digital image of the user's oral gingival area; S2, receiving graphical marking input of at least one sensitive area on the digital image by the user, the marking input containing position information of the sensitive area; S3, generating a sub-regional oral sensitivity map based on the digital image and the marking input, the map dividing the oral region into a plurality of grids and assigning each grid with a level identifier representing the cleaning sensitivity; S4, establishing a projection transformation relationship between a two-dimensional image coordinate system of the sub-regional oral sensitivity map and a three-dimensional physical coordinate system in which the oral irrigator nozzle moves in the oral cavity; S5, acquiring real-time pose data of the nozzle in the three-dimensional physical coordinate system through a sensor built in the oral irrigator nozzle; S6, mapping the real-time acquired nozzle pose data to the sub-regional oral sensitivity map according to the projection transformation relationship to determine the grid and the corresponding sensitivity level identifier thereof; S7, automatically adjusting the working parameters of the oral irrigator at the current pose according to the sensitivity level identifier, and issuing corresponding oral irrigator control instructions to control the oral irrigator to adjust the water pressure or water flow mode.

[0014] Further, the step S3 specifically comprises: S31, analyzing the marking input of the user, calculating the area proportion of each marking area relative to the digital image, and classifying the marking into microscopic marking, mesoscopic marking or macroscopic marking; S32, based on the digital image and the classified marking information, performing feature extraction through parallel global feature extraction flow, local high-resolution feature extraction flow and marking guidance flow, and performing feature fusion through a scale adaptive attention fusion mechanism to output a fused feature map; wherein the global feature extraction flow is implemented by a deep convolutional neural network and is used to extract the global context features of the image; the local high-resolution feature extraction flow is used to capture fine texture features by cropping local images centered on microscopic markings and performing feature extraction; the marking guidance flow is used to provide spatial attention priors by generating multi-scale heat maps according to the categories and coordinates of the markings; S33, dividing the fused feature map into grids, predicting the sensitivity level of each grid through a classifier, and generating the sub-regional oral sensitivity map.

[0015] As an implementation scheme, further, the S31 specifically comprises: receiving the graphical marking input from step S2, which contains the boundary coordinate sequence of each sensitive area defined by the user on the digital image; calculating the pixel area surrounded by each marking area according to the boundary coordinate sequence Simultaneously calculate the total pixel area of ​​the digital image. ; Calculate the area percentage of each marked region ; Area proportion Less than the first preset threshold The markers are classified as micro-markers, and the first preset threshold It can be set to 1%; the area percentage Greater than or equal to And less than the second preset threshold The markers are classified as mesoscopic markers, and the second preset threshold... It can be set to 5%; the area percentage Greater than or equal to The tags are classified as macro tags; Extract the shape features of each marked region, including the roundness calculated based on the boundary coordinates and the aspect ratio of the minimum bounding rectangle; the roundness C is defined as: ;in This represents the perimeter of the region. This sub-step outputs structured data containing the region coordinates, area percentage classification results, and shape feature parameters for each label, which serves as the input for step S32.

[0016] As an implementation scheme, furthermore, in step S32, the global feature extraction stream is implemented using a deep convolutional neural network ResNet50 to extract global contextual features of the image. The ResNet50 deep convolutional neural network includes an initial convolutional layer, four sequentially connected stages (Stage 1 to Stage 4), and a global average pooling layer; Stage 1 to Stage 4 sequentially downsample the input image, and the final output spatial size is... Global feature map with 2048 channels The network was trained using a large-scale oral medicine image dataset, in which each image was labeled with a gum health status tag.

[0017] The training objective of this deep convolutional neural network, ResNet50, is to minimize the cross-entropy loss function. L ce : ; in For real labels, To predict probabilities for the model, This represents the total number of categories. During training, a stochastic gradient descent optimizer was used with an initial learning rate of 0.01 and weight decay regularization was applied. The trained network weights were then used to extract the global feature map. This feature map serves as the basis for subsequent scale-adaptive attention fusion.

[0018] As one implementation, further, in step S32, the local high-resolution feature extraction stream crops high-resolution local image patches centered on micro-markers, and extracts fine texture features through a shallow convolutional network to obtain high-resolution local features; the specific process includes: receiving the coordinate information of the micro-markers obtained from the classification in sub-step S31. For each micro-marker, its geometric center point... Centered on the original digital image obtained in step S1, a high-resolution local image patch of a fixed size is cropped out. The size of the local image patch is... Pixels. This local image patch serves as the input to the focus stream. The focus stream contains a shallow convolutional network, which consists of two convolutional cascades, each containing one pixel. Convolutional layers, batch normalized layers, and ReLU activation function. The network does not perform downsampling and introduces an inflation rate of 1 / 2 in the second cascade. Dilated convolutions are used to preserve the spatial resolution of the feature maps. The focusing stream processes local image patches from the input, outputting a spatial size of... High-resolution local feature map with 256 channels This local feature map It preserves the fine texture and structural information of the micro-regions.

[0019] As one implementation scheme, furthermore, the marker guidance flow generates a multi-scale Gaussian heatmap based on the coordinates of all markers, as a spatial attention prior, which specifically includes: Receive the classification results and coordinate information of all tags output from substep S31. For each tag, apply different standard deviations based on its classification result. The two-dimensional Gaussian kernel function at its center coordinates A heatmap is generated at the location. For micro-markers, a heatmap with the same resolution as the original image is generated. Gaussian kernel standard deviation for Pixels. For mesoscopic markers, generate a quarter-resolution heatmap. Gaussian kernel standard deviation for Pixels. For macroscopic markings, generate a heatmap with a resolution of 1 / 16th. Gaussian kernel standard deviation The value is 20 pixels. All heatmap values ​​are normalized to [0,1]. The label guide stream further includes a lightweight coding network consisting of three parallel... Convolutional layers are constructed, respectively for , and The features are encoded and concatenated along the channel dimension, then fused through a fully connected layer to generate a multi-scale attention prior feature vector. This feature vector It encapsulates user-labeled multi-scale spatial prior information.

[0020] Furthermore, feature fusion is performed via a scale-adaptive attention fusion mechanism. Specifically, based on the label classification results, the scale-adaptive attention fusion mechanism is used to fuse the high-resolution local features or the spatial attention prior with the global context features, outputting a fused feature map. The scale-adaptive attention fusion mechanism is configured as follows: for micro-labels, local features are enhanced using the local feature map and corresponding heatmap and fused with the global features; for meso-labels, the corresponding regions in the global feature map are spatially reweighted using the heatmap; for macro-labels, the global feature map is channel-modulated using the heatmap; finally, the features processed by each branch are fused. Specifically, this includes: Receive global feature maps from the global stream. Local feature map from the focused flow and multi-scale attention prior feature vectors from the label-guided flow ; For micro-labeling, firstly, the local feature map is... Microscopic thermal maps of corresponding locations Feature enhancement is achieved through element-wise multiplication, resulting in... Use one Convolutional layers will Compressed into a single feature vector ; via space broadcast Add to global feature map The eigenvector at the spatial coordinates corresponding to the center position of the micro-marker; For mesoscopic markers, mesoscopic heatmaps Bilinear upsampling to Same space size The spatial attention weight map is obtained. ;Will and Perform element-wise multiplication, for The corresponding regions in the data are reweighted to obtain the following results. ; For macroscopic marking, macroscopic heatmap After global average pooling, the data is fed into a two-layer perceptron to generate channel attention weight vectors. The weight vector and The number of channels is the same; Each channel is multiplied by its corresponding weight to achieve channel modulation, resulting in... ; The features processed by the micro-label, meso-label, and macro-label branches are summed to output the fused image feature map. .

[0021] As one implementation, further, S33 includes: receiving the fused image feature map output from sub-step S32. The feature map The space dimensions are This corresponds to dividing the original digital image into spatially uniform portions. There are a total of 49 grid sub-regions. (Targeting the feature map) The feature vector at each spatial location (i,j) A classifier is used to map it to a sensitivity level value. The classifier consists of a fully connected layer followed by a Softmax activation function. Sensitivity level value. The sensitivity level is selected from a predefined set of levels, which contains discrete levels representing different cleaning power requirements. Each grid sub-region and its corresponding sensitivity level value are then assigned. The correlation is used to generate a spatial index matrix, which is the regional oral cavity sensitivity map. This map fully describes the predicted sensitivity level information of each segmented region of the oral cavity image.

[0022] As one implementation, further, step S4 includes: receiving a regional oral sensitivity map generated from sub-step S33, wherein the map stores sensitivity level information in the form of image grid coordinates. Before the user uses the water flosser for the first time, an oral coordinate system calibration procedure is performed: guiding the user to sequentially touch multiple preset physical reference points in the oral cavity with the tip of the water flosser nozzle, wherein the reference points include at least the midline point of the maxilla, the buccal point of the left upper first molar, and the lingual point of the right lower first molar. The nozzle pose data at each reference point is recorded by the water flosser's built-in sensor. The pixel coordinates corresponding to the above physical reference points are manually or automatically marked on the digital image. Based on the correspondence between multiple pairs of three-dimensional physical coordinates and two-dimensional image coordinates, a projection transformation matrix is ​​calculated using a direct linear transformation algorithm. The transformation matrix A mapping relationship was established between the real-time three-dimensional position data of the water flosser nozzle and the two-dimensional grid coordinates of the sensitivity map. This mapping relationship is stored in the system and invoked during the position matching step.

[0023] As one implementation, further, in step S5, the sensor includes an inertial measurement unit and a magnetometer. A Kalman filter algorithm is used to fuse multi-source sensor data, and the three-dimensional position coordinates and three-dimensional orientation Euler angles of the nozzle are calculated in real time. Specifically, step S5 includes: during the operation of the water flosser, the position tracking step continuously collects raw sensor data through the inertial measurement unit and magnetometer integrated within the water flosser nozzle. This data includes triaxial acceleration... Triaxial angular velocity and triaxial magnetic field strength The Kalman filter algorithm is used to fuse data from multiple sensor sources, and the three-dimensional position coordinates of the nozzle relative to the calibrated oral cavity coordinate system at the current moment are calculated in real time. The three-dimensional orientation Euler angles are used. The position matching step receives the real-time three-dimensional position coordinates. And call the projection transformation matrix established in step S4. .

[0024] Through formula The three-dimensional position coordinates are projected onto the two-dimensional image coordinate system of the sensitivity map to obtain the corresponding grid index. According to the grid index Search within the segmented oral sensitivity map to retrieve the sensitivity level identifier stored at the index location. This identifier As a direct input for intelligent control steps.

[0025] As one implementation, step S7 further includes a feedback-driven online calibration mechanism, which includes: Record user-manually overriding automatic control commands operation events, the events including manually set working parameters, nozzle position at the time of the event, and corresponding map grid index; When the frequency of manual operation events at a specific grid index exceeds a preset threshold within a unit of time, the user's actual desired sensitivity level in that area is calculated in reverse based on the manually set working parameters. Calculate the deviation between the actual expected level and the original predicted level in the map, and fine-tune the relevant parameters in the feature fusion model based on the deviation; The sensitivity levels of the affected areas were recalculated using the fine-tuned model, and the sub-regional oral sensitivity maps were updated.

[0026] According to a second aspect of this application, a smart control system for a water flosser based on the identification of oral sensitive areas is provided, comprising: The image acquisition module is used to acquire digital images of the user's oral cavity; The user interaction module is used to receive and process graphical mark input from users on digital images; The atlas generation module is used to generate regional oral sensitivity atlases based on images and labeling information; The coordinate mapping module is used to establish the projection transformation relationship between the image coordinate system and the oral cavity physical coordinate system; The pose tracking module is integrated into the water flosser nozzle and is used to track the nozzle's pose in the oral cavity in real time. The processing and control module is used to match the sensitivity level in the pose data atlas and generate control instructions for the working parameters of the water flosser accordingly.

[0027] According to a third aspect of this application, a water flosser is provided, which includes the above-described intelligent control system for water flossers based on the identification of oral sensitive areas.

[0028] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the steps of the above-described intelligent control method for a water flosser based on the identification of oral sensitive areas.

[0029] Compared with the prior art, the present invention has the following beneficial effects: 1. By adopting an image recognition model that combines multi-scale feature extraction and adaptive fusion, the scale mismatch between the user's micro-level subjective markings and the model's macro-level feature perception is effectively solved, significantly improving the accuracy of oral sensitivity map generation. This enables the model to accurately reflect the subtle discomfort experienced by individual users, balancing medical objectivity with individual subjective perception.

[0030] 2. By introducing a closed-loop calibration mechanism based on online user feedback, the system can dynamically optimize model parameters and spectral data according to real-time manual operations by users, realizing a leap from static preset to dynamic evolution of cleaning strategies, thereby continuously improving the accuracy of personalized adaptation and user experience.

[0031] 3. By establishing a precise mapping relationship from two-dimensional image space to three-dimensional oral physical space, and combining it with high-precision real-time pose tracking, a complete technical closed loop of "image recognition - spatial positioning - dynamic control" is opened up, ensuring that personalized cleaning decisions can be accurately and in real time converted into physical actions for specific oral locations, thus enabling precise personalized cleaning to be reliably achieved.

[0032] Overall, this application proposes a novel intelligent control method for oral cleaning that integrates user subjective perception guidance with multimodal data fusion. By quantifying, learning, and responding to user subjective labels as key prior information, and driving the actuator to complete graded cleaning operations, a new paradigm of oral care integrating human-machine collaboration, perception, and execution is realized for the first time in a single system. Ultimately, this achieves a truly effective, comfortable, and safe personalized cleaning effect that is tailored to the individual's immediate condition. Attached Figure Description

[0033] The present invention can be better understood by referring to the description given below in conjunction with the accompanying drawings, in which the same or similar reference numerals are used throughout the drawings to denote the same or similar parts. These drawings, together with the following detailed description, are incorporated in and form part of this specification, and are used to further illustrate preferred embodiments of the invention and explain the principles and advantages of the invention. In the drawings: Figure 1 This is a flowchart illustrating the overall process of intelligent control for oral sensitive area identification and graded cleaning in an embodiment of the present invention. Figure 2 This is a detailed flowchart of the process for generating multi-scale oral sensitivity maps according to an embodiment of the present invention; Figure 3 This is a block diagram illustrating the operation of the water flosser position tracking and map matching in an embodiment of the present invention. Figure 4 This is a block diagram of online calibration and optimization of the feedback-driven model in an embodiment of the present invention. Detailed Implementation

[0034] Embodiments of the present invention will now be described with reference to the accompanying drawings. Elements and features described in one drawing or embodiment of the invention may be combined with elements and features shown in one or more other drawings or embodiments. It should be noted that, for clarity, representations and descriptions of components and processes unrelated to the invention and known to those skilled in the art have been omitted from the drawings and description.

[0035] This invention provides an intelligent control method for oral irrigators based on the identification of oral sensitive areas. See [link to relevant documentation]. Figure 1 It includes the following steps: S1. Image acquisition steps: Acquire a digital image of the user's oral gingival region; S2, User Marking Step: Receive graphical marking input from a user on the digital image for at least one sensitive area, the marking input containing location information of the sensitive area; S3. Sensitivity Map Generation Step: Based on the digital image and the location information of the user-marked areas, generate a regional oral sensitivity map containing different sensitivity level identifiers; see [link to relevant documentation]. Figure 2This step specifically includes: S31, Marker Scale Analysis and Classification Sub-step: Receive the location information of the user's marks, calculate the area ratio and shape features of each marked region relative to the digital image, and draw and classify them into micro-markers, meso-markers or macro-markers based on this. S32, Multi-scale Feature Extraction and Fusion Sub-step: Receive the digital image and the classified label information, and process them through a multi-stream feature extraction network; the network includes a global stream for extracting macroscopic contextual features, a focusing stream for extracting high-resolution local features, and a label-guided stream for encoding multi-scale label priors; based on the label classification results, a scale-adaptive attention fusion mechanism is used to fuse the microscopic features of the focusing stream or the multi-scale attention priors of the label-guided stream with the macroscopic features of the global stream, and output the fused image features; S33, Atlas Division and Assignment Sub-step: Receive the fused image features, divide the oral cavity region corresponding to the digital image into several sub-regions, and assign a sensitivity level identifier to each sub-region based on the image features corresponding to each sub-region, thereby generating the regional oral cavity sensitivity atlas; S4. Mapping Step: Establish the mapping relationship between the regional oral sensitivity map and the user's actual oral physical region; S5. Position tracking step: During the operation of the water flosser, the actual position data of the water flosser nozzle in the user's mouth is obtained in real time; S6. Position matching step: Match the actual position data of the water flosser nozzle with the regional oral sensitivity map to determine the sub-region and its sensitivity level corresponding to the current nozzle position; S7. Intelligent control steps: Based on the matched sensitivity level, automatically generate and issue corresponding oral irrigator working parameter control commands to control the oral irrigator to adjust the output water pressure or water flow mode.

[0036] Step S1 begins with the user using a mobile terminal equipped with a high-definition camera to capture images of the gingival region inside the oral cavity. The user needs to open their lips and adjust the phone angle under sufficient and uniform lighting conditions to ensure that the camera can capture the target tissues, including the gingival margins, gingival papillae, and attached gingiva, clearly and completely. The dedicated application on the mobile terminal will call the camera module and guide the user to complete the shooting. The raw image data obtained is immediately transmitted to the image preprocessing module built into the application. This module performs a series of standardized operations, including automatic cropping to focus on the oral cavity area, color correction to eliminate the influence of ambient light color cast, uniform scaling of resolution to a preset pixel size, and contrast enhancement to highlight the boundary between the gingival tissue and the teeth. The preprocessed standardized digital image is then stored in the temporary memory of the mobile terminal and serves as the sole visual data source for all subsequent image analysis processes of the entire system. The principle of this step is to eliminate the interference caused by hardware differences and shooting environment variables through standardized image acquisition and preprocessing processes, providing high-quality and consistent input data for subsequent machine learning-based image recognition models. This is the physical basis for ensuring the accuracy and stability of model recognition.

[0037] Step S2 is implemented based on the graphical user interface provided by the mobile terminal application. This interface loads and displays the standardized digital image generated in step one. The user interacts directly with the image via the touchscreen, using preset marking tools to precisely indicate subjectively perceived sensitive or uncomfortable areas on the displayed gingival image. The marking tool library includes dot marking tools for marking single sensitive locations, freehand drawing tools for outlining irregularly shaped sensitive areas, and filling tools for covering larger sensitive areas. Each touch interaction is recorded as a series of continuous screen coordinate points. The application's background logic converts these coordinate points into closed polygon vector graphic data relative to the pixel coordinate system of the standardized digital image, and simultaneously records... The tagging operation timestamps and generates a unique tag identifier. All the vector data, timestamps, and identifiers of the tags are integrated into a structured tag data set. This set is bound to the corresponding standardized digital image and prepared for transmission to the subsequent processing module. The principle of this step is to transform the user's subjective, qualitative oral discomfort into objective, quantifiable computer graphics data. This precise location information provides crucial supervision signals and spatial attention guidance for the subsequent image recognition model. In particular, it lays the data foundation for solving the scale mismatch problem between the user's micro-tags and the model's macro-feature perception, enabling the model's learning to closely match the user's actual physiological experience rather than relying solely on general pathological features.

[0038] The specific process of step S31, label scale analysis and classification sub-step, is as follows: Receive graphical marker input from the user marking step S2, which contains a sequence of boundary coordinates for each sensitive region defined by the user on the digital image; The pixel area enclosed by each marked region is calculated based on the boundary coordinate sequence. Simultaneously calculate the total pixel area of ​​the digital image. ; Calculate the area percentage of each marked region ; Area proportion Less than the first preset threshold The markers are classified as micro-markers, and the first preset threshold It is 1%. The area percentage is... Greater than or equal to And less than the second preset threshold The markers are classified as mesoscopic markers, and the second preset threshold... It is 5%. The area percentage is... Greater than or equal to The markers are classified as macro-level markers. Simultaneously, the shape features of each marker region and the aspect ratio of its minimum bounding rectangle are extracted. Shape features include roundness calculated based on boundary coordinates. ;in The perimeter of the region is given. This sub-step outputs structured data containing the region coordinates, area percentage classification results, and shape feature parameters for each label, which serves as input for the multi-scale feature extraction and fusion sub-step.

[0039] The global flow in step S32, multi-scale feature extraction and fusion, is implemented using a deep convolutional neural network. The input to this network is the original digital image obtained in step S1, image acquisition, with the image size normalized to [value missing]. Pixels. The deep convolutional neural network uses the ResNet50 architecture, which includes an initial convolutional layer, four sequentially connected stages (Stage 1 to Stage 4), and a global average pooling layer. Stages 1 to 4 downsample the input image sequentially, resulting in a final output spatial size of [missing information]. Global feature map with 2048 channels The network was trained using a large-scale oral medicine image dataset, in which each image was labeled with a gum health status tag.

[0040] The training objective of the ResNet50 deep convolutional neural network architecture is to minimize the cross-entropy loss function: ; in For real labels, To predict probabilities for the model, This represents the total number of categories. During training, a stochastic gradient descent optimizer was used with an initial learning rate of 0.01 and weight decay regularization was applied. The trained network weights were then used to extract the global feature map. This feature map serves as the basis for subsequent scale-adaptive attention fusion.

[0041] The focusing flow processing in step S32, multi-scale feature extraction and fusion sub-step, is as follows: Receive the coordinate information of the micro-markers obtained from the classification in sub-step S31. For each micro-marker, determine its geometric center point. Centered on the original digital image obtained in step S1, a high-resolution local image patch of a fixed size is cropped out. The size of the local image patch is... Pixels. This local image patch serves as the input to the focus stream; Focused Stream contains a shallow convolutional network, which consists of two convolutional cascades, each containing a... Convolutional layers, batch normalized layers, and ReLU activation function. The network does not perform downsampling and introduces an inflation rate of 1 / 2 in the second cascade. Dilated convolutions are used to preserve the spatial resolution of the feature maps. The focusing stream processes local image patches from the input, outputting a spatial size of... High-resolution local feature map with 256 channels This local feature map It preserves the fine texture and structural information of the micro-regions.

[0042] The label-guided flow processing in step S32, multi-scale feature extraction and fusion sub-step, is as follows: Receive the classification results and coordinate information of all tags output from sub-step S31; For each label, different standard deviations are applied based on its classification result. The two-dimensional Gaussian kernel function at its center coordinates Heatmap generated at: For micro-labels, a heatmap with the same resolution as the original image is generated. Gaussian kernel standard deviation for Pixel; For mesoscopic markers, generate a quarter-resolution heatmap. Gaussian kernel standard deviation for Pixel; For macroscopic markings, generate a heatmap with a 1 / 16th resolution. Gaussian kernel standard deviation 20 pixels; The numerical range of all heatmaps is normalized to [0,1]; The marker-guided flow further includes a lightweight coding network consisting of three parallel... Convolutional layers are constructed, respectively for , and The features are encoded and concatenated along the channel dimension, then fused through a fully connected layer to generate a multi-scale attention prior feature vector. This feature vector It encapsulates user-labeled multi-scale spatial prior information.

[0043] The specific operation flow of the scale-adaptive attention fusion mechanism in step S32, multi-scale feature extraction and fusion sub-step, is as follows: Receive global feature maps from the global stream. Local feature map from the focused flow and multi-scale attention prior feature vectors from the label-guided flow ; The scale-adaptive attention fusion mechanism performs branching operations based on the classification results of the S31 sub-step. For micro-labels, the local feature map is first... Microscopic thermal maps of corresponding locations Feature enhancement is achieved through element-wise multiplication, resulting in... Then, using a Convolutional layers will Compressed into a single feature vector Finally, it will be broadcast via space. Add to global feature map The eigenvector at the spatial coordinates corresponding to the center position of the microscopic marker. For mesoscopic markers, the mesoscopic heatmap... Bilinear upsampling to Same space size The spatial attention weight map is obtained. .Will and Perform element-wise multiplication, for The corresponding regions in the data are reweighted to obtain the following results. For macroscopic markings, the macroscopic heatmap will be used. After global average pooling, the data is fed into a two-layer perceptron to generate channel attention weight vectors. The weight vector and The number of channels is the same. Each channel is multiplied by its corresponding weight to achieve channel modulation, resulting in... Finally, the features processed by all branches are summed to output the fused image feature map. .

[0044] The specific process of step S33, the map partitioning and assignment sub-step, is as follows: Receive the fused image feature map output from sub-step S32 The feature map The space dimensions are This corresponds to dividing the original digital image into spatially uniform portions. There are a total of 49 grid sub-regions; For feature maps The feature vector at each spatial location (i,j) A classifier is used to map it to a sensitivity level value. The classifier consists of a fully connected layer followed by a Softmax activation function. Sensitivity level value. The sensitivity level is selected from a predefined set of levels, which contains discrete levels representing different cleaning power requirements. Each grid sub-region and its corresponding sensitivity level value are then assigned. The correlation is used to generate a spatial index matrix, which is the regional oral cavity sensitivity map. This map fully describes the predicted sensitivity level information of each segmented region of the oral cavity image.

[0045] The specific process of step S4, the map mapping step, is as follows: The user receives a regional oral sensitivity map generated from sub-step S33. This map stores sensitivity level information in the form of image grid coordinates. Before the user uses the water flosser for the first time, an oral coordinate system calibration procedure is performed: the user is guided to sequentially touch multiple preset physical reference points in the oral cavity with the tip of the water flosser nozzle. These reference points include at least the midline point of the maxilla, the buccal point of the left upper first molar, and the lingual point of the right lower first molar. The nozzle pose data at each reference point is recorded using the water flosser's built-in sensor. The pixel coordinates corresponding to the aforementioned physical reference points are manually or automatically marked on the digital image. Based on the correspondence between multiple pairs of three-dimensional physical coordinates and two-dimensional image coordinates, a projection transformation matrix is ​​calculated using a direct linear transformation algorithm. The transformation matrix A mapping relationship was established between the real-time three-dimensional position data of the water flosser nozzle and the two-dimensional grid coordinates of the sensitivity map. This mapping relationship is stored in the system and invoked during the position matching step.

[0046] See Figure 3 The specific process of the S5 position tracking step and the S6 position matching step is as follows: During the operation of the water flosser, the position tracking step continuously collects raw sensor data through the inertial measurement unit and magnetometer integrated in the water flosser nozzle. The data includes triaxial acceleration. Triaxial angular velocity and triaxial magnetic field strength The Kalman filter algorithm is used to fuse data from multiple sensor sources, and the three-dimensional position coordinates of the nozzle relative to the calibrated oral cavity coordinate system at the current moment are calculated in real time. The three-dimensional orientation Euler angles are used. The position matching step receives the real-time three-dimensional position coordinates. And call the projection transformation matrix established in step S4. .

[0047] Through formula The three-dimensional position coordinates are projected onto the two-dimensional image coordinate system of the sensitivity map to obtain the corresponding grid index. According to the grid index Search within the segmented oral sensitivity map to retrieve the sensitivity level identifier stored at the index location. This identifier As a direct input for intelligent control steps.

[0048] See Figure 4 The S7 Smart Control Steps also include a feedback-driven online calibration mechanism. This mechanism continuously records user actions during operation, such as manually overriding automatic control commands via the physical buttons on the water flosser or the accompanying app. Each event includes the manually set operating parameters. Real-time position of the nozzle at the time of the incident and the corresponding mapping grid index A mapping table between the system's preset sensitivity levels and standard operating parameters. When in a specific grid index The system detected that the frequency of manual operation events within a unit of time exceeded a threshold. At that time, it was determined that there was a discrepancy between the automatic control of the area and the user's subjective experience.

[0049] At this point, the system initiates the calibration process: First, based on the manually set parameters... Reverse lookup of the mapping table This allows us to calculate the user's actual desired sensitivity level in that area. Next, the grade deviation will be... ,in This represents the original predicted rank of the map, associated with the user-labeled data used to initially generate the predicted rank for this region. The system is based on bias. The gradient descent algorithm is used to fine-tune the fusion weight parameters related to the label category within the attention fusion mechanism in sub-step S32. The model's feature representation and sensitivity level are recalculated using fine-tuned parameters, and the corresponding grid level values ​​in the regional oral sensitivity map are updated, thus achieving personalized online optimization of the model.

[0050] To address the contradiction between the limitations of traditional oral hygiene devices in simultaneously considering the user's subjective microscopic sensations and objective medical pathological characteristics, this invention provides an innovative intelligent control scheme. This scheme constructs an image recognition model architecture that combines multi-scale feature extraction and adaptive fusion. It deploys a global feature stream, a local high-resolution feature stream, and a label-guided stream in parallel within a neural network. Based on the scale classification results actively labeled by the user, a differentiated attention fusion strategy is employed to seamlessly embed the user's microscopic subjective labels into a macroscopic medical feature atlas, thereby generating an oral sensitivity atlas that is both objective and individualized.

[0051] To achieve a precise closed loop from visual perception to physical execution, this invention proposes a full-link mapping and control method: by calibrating physical reference points, a projection transformation relationship between a two-dimensional sensitivity map and a three-dimensional oral cavity space is established; by using high-frequency inertial sensor data fusion, sub-centimeter-level real-time positioning of the water flosser nozzle is achieved; and the water pressure and spray mode are adjusted in real time according to the sensitivity level corresponding to the grid.

[0052] Furthermore, in response to the dynamic changes in the user's oral cavity status, this invention introduces a feedback-driven online calibration technique: real-time capture and analysis of the user's manual coverage operation of the device settings, correlation with the device pose and atlas prediction results to form a supervisory signal, driving the reverse fine-tuning of feature fusion weights and dynamic updates of the atlas.

[0053] The technical effects of this invention are significant: 1) It fundamentally solves the scale mismatch problem between the user's subjective feelings and the model's macroscopic feature perception, ensuring individualized and accurate response of cleaning intensity; 2) It realizes the transformation from static preset personalization to dynamic evolutionary personalization, continuously adapting to the user's real-time state; 3) It opens up a complete technical closed loop of "image recognition-spatial positioning-dynamic control", reliably transforming intelligent decisions into specific physical cleaning actions, creating a new paradigm of oral care that integrates human-machine collaboration, perception and execution.

[0054] The method of this invention is not limited to being executed in the chronological order described in the specification, but can also be executed in other chronological orders, in parallel, or independently. Therefore, the execution order of the method described in this specification does not constitute a limitation on the technical scope of this invention.

[0055] Although the invention has been disclosed above through the description of specific embodiments, it should be understood that all the embodiments and examples described above are exemplary and not restrictive. Those skilled in the art can design various modifications, improvements, or equivalents to the invention within the spirit and scope of the appended claims. These modifications, improvements, or equivalents should also be considered to be included within the protection scope of the invention.

Claims

1. A method for intelligent control of a water flosser based on the identification of oral sensitive areas, characterized in that, Includes the following steps: S1. Acquire a digital image of the user's oral gingival region; S2. Receive graphical mark input from the user on at least one sensitive area in the digital image, wherein the graphical mark input contains location information of the sensitive area; S3. Based on the digital image and the graphical marker input, a regional oral cavity sensitivity map is generated by multi-scale analysis that integrates user subjective marker information and objective image features; the regional oral cavity sensitivity map divides the oral cavity region into multiple grids and assigns a level label representing the cleanliness sensitivity to each grid. S4. Establish the projection transformation relationship between the two-dimensional image coordinate system of the sub-regional oral sensitivity map and the three-dimensional physical coordinate system of the water flosser nozzle moving in the oral cavity. S5. The position and pose data of the oral irrigator nozzle in the three-dimensional physical coordinate system are obtained in real time through the sensor built into the nozzle. S6. Based on the projection transformation relationship, the real-time acquired nozzle pose data is mapped to the regional oral cavity sensitivity map to determine the grid in which it is located and the corresponding sensitivity level identifier. S7. Based on the sensitivity level indicator, automatically adjust the working parameters of the water flosser in the current position.

2. The intelligent control method for a water flosser according to claim 1, characterized in that, Step S3 specifically includes: S31. Parse the user's marker input, calculate the area ratio of each marker region relative to the digital image, and classify the markers into micro markers, meso markers, or macro markers; S32. Based on the digital image and the classified label information, feature extraction is performed through parallel global feature extraction stream, local high-resolution feature extraction stream and label guidance stream, and feature fusion is performed through scale-adaptive attention fusion mechanism to output the fused feature map. S33. Divide the fused feature map into grids, and predict the sensitivity level of each grid using a classifier to determine the regional oral sensitivity map.

3. The intelligent control method for a water flosser according to claim 2, characterized in that, The global feature extraction stream in step S32 is implemented using a deep convolutional neural network to extract global contextual features of the image; the local high-resolution feature extraction stream extracts features by cropping a local image centered on micro-markers and performing feature extraction to capture fine texture features; the marker guidance stream generates a multi-scale heatmap based on the category and coordinates of the markers to provide spatial attention priors.

4. The intelligent control method for a water flosser according to claim 3, characterized in that, In step S32, the scale-adaptive attention fusion mechanism is configured to: for micro-labels, enhance local features using local feature maps and corresponding heatmaps and fuse them with global features; For mesoscopic labels, spatial reweighting of corresponding regions in the global feature map is performed using heatmaps; for macroscopic labels, channel modulation of the global feature map is performed using heatmaps; finally, the features processed by each branch are fused.

5. The intelligent control method for a water flosser according to claim 1, characterized in that, Step S4 includes: Receive a regional oral cavity sensitivity map, wherein the regional oral cavity sensitivity map stores sensitivity level information in the form of image grid coordinates; Before a user uses the water flosser for the first time, an oral coordinate system calibration procedure is performed: the user is guided to use the tip of the water flosser nozzle to touch multiple preset physical reference points in the oral cavity in sequence. The physical reference points include at least the midline point of the maxilla, the buccal point of the upper left first molar, and the lingual point of the lower right first molar. The nozzle pose data when each reference point is touched is recorded by the built-in sensor of the water flosser. Manually or automatically mark the pixel coordinates corresponding to the aforementioned physical reference points on the digital image; Based on the correspondence between multiple pairs of three-dimensional physical coordinates and two-dimensional image coordinates, a projection transformation matrix is ​​calculated using a direct linear transformation algorithm. This transformation matrix establishes a mapping relationship from the real-time three-dimensional position data of the water flosser nozzle to the two-dimensional grid coordinates of the sub-regional oral sensitivity map.

6. The intelligent control method for a water flosser according to claim 1, characterized in that, In step S5, the sensor includes an inertial measurement unit and a magnetometer. A Kalman filter algorithm is used to fuse data from multiple sensor sources and calculate the three-dimensional position coordinates and three-dimensional orientation Euler angles of the nozzle in real time.

7. The intelligent control method for a water flosser according to claim 1, characterized in that, Step S7 further includes a feedback-driven online calibration mechanism, which includes: Record user-manually overriding automatic control commands operation events, the operation events including manually set working parameters, nozzle position at the time of the event, and corresponding map grid index; When the frequency of manual operation events at a specific grid index exceeds a preset threshold within a unit of time, the user's actual desired sensitivity level in that area is calculated in reverse based on the manually set working parameters. Calculate the deviation between the sensitivity level and the original predicted level in the map, and fine-tune the relevant parameters in the feature fusion model based on the deviation; The sensitivity levels of the affected areas were recalculated using the fine-tuned model, and the sub-regional oral sensitivity maps were updated.

8. A smart control system for a water flosser based on the identification of oral sensitive areas, used to implement the method according to any one of claims 1-7, characterized in that, include: The image acquisition module is used to acquire digital images of the user's oral cavity; The user interaction module is used to receive and process graphical mark input from users on digital images; The atlas generation module is used to generate regional oral sensitivity atlases based on images and labeling information; The coordinate mapping module is used to establish the projection transformation relationship between the image coordinate system and the oral cavity physical coordinate system; The pose tracking module is integrated into the water flosser nozzle and is used to track the nozzle's pose in the oral cavity in real time. The processing and control module is used to match the sensitivity level in the pose data atlas and generate control instructions for the working parameters of the water flosser accordingly.

9. A dental flosser, characterized in that, It includes the intelligent control system for the oral irrigator as described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the intelligent control method for a water flosser as described in any one of claims 1-7.

Images