Image processing system and method of processing images obtained from personal care device
By removing the presence of bristles in the image processing system of the head of the personal care device, a background image is generated, the problem of bristle interference is solved, and the image quality and accuracy of diagnostic information is improved.
Patent Information
- Application Number
- CN202380051598.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-07-04
- Filing Date
- 2023-07-03
- Publication Date
- 2025-02-18
AI Technical Summary
When the camera is mounted on the head of a personal care device, the presence of bristles causes interference and occlusion within the camera image, increasing the complexity of image processing.
An image processing system is provided to receive an image sequence from the head of a personal care device through a processor, remove moving parts in the image sequence, and generate a background image, thereby removing the presence of a cleaning element.
Improved image quality, addressed occlusions generated by cleaning elements, simplified subsequent image processing, such as caries and plaque detection, and improved the quality of user experience and diagnostic information.
Smart Images

Figure CN119486636A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to integrating imaging into the head of a personal care device. In particular, the present invention relates to the processing of said images. Background Art
[0002] Cameras are becoming ubiquitous in the personal care field in general, and in particular, electric toothbrushes with integrated cameras are emerging in the market. The camera images are processed, for example, to provide feedback to the user about their brushing routine, or to provide diagnostic information. For example, caries and plaque can be determined. The imaging can also be used to determine the position of the oral care head within the oral cavity (more accurately than using only motion sensors), so that the user's oral care routine can be analyzed more accurately. This information can be used to guide the consumer so that he / she reaches all positions during his / her oral care routine. The history of the user's oral health can be collected.
[0003] Thus, an imaging-assisted oral care device can be an enabler for image-based diagnostics, advanced position sensing for the head of the oral care device, oral care treatment planning (e.g., orthodontics, braces), or treatment monitoring (e.g., periodontitis). These new capabilities can be seamlessly integrated into the user's normal brushing hygiene regimen without the need for a separate device, such as a smartphone or dental endoscope.
[0004] With such devices, a new generation of problems arises in relation to the information that these sensors can detect. In particular, camera-equipped toothbrush devices suffer from the disadvantage that the presence of bristles causes interference and occlusion within the camera image. As a result, there is algorithmic complexity caused by the image processing required to handle complex images.
[0005] Acquiring images and video streams free of such interferences and occlusions would greatly improve both: the quality of the user experience (usability, enjoyment, and aesthetic appeal of the images); and the quality and quantity of information that can be derived from the oral images themselves, e.g., for diagnostic purposes.
[0006] Miniaturized cameras can be easily placed almost anywhere in the device, such as in the head of an oral care device. In electric toothbrushes, the key design features are the type of oscillation (i.e., vibrations in the plane of the toothbrush head) and the geometry of the bristles. Because there are many different head types dedicated to specific purposes (gum types, etc.), or simply because new more optimized versions are available, there is a great deal of variability in the geometry of the bristles. The type of oscillation is usually in a well-defined plane, such as the vertical plane (when brushing the sides of the teeth).
[0007] Depending on the force and type of movement applied by the user, the oscillating mode causes the bristles that contact the teeth to drag and bend during the oscillation. The problem is that this mechanical behavior will cause some of the bristles to be within the field of view of the camera mounted in the head of the oral care device. This blocking or occlusion of the image is a problem to be solved.
[0008] Bristle position, geometry, density, etc. vary with head type. It is possible that the camera and optics could be designed to have minimal obstruction for specific bristle characteristics, but this would result in expensive and laborious adaptation of the optics or camera design or mounting to the specific bristle design.
[0009] Instead, the present invention aims to solve the problem in a more general way.
[0010] US2015107034A1 describes a dental care device for automatically and independently detecting and removing plaque. The document does not describe removing the bristles recorded in the acquired image. Summary of the invention
[0011] The invention is defined by the claims.
[0012] According to an example according to one aspect of the present invention, there is provided an image processing system for processing an image received from an imaging device mounted at a head of a personal care device, the image processing system comprising:
[0013] A processor configured to:
[0014] receiving a sequence of images from an imaging device at a head of a personal care device, the personal care device including a cleaning element visible in at least one image in the sequence of images; and
[0015] The sequence of images is processed to generate a background image with moving portions in the sequence of images removed such that cleaning elements of the personal care device are removed from the background image.
[0016] This image processing approach provides greatly improved image quality, resolving occlusions created by cleaning elements. This simplifies subsequent image processing, such as that used for caries and plaque detection. Because data analysis can be achieved using subsets of the image, the required data processing power and amount of data is reduced.
[0017] The imaging device may be a camera. Of course, other imaging devices (eg, laser scanners, etc.) capable of capturing images from the head of the personal care device may be used.
[0018] Digital cameras are known to be embedded with technology (in the form of image processing algorithms) that can remove moving objects from static objects. For example, such algorithms enable taking a picture of a building, for example, without a person in front of the building. The present invention uses a similar approach to remove the presence of cleaning elements from images and videos captured by a camera installed in a personal care device. This improvement in the generated background image improves the quality of the user experience and increases the quality and amount of information that can be derived from the image itself for diagnostic purposes or for providing advice to the user.
[0019] The personal care device may be a toothbrush and the imaging device may be embedded in the head of the toothbrush.In the context of a toothbrush, the cleaning elements are bristles.
[0020] The personal care device may be a skin exfoliating brush, and the imaging device may be embedded in the exfoliating portion of the exfoliating brush.In the context of an exfoliating brush, the cleaning element is a filament used for exfoliation.
[0021] Typically, a cleaning element is an element that is used to perform a cleaning action on a user. The cleaning element is typically flexible. This can enable the cleaning element to perform a cleaning action when they are pushed against a portion of the user (e.g., teeth, skin, etc.).
[0022] Image registration may be used for the image sequence. However, this may not be necessary since the background moves much slower than the cleaning elements (eg bristles).
[0023] It is equivalent to processing the sequence of images to generate a background image with moving parts of the images removed.
[0024] The processor may be adapted to process the sequence of images to generate the background image by applying a median filter to the sequence of images.
[0025] This has been shown to be an effective and simple method for removing or reducing the effects due to moving objects so that only static objects are rendered in the picture. The "reduced" static objects can then be further analyzed.
[0026] The processor may also be configured to iteratively update the background image using the most recently available image in the sequence of images.
[0027] The processor may be configured to iteratively update the background image by summing the background image with a most recently available image, wherein the most recently available image is weighted based on the cleaning element threshold.
[0028] The processor can also be configured to:
[0029] Detect features in image sequences;
[0030] Computing the optical flow of features between images in an image sequence;
[0031] Using independent component analysis of features to identify components in an image sequence;
[0032] Use the corresponding optical flow to determine the frequency of the component;
[0033] identifying a cleaning element in the sequence of images based on a component having a frequency within a frequency range based on a known frequency of the cleaning element;
[0034] generating a cleaning element mask based on the identified cleaning elements; and
[0035] The bristle threshold is updated by comparing the cleaning element mask to a background mask generated by determining the difference between a background image and an image in the image sequence.
[0036] The cleaning element threshold may be a time-varying threshold that is updated over time. Each update of the cleaning element threshold may be based on the above steps as they include detection and identification of cleaning elements, thereby improving the accuracy of the cleaning element threshold.
[0037] The cleaning element mask generated above is a mask complementary to a background generated mask that can be obtained using a background image (eg, background image - most recent image).
[0038] When the personal care device is a toothbrush, the cleaning element mask may be, for example, a bristle mask. Of course, when other personal care devices (such as exfoliating brushes) are used, the cleaning element mask may also be used.
[0039] Detecting features in an image sequence can be performed using known feature detection methods. Examples of such methods are provided in "Object recognition from Local Scale-Invariant Features" by Lowe, DG, and "Semi-Local Affine Parts for Object Recognition" by S. Lazebnik et al. Of course, other feature detection methods may also be used.
[0040] Calculating the optical flow can be performed using known optical flow methods. Examples are provided in Chapter 15, "Optical Flow Estimation," of the Handbook of Mathematical Models in Computer Vision by D. Fleet and Y Weiss. Of course, other optical flow methods may also be used.
[0041] Independent component analysis can be performed using the method provided in the book "Independent Component Analysis: A Tutorial Introduction" by James V. Stone. Of course, other independent component analysis methods can be used.
[0042] The processor may be configured to generate a cleaning element mask by applying a morphological operation to the shapes of the identified cleaning elements relative to the images in the sequence of images.
[0043] In other words, the shapes of the cleaning elements in the image (ie, the areas of the image occluded by the cleaning elements) are used to generate a cleaning element mask by applying known morphological operations (eg, erosion and dilation) to the shapes of the cleaning elements.
[0044] The processor may be configured to use one or more of the known geometry of the cleaning elements and the known color of the cleaning elements to detect features in the sequence of images.
[0045] A priori information of the cleaning elements (ie natural frequency and / or geometry and / or color) can be used to improve the likelihood of feature detection to detect the cleaning elements.
[0046] The processor may also be configured to apply a high pass flow filter to the optical flow and to denoise the optical flow.
[0047] The present invention also provides an electric toothbrush, comprising:
[0048] a brush head including bristles, an imaging device, and a lighting system; and
[0049] The above-mentioned image processing system is used to process images captured by an imaging device.
[0050] The present invention also provides a method for processing an image received from an imaging device mounted at a head of a personal care device, the method comprising:
[0051] receiving a sequence of images from an imaging device, the personal care device comprising a cleaning element visible in at least one image in the sequence of images; and
[0052] The sequence of images is processed to generate a background image with moving portions in the sequence of images removed such that cleaning elements of the personal care device are removed from the background image.
[0053] Processing the sequence of images to generate the background image may include applying a temporal median filter to the sequence of images.
[0054] The method may further comprise iteratively updating the background image using the most recently available image in the sequence of images.
[0055] Iteratively updating the background image may include summing the background image with a most recently available image, wherein the most recently available image is weighted based on the cleaning element threshold.
[0056] The method may further include:
[0057] Detect features in image sequences;
[0058] Computing the optical flow of features between images in an image sequence;
[0059] Using independent component analysis of features to identify components in an image sequence;
[0060] Use the corresponding optical flow to determine the frequency of the component;
[0061] identifying cleaning elements in the sequence of images based on components having frequencies within a frequency range based on known frequencies of the cleaning elements; and
[0062] generating a cleaning element mask based on the identified cleaning elements; and
[0063] The bristle detection threshold is updated by comparing the cleaning element mask to a background mask generated by determining the difference between a background image and an image in the image sequence.
[0064] The invention also provides a computer program comprising computer program code, which is suitable for implementing the above method when the program is run on a computer.
[0065] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] For a better understanding of the invention, and in order to more clearly show how the invention may be implemented, reference will now be made, by way of example only, to the accompanying drawings, in which:
[0067] Figure 1 A toothbrush head with a camera is shown;
[0068] Figure 2 A method for providing feedback to a user of a toothbrush is shown;
[0069] Figure 3 A method for providing a video stream without bristles is shown;
[0070] Figure 4 A method for updating the bristle threshold is shown;
[0071] Figure 5A diagram showing a method for updating a bristle threshold is shown;
[0072] Figure 6 An image with bristles segmented out is shown;
[0073] Figure 7 The average grey level of the frame showing the oral cavity; and
[0074] Figures 8 to 10 Binary masks generated in simulation using different thresholds are shown. DETAILED DESCRIPTION
[0075] The present invention will be described with reference to the accompanying drawings.
[0076] It should be understood that the detailed description and specific examples, although indicating exemplary embodiments of the device, system and method, are intended only for illustrative purposes and are not intended to limit the scope of the invention. These and other features, aspects and advantages of the device, system and method of the present invention will become better understood from the following description, the appended claims and the accompanying drawings. It should be understood that the drawings are only schematic and are not drawn to scale. It should also be understood that the same reference numerals are used in all drawings to indicate the same or similar parts.
[0077] The present invention provides an image processing system in which images received from a camera mounted at a head of a personal care device are processed. A cleaning element of the personal care device is visible in at least one of the images. The images are processed to generate a background image with moving parts in the image sequence removed, so that the cleaning element of the personal care device is removed from the background image. The image provides improved image quality, resolving occlusions caused by the cleaning element. This simplifies subsequent image processing, such as image processing for caries and plaque detection.
[0078] Figure 1 A toothbrush head 100 is shown with a camera 104. The camera 104 is placed approximately in the center of the bristles 102. An illumination device 106 (e.g., a light emitting diode LED) is also disposed approximately in the center of the toothbrush head 100 to illuminate the oral cavity when the toothbrush is in use, so that the camera 104 can obtain a visible image of the oral cavity when the toothbrush is in use. Thus, a camera (e.g., an RGB camera) is used to capture images or video sequences during regular brushing.
[0079] A toothbrush will be used in the following examples. However, other personal care devices (e.g., exfoliating skin brushes, hair brushes, etc.) can also be used using the same method described below. For example, an oral tissue cleaning / treatment unit having cleaning elements (e.g., bristles, silicone filaments, columns, etc.) can also be used.
[0080] Preferably, the personal care device has a cleaning element having a known and well-defined geometry and oscillation frequency.
[0081] The processor may be used to perform object identification of the cleaning elements and subsequent background subtraction. These steps may utilize the free oscillation frequency of the cleaning elements (eg, in an electric personal care device) and modified adaptive median background subtraction.
[0082] Figure 2 A method for providing feedback to a user of a toothbrush is shown. An image of the teeth needs to be obtained during brushing. This is performed by checking in step 202 whether the toothbrush is in operation, in step 204 checking whether the toothbrush has changed direction, and if so, obtaining / capturing an image from a camera on the toothbrush in step 206.
[0083] The central assumption is that the motion of the bristles of the toothbrush appears to be faster than the motion of other elements in the acquired image. It is important to consider the appropriate position of the camera. Since the goal of the image processing is to eliminate the bristles from the camera's field of view while vibrating, possible positions are anywhere in the toothbrush head in the same plane as the toothbrush head (e.g., Figure 1 shown).
[0084] In these solutions, the toothbrush bristles are moving relative to the teeth being imaged or recorded, for example through operation of an electric toothbrush.
[0085] A light source is used to illuminate the area where the image is being taken. Ideally this is a light emitting diode (LED). The advantage of an LED is that it can be easily triggered, facilitating subsequent images to be taken with the same bristle orientation.
[0086] In step 208, an algorithm is used to distinguish between moving objects and stationary objects in the obtained image. For this step, it may be advantageous to know and define the free vibration frequency of the bristles and / or the geometry of the bristles. This may improve the distinction between moving objects and stationary objects, since it is known that the bristles will be moving. The distinction between the moving and stationary parts of the image enables the generation of a background image that does not include the (moving) bristles.
[0087] The camera acquisition characteristics (eg, resolution in pixels, exposure time, frames acquired per second, acquisition time, etc.) are also preferably known.
[0088] The optical properties of the camera (readout focal length, field of view, etc.) are also preferably known.
[0089] In most cases, it will be possible to assume that the camera acquisition and / or optical characteristics are optimal for these conditions. However, the characteristics can be used to filter the images. For example, if the vibration frequency of the bristles is consistent with the frame rate of the camera, the bristles may appear to be stationary in the image sequence. Therefore, these images can be filtered out. In some cases, it is also possible that the frame rate (or other characteristics) can be adjusted based on the image sequence, or even the field of view of the camera can be adapted to the content of the image.
[0090] The position of the toothbrush in the mouth is then determined in step 210 (e.g., using a position sensor). Additionally or alternatively, the presence of caries / plaque in the obtained image may then be determined. In step 212, feedback is then provided, e.g., to the user, dentist, or used to adjust the toothbrush (e.g., adjusting toothbrush amplitude based on the feedback).
[0091] Figure 3 A method for providing a video stream without bristles is shown. A video stream is a sequence of images. Initially, in step 302, images from a camera are accumulated in a video stream. Then, in step 304, frame separation (i.e., video stream decomposition) is applied to the video stream.
[0092] The first background frame can be determined by finding the temporal median of the sequence of images obtained from the camera. This provides a background image with the bristles removed. This is possible because the bristles move much faster than the movement of the toothbrush. Therefore, the median pixel value over time will most likely be the background pixel value (i.e. of the mouth) because the faster moving bristles will only appear in the same position in the image for a short period of time.
[0093] Note that it is possible to have a background subtraction algorithm that consists simply of calculating, for each new frame, the per-pixel temporal median of the image. However, it may be advantageous to modify such an algorithm to provide a more robust background frame.
[0094] For example, frames from a video stream may be analyzed to iteratively update the background of a scene according to a modified background subtraction algorithm. In step 310, the current (most recently available) frame 306 and the previous background frame 308 are also input into the background subtraction algorithm to obtain an updated background frame. An updated background frame 312 at time / frame n is iteratively defined as follows:
[0095] Background n =(1-θ)·F n +θ·Background n-1
[0096] where F nis the current image / frame 306 from the camera (at the current time / iteration n), θ is the bristle threshold, and background_n-1 is the previous background frame 308. The current background frame 312 is then used in step 314 to update the previous background frame 308 for use in the next available frame. As previously discussed, the first background frame (e.g., at time=0) can be found by applying a median filter to the video stream.
[0097] The bristle threshold thus acts as a weighting mechanism between the previous background frame and the latest current frame. Typically, the bristle threshold is found based on detecting bristles in the image. Thus, updating the background image over time using the bristle threshold provides an improvement in the quality of the background frame compared to simply finding the median of the image sequence to generate the background frame. This is because the median-based background image is typically blurry, especially if the camera may be moving over time while the images are being acquired. Thus, adding the latest frame may improve the quality of the background frame.
[0098] As previously discussed, the background subtraction algorithm can be modified to include known properties of the bristles (free vibration frequency, shape, etc.). This is accomplished by defining a bristle threshold using information from the bristles (e.g., free vibration frequency, shape, color, etc.) and possibly camera information (e.g., acquisition frequency, resolution, etc.). The bristle threshold is used to distinguish vibrating bristles from other elements of the image. The method for determining and further updating the bristle threshold is further explained below.
[0099] Figure 4 A method for defining a bristle threshold is shown, and Figure 5 A diagram showing a method for defining a bristle threshold is shown. Figure 5 Graphical explanation of Figure 4 method.
[0100] The bristle threshold may be referred to as a bristle detection threshold.
[0101] According to the method for defining the bristle threshold, the known free vibration frequency f of the bristles is used. The free vibration frequency is the vibration of the bristles without interaction with the bristles (i.e. no pressure from the user, no contact with the surface, etc.). Typical values of the free vibration frequency can be about 300 Hz to 520 Hz.
[0102] The algorithm is used to detect bristles in frame 502 of a video captured from a toothbrush. The algorithm steps are as follows:
[0103] - using the geometric descriptors of the bristles in step 402 to detect feature points 504;
[0104] - performing optical flow calculation 506 on the feature points 504 in step 404;
[0105] - The fastest moving feature point 504 is identified in step 406. This is consistent with the assumption that the bristles are the fastest moving objects in the video.
[0106] - Temporal filtering and / or denoising may be performed on the signal obtained from the optical flow calculation 506 to clean the signal from random perturbations;
[0107] - In step 408, an independent component analysis algorithm may be used on the signal from the optical flow calculation 506 to separate the principal component signal 508;
[0108] - In step 410, a peak detection algorithm is applied to each of the principal components from step 408 to identify peaks 510. Peaks 510 may be used to define component frequencies (e.g., average time between peaks);
[0109] - in step 412, using the known free frequency of the bristles, identifying the bristles in the image based on the component signal 508 having the closest frequency matching (or within the range of) the known bristle frequency; and
[0110] - In step 414, morphological operations (such as erosion and dilation) are applied to the region bounded by the selected feature points to produce a bristle mask capable of segmenting the bristles from the image.
[0111] The bristle mask generated using this method is complementary to the background mask that can be generated by using the background frame. In particular, the background mask defines the area of the image corresponding to the background of the image, while the bristle mask defines the area of the image corresponding to the bristles.
[0112] Typically, a mask is a map of binary values, one of which indicates, for example, the possible presence of bristles in the image.
[0113] The background mask may be determined by finding the difference between the background frame and, for example, the most recently available frame in the sequence of frames.
[0114] Thus, the generated bristle mask can be used to update the bristle threshold so that the bristle threshold used returns a more accurate background mask compared to the bristle mask.
[0115] The bristle threshold may need to be recalculated over time. For example, if the toothbrush is moved to a new area of the mouth, the background of the image may change, and thus more recent frames may need to be weighted more heavily than previous background images.
[0116] Preferably, the bristle threshold is updated less often than the background image is determined. This is because updating the bristle threshold will generally require more processing resources than applying the bristle threshold to the previous background frame and the most recently available frame.
[0117] Figure 6An image is shown with the bristles segmented out. A first image 602 is the most recently available frame from a sequence of images (ie, a video stream). The previously mentioned binary mask may be applied to the image 602 to generate a second image 604 with the bristles segmented out.
[0118] Morphological operators of erosion and dilation are preferably applied to the regions of the image where the bristles have been identified to restore in the bristle mask the complete geometry of the other elements present in the current image / frame. The morphological operators are applied to complete the mask definition process. In other words, the morphological operators are designed to make the binary bristle mask more regular and closer to the actual shape of the bristles.
[0119] Figure 7 The average gray level of the oral frame is shown. The average gray level can be used as a measurement probe for the background image to check if there is a sudden noticeable deformation of the color component profiles of the image due to the background removal algorithm. The average gray level of the background image can be used to determine if the bristle threshold needs to be updated.
[0120] Other methods may also be used to determine or update the bristle threshold. For example, simulations may also be performed to determine the bristle threshold.
[0121] Figure 8 , Fig. 9 and Fig.10 The binary masks generated in simulation using thresholds of 0.8, 0.5 and 0.2 respectively are shown.
[0122] A skilled person will be able to easily develop a processor for executing any method described herein. Therefore, each step of the flowchart may represent a different action performed by the processor and may be performed by a corresponding module of the processor.
[0123] As discussed above, the system utilizes a processor to perform data processing. The processor can be implemented in a variety of ways using software and / or hardware to perform the various functions required. The processor typically employs one or more microprocessors that can be programmed using software (e.g., microcode) to perform the required functions. The processor can be implemented as a combination of dedicated hardware that performs some functions and one or more programmed microprocessors and associated circuits that perform other functions.
[0124] Examples of circuits that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).
[0125] In various implementations, the processor may be associated with one or more storage media such as volatile and non-volatile computer memory (such as RAM, PROM, EPROM, and EEPROM). The storage media may be encoded with one or more programs that perform the desired functions when executed on one or more processors and / or controllers. The various storage media may be fixed within the processor or controller, or may be transportable so that one or more programs stored thereon can be loaded into the processor.
[0126] By studying the drawings, the disclosure and the appended claims, those skilled in the art can understand and implement variations of the disclosed embodiments when practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the word "one" or "an" does not exclude a plurality.
[0127] The functions implemented by a processor may be implemented by a single processor or by multiple separate processing units, which may be collectively considered to constitute a “processor.” Such processing units may be remote from each other in some cases and communicate with each other in a wired or wireless manner.
[0128] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
[0129] The computer program may be stored / distributed on an appropriate medium, such as an optical storage medium or solid-state medium provided together with or as part of other hardware, but the computer program may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.
[0130] If the term "suitable for" is used in the claims or the description, it should be noted that the term "suitable for" is intended to be equivalent to the term "configured to". If the term "arranged" is used in the claims or the description, it should be noted that the term "arranged" is intended to be equivalent to the term "system" and vice versa.
[0131] Any reference signs in the claims should not be construed as limiting the scope.
Claims
1. An image processing system for processing an image received from an imaging device (104) mounted at a head of a personal care device (100), the image processing system comprising: A processor (110) configured to: receiving (302) a sequence of images from the imaging device at the head of the personal care device, the personal care device including a cleaning element visible in at least one image in the sequence of images; as well as The sequence of images is processed (312) to generate a background image with moving portions of the sequence of images removed such that the cleaning elements of the personal care device are removed from the background image. 2 . The system of claim 1 , wherein the processor is adapted to process the image sequence to generate the background image by applying a median filter to the image sequence.
3. The system of claim 1 or 2, wherein the processor is further configured to iteratively update (314) the background image using the most recently available image in the sequence of images. 4 . The system of claim 3 , wherein the processor is configured to iteratively update the background image by summing the background image with the most recently available image, wherein the most recently available image is weighted based on a cleaning element threshold.
5. The system of claim 4, wherein the processor is further configured to: detecting (402) features in the image sequence; calculating (404) an optical flow of the feature between images in the sequence of images; identifying (408) components in the image sequence using independent component analysis of the features; determining the frequency of the component using the corresponding optical flow; identifying (412) the cleaning element in the sequence of images based on the component having a frequency within a frequency range based on a known frequency of the cleaning element; generating (414) a cleaning element mask based on the identified cleaning elements; as well as The bristle threshold is updated by comparing the cleaning element mask to a background mask generated by determining differences between the background image and an image in the sequence of images.
6. The system of claim 5, wherein the processor is configured to generate the cleaning element mask by applying a morphological operation to the identified shapes of the cleaning elements relative to the images in the sequence of images.
7. The system of claim 5 or 6, wherein the processor is configured to use one or more of a known geometry of the cleaning elements and a known color of the cleaning elements to detect features in the sequence of images.
8. The system according to any one of claims 5 to 7, wherein the processor is further configured to: applying a high pass flow filter to the optical flow; and The optical flow is denoised.
9. An electric toothbrush (100), comprising: A brush head comprising bristles (102), an imaging device (104) and an illumination system (106); as well as An image processing system according to any one of claims 1 to 8, for processing the image captured by the imaging device.
10. A method for processing an image received from an imaging device mounted at a head of a personal care device, the method comprising: receiving (302) a sequence of images from the imaging device, the personal care device including a cleaning element visible in at least one image in the sequence of images; as well as The sequence of images is processed (304) to generate a background image with moving portions of the sequence of images removed, such that the cleaning elements of the personal care device are removed from the background image.
11. The method of claim 10, wherein processing the image sequence to generate the background image comprises: A median filter is applied to the image sequence.
12. The method according to claim 10 or 11, further comprising: The background image is iteratively updated (314) using the most recently available image in the sequence of images.
13. The method of claim 12, wherein iteratively updating the background image comprises: The background image is summed with the most recently available image, wherein the most recently available image is weighted based on a cleaning element threshold.
14. The method according to claim 13, further comprising: detecting (402) features in the image sequence; calculating (404) an optical flow of the feature between images in the sequence of images; identifying (408) components in the image sequence using independent component analysis of the features; determining the frequency of the component using the corresponding optical flow; identifying (412) the cleaning element in the sequence of images based on the component having a frequency within a frequency range based on a known frequency of the cleaning element; as well as generating (414) a cleaning element mask based on the identified cleaning elements; as well as The bristle threshold is updated by comparing the cleaning element mask to a background mask generated by determining differences between the background image and an image in the sequence of images.
15. A computer program comprising computer program code adapted to implement the method according to any one of claims 10 to 14 when said program is run on a computer.
Citation Information
Patent Citations
Dental Care Device For Detection And Removal Of Plaque
US20150107034A1