Personal Care Devices

Abstract

Schemes, solutions, concepts, designs, methods, and systems are proposed for assisting and / or improving image acquisition from a vibrating personal care device with a camera. The embodiments propose to map at least one region of the acquired image to a certain pose of the vibrating part. Using the pose of the vibrating part, a pixel shift for the at least one region can be calculated, and the pixel data is then shifted to construct an improved image. Thus, by correcting for the vibration motion, an improved image (e.g., an image exhibiting less distortion) can be constructed / generated.

Description

[Technical field]

[0001] The present invention relates to the field of personal care devices, and in particular to the field of personal care devices that capture a digital impression of one or more characteristics of a user. [Background technology]

[0002] It is known to provide personal care devices with the capability or functionality to capture digital impressions (e.g., images) of a user's features or parts during use. For example, intraoral scanner (IOS) devices capture images of dentogingival tissue using projected light (e.g., laser or structured light) and an image sensor. These images can be processed to create a three-dimensional (3D) model of the scanned surface. Thus, data provided by the IOS can be useful for oral care, including detection of common dental conditions such as caries, tooth discolouration, misalignment of teeth, etc. It is also advantageous to capture and compare IOS images repeatedly, for example to enable identification of changes in dentogingival tissue over time. Summary of the Invention [Problem to be solved by the invention]

[0003] Personal care devices such as powered brush devices are used in the oral cavity on a regular basis (e.g., daily), providing a suitable vehicle for regularly acquiring images of a portion of the user. Thus, there is a trend to integrate cameras or imaging sensors into personal care devices, such as toothbrushes. However, because a primary usage characteristic of such personal care devices is their vibration or cyclical / periodic motion, images acquired by cameras integrated into personal care devices are typically distorted and / or blurred due to the motion / vibration of the device.

[0004] While image stabilization techniques are known for portable handheld devices (e.g., smartphones, digital photo cameras, etc.), such techniques are designed and optimized only for low frequency (e.g., less than 10 Hz), irregular (e.g., non-periodic, random, etc.) user-generated motion. Because personal care devices (e.g., electric toothbrushes) typically move or vibrate at higher periodic frequencies (e.g., greater than 20-300 Hz), a need remains to stabilize the image acquisition process and / or make the image acquisition process robust to device motion / vibration generated by personal care devices. [Means for solving the problem]

[0005] The invention is defined by the claims.

[0006] According to an example of an aspect of the present invention, there is provided a method for processing images acquired from a personal care device, said personal care device comprising vibration means (12) adapted to vibrate a vibration part of the personal care device in use to vibrate the personal care device, and a camera adapted to acquire images of one or more features of a user in use, the acquired images being affected by the vibration of the personal care device, said method comprising: - mapping at least one region of the acquired image to a pose of the vibration part, said pose describing at least one of a location of the vibration part and an orientation of the vibration part; calculating a pixel shift for the at least one region based on the orientation of the vibration unit; and shifting pixel data of the acquired image based on the calculated pixel shift for the at least one region to generate a corrected image.

[0007] The proposed concept therefore aims to provide a scheme, solution, concept, design, method and system for assisting and / or improving image acquisition from a vibrating personal care device with a camera configured to acquire images affected by device vibrations. This may be a camera rigidly fixed to the vibrating device and then vibrating with the device. Alternatively, the camera sensor itself is located in a stationary part of the device (e.g. the brush handle), while the optical system of the device (optical fiber, imaging lens of the optical system, etc.) is subject to the device vibrations. In detail, an embodiment of the invention proposes to determine the attitude (i.e. relative location and / or orientation) of the vibrating part at the time when the image or part of the image is acquired. Then, the pixel shift caused by the difference in the attitude of the vibrating part, e.g. with respect to the attitude or orientation relative to a reference point / axis, can be calculated and the pixel data is shifted / moved accordingly. Thus, by shifting / adjusting the pixel data based on the attitude of the vibrating part, an improved image (e.g. an image exhibiting less wobble) is constructed / generated. Thereby, an improved image is obtained from the vibrating personal care device.

[0008] The pose of the vibrating part describes at least one of its location and orientation, such that when the pose is known, the location and / or orientation of the vibrating part is known, the description being relative to (i.e. defined with respect to) a reference configuration, such as a rest position and orientation.

[0009] In particular, to reduce or minimize fluctuations caused by acquisition of data at different times during the vibration cycle, the image data is adapted based on the time during the vibration cycle at which the data is acquired, e.g., the image data is shifted based on the time during the vibration cycle, thus ensuring that the image data is shifted and repositioned to generate a reconstructed image that has less fluctuation due to the vibration motion.

[0010] It is recognized that the pose (including location and / or orientation) of the vibration part is identified, and thereby the direction of the optical path of the camera is also known. Thus, the pose at which the image data is acquired is determined, and this pose can be used to ensure that the image data is assigned to the correct area of ​​the image, thus reducing the influence of data acquired at different times of the vibration cycle. This pose can then be used to construct a higher quality image (in terms of reduced distortion), thus facilitating improved oral / dental care decision-making. Thus, by correcting for the vibration motion, improved images (e.g. images exhibiting less distortion) can be constructed / generated.

[0011] In other words, the embodiments propose to adjust pixel data positions for a region based on the specific placement / configuration of the vibrating portion when data for that region is acquired.

[0012] By adjusting the pixel data position and reducing / eliminating the effect of different orientations of the vibration unit, improved quality images are generated. Thus, the embodiments facilitate obtaining stable, high quality images while brushing the teeth. Such embodiments are particularly relevant to tele-dentistry proposals, for example by enabling improved imaging of the user's teeth, gums, tongue, etc. For example, images obtained by the proposed embodiments aid in dental care procedures and / or decision making. Thus, the embodiments are used in connection with dental treatment selection, to assist dental care professionals in selecting treatments for subjects.

[0013] Additionally, embodiments facilitate image-based diagnosis, superior location sensing, treatment planning (eg, orthodontics, aligners), and / or dental treatment monitoring.

[0014] By being integrated into the user's regular brushing schedule (instead of using separate devices such as a smartphone, a dental endoscope, or a handheld intraoral scanner), the embodiments help improve dental care. Thus, improved image-based dental care is brought about by the proposed concept.

[0015] However, the embodiments are applicable to other vibratory personal care devices, such as shavers, skin cleaning devices, etc. Thus, the embodiments provide the advantage that decision-making in selecting a care treatment can be improved by using images acquired by the vibratory personal care device. For example, the embodiments can make available a number of oral care treatment options (e.g., by providing more and / or improved information regarding the subject's oral health).

[0016] In some embodiments, the mapping step comprises mapping each of a plurality of distinct regions of the acquired image to a respective orientation of the vibration part, such that each distinct region of the acquired image is mapped to a certain orientation of the vibration part and then pixel data for each region is shifted depending on the orientation of the vibration part when the data is acquired.

[0017] In some embodiments, the calculating step includes calculating a pixel shift for each region based on the orientation of the vibration unit. Thus, an appropriate pixel shift for each region can be calculated such that the pixel data for each region is shifted by a correct amount. The pixel shift for different regions is different, but the pixel shift for some regions may be the same.

[0018] In some embodiments, the shifting step comprises shifting the pixel data for each region by a calculated pixel shift for each region to generate the modified image, again ensuring that different regions can be corrected by different amounts.

[0019] By mapping each region, calculating the pixel shift for each region, and shifting each region, displacements caused by movement (and different orientations) of the personal care device can be corrected to produce a reconstructed image with less distortion and fluctuation.

[0020] In some embodiments, the pixel shift is proportional to the distance d between the vibrating part and the imaging plane of the camera. The angle of the orientation of the vibrating part from the central axis is Θ vib and the pixel shift is given by the distance d and the angle Θ vib Based on both, preferably d.tanΘ as pixel shift distance vib It is calculated by:

[0021] As a further example, an embodiment defines a region of minimum pixel deviation within the acquired image, where calculating the pixel deviation for the at least one region is further based on a distance of the at least one region from the region of minimum pixel deviation.

[0022] In one embodiment, the pixel shift for at least one region is determined by the rotation angle Θ of the oscillation. rot Based further on.

[0023] That is, some personal care devices rotate from a central axis, and the angle of the orientation of the vibration part from the central axis is Θ vib In some embodiments, the pixel shift is then given by d.tan Θ vib , or distance d and angle Θ vib and the distance between the vibrator and the camera's imaging plane is given by d, giving a pixel shift distance. This allows the pixel shift to be calculated for a region of the image.

[0024] In other personal care devices, the vibration unit rotates and vibrates around the z-axis, and the angle of the vibration unit's orientation from the central rotation orientation is Θ rot In this example, the rotational displacement depends on the distance r from the axis of rotation. The pixel shift distance d in the x direction x is r.sinΘ rot .sinΘ rot The pixel shift distance in the y direction is calculated using y is r.sinΘ rot .cosΘ rotwhere d is the distance between the vibrating part and the imaging plane of the camera. These provide an accurate calculation of the pixel shift required to correct for the rotational motion of the optical path of the camera.

[0025] By way of example, the step of mapping each region of the acquired image to a certain orientation of the vibrating part comprises determining the orientation of the vibrating part when the image of the respective region was acquired.

[0026] In some embodiments, the step of calculating the pixel shift for each region uses the pixel density of the captured image (or the sensor of the camera) to calculate the pixel shift for each region. This can be used to determine the number of pixels by which the pixel data must be shifted to generate a corrected image. Here, reference to a pixel includes a single sensing element of an image sensor, or a sensing element (single sensor) of an image sensor array. That is, a pixel is considered to be a single element of a sensed image, and the sensed image is formed from multiple sensed elements (i.e. pixels).

[0027] A region is assigned according to when a portion of an image is acquired, for example, a region is an area where image data is acquired at a particular time or within a particular period of time.

[0028] According to yet another aspect of the invention there is provided a computer program comprising computer program code means adapted to perform the method according to the proposed embodiments, when said computer program is run on a computer.

[0029] According to another aspect of the invention, there is provided a system for processing images acquired from a personal care device, said personal care device comprising vibration means (12) adapted to vibrate a vibration part of the personal care device in use to vibrate the personal care device, and a camera adapted to acquire images of one or more features of a user in use, the acquired images being affected by the vibration of the personal care device. The system comprises: 1. A processor device configured to process images acquired from a camera using an image processing algorithm, the image processing algorithm comprising: Mapping at least one region of the acquired image to a certain orientation of the vibration part; calculating a pixel shift for the at least one region based on the orientation of the vibration unit; and shifting pixel data of the acquired image based on the calculated pixel shift for the at least one region to generate a corrected image.

[0030] Thus, the pixel data can be shifted to compensate for the orientation of the vibrating part, resulting in a less distorted image.

[0031] Such a system may be provided as a stand-alone system for use with one or more personal care devices. Thus, a system according to one embodiment may be employed with conventional vibratory personal care devices to provide improved and / or expanded functionality.

[0032] According to one embodiment, there is provided a personal care device comprising: a vibration means (12) adapted to vibrate the vibration part of the personal care device in use to vibrate the personal care device; a camera adapted to capture, in use, an image of one or more features of a user, the captured image being affected by vibrations of the personal care device; and a system for processing the acquired images according to the proposed embodiments. Such a personal care device allows images with reduced distortion to be obtained, thereby allowing the quality of oral care to be improved.

[0033] In some embodiments, the camera is located on the vibrating part. When the camera is located on the vibrating part, the optical path of the camera is changed as well as the attitude, i.e., location or orientation, of the vibrating part. In other embodiments, the camera is located on another non-vibrating part of the personal care device, but the optical path passes through the vibrating part.

[0034] The personal care device includes a toothbrush, preferably an electric toothbrush adapted to vibrate when in use. In other embodiments, the personal care device includes a mouthpiece, a shaver, or a skin cleaning device adapted to vibrate when in use. Thus, one or more of the proposed concepts may be employed within a range of personal care devices. Thus, the embodiments have broad application in the field of personal care devices (as well as image acquisition and / or processing concepts for images acquired by vibrating personal care devices).

[0035] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0036] For a better understanding of the present invention, and to show more clearly how the same may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which: [Brief description of the drawings]

[0037] [Figure 1] FIG. 1 is a simplified schematic block diagram of an electric toothbrush according to a proposed embodiment. [Diagram 2] FIG. 1 is a simplified block diagram of a portion of an electric toothbrush illustrating different vibration modes. [Diagram 3] 2 is a graph showing an exemplary variation over time of the (displacement) angle Θ and angular velocity of the brush head of FIG. 1 as the brush head vibrates, in which the variation in the brush head displacement angle Θ is depicted by a solid line and the variation in the brush head angular velocity v is depicted by a dashed line. [Figure 4A] FIG. 1 depicts a calculation of displacement for a first vibration mode. [Figure 4B] FIG. 13 depicts a calculation of displacement for a second vibration mode. [Diagram 5] FIG. 1 is a simplified schematic block diagram of a mouthpiece according to a proposed embodiment. [Figure 6A] FIG. 1 illustrates an unmodified image. [Figure 6B] FIG. 13 illustrates the results of a simulation of image frames acquired with a camera excited by a square wave and vibrating at 30 frames per second to vary the camera exposure time, vibration frequency, and vibration period. [Figure 7A] FIG. 2 illustrates the extraction of image data from an acquired image according to the proposed embodiment. [Figure 7B] 7B depicts the use of extracted image data from FIG. 7A to construct an improved (eg, higher quality) image according to one embodiment. [Figure 8] 1 is a flow diagram of a method for processing images acquired from a personal care device according to a proposed embodiment. [Figure 9] FIG. 1 is a simplified block diagram of a computer in which one or more components of an embodiment are employed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] The present invention will now be described with reference to the figures.

[0039] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the devices, systems and methods, are intended for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects and advantages of the devices, systems and methods of the present invention will become better understood from the following description, the appended claims and the accompanying drawings. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0040] Variations of the disclosed embodiments may be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and singular elements do not exclude a plurality.

[0041] It should be understood that the drawings are only schematic and are not drawn to scale, and that the same reference numerals are used throughout the drawings to denote the same or similar parts.

[0042] The present invention proposes a concept for aiding and / or improving image acquisition from a vibrating personal care device having a camera where the camera image vibrates with the device such that the image acquired by the camera is affected by the vibration of the personal care device. In particular, embodiments provide a system, device, and / or method for identifying image data acquired when the vibrating part is not in a specific orientation during a vibration cycle. The vibrating part may be measured relative to a central target orientation (e.g., a reference location and / or a reference orientation) or other orientation. Image data acquisition when the vibrating part is not in a specific orientation is identified and the displacement from the specific target orientation is calculated. The pixel data can then be moved (e.g., shifted or translated) to correct for deviations from the target position. For example, the pixel data can then be moved to a specific target position. Such control of image data acquisition results in improved images (e.g., images exhibiting reduced distortion and shimmer).

[0043] With reference to Figure 1, there is shown a simplified schematic block diagram of an electric toothbrush 10 according to a proposed embodiment. The electric toothbrush 10 comprises vibration means 12 (specifically a motor) adapted to vibrate a brush head 14 of the electric toothbrush 10 in use (at a frequency of about 200Hz).

[0044] The electric toothbrush 10 also includes a camera 16 (e.g., a digital camera) adapted to capture images of one or more oral features of a user during use. Furthermore, Figure 1 shows the distance d between the vibration axis of the vibrating part and the imaging plane of the camera.

[0045] More specifically, the motor 12 is configured to repeatedly oscillate the brush head 14. The oscillation may be either a rotation about a y-axis (as depicted in FIG. 1) as shown in FIG. 2A, or a rotation about a central z-axis as shown in FIG. 2B. The oscillation periodically rotates clockwise and then counterclockwise from a rest position approximately every 10°-25°. This causes the brush head 14 to oscillate with a periodic vibration waveform. By way of example, the rotation is at a frequency of 200 Hz. As a further example, the maximum angular rotation is 15°.

[0046] By way of example only, the brush head 14 periodically rotates clockwise and then counterclockwise about every 10°-25° from the rest position. However, other embodiments employ brush heads that oscillate in different ways. For example, alternative embodiments include a brush head that oscillates repeatedly (side to side or up and down), i.e., is repeatedly displaced in opposite directions (laterally or vertically) from the rest position.

[0047] 2C depicts a graph showing exemplary variations in the angle (of displacement) Θ and angular velocity of the brush head 14 over time as the brush head 14 oscillates. The variations in the displacement angle Θ of the brush head 14 are depicted by the solid lines, while the variations in the angular velocity v of the brush head 14 are depicted by the dashed lines.

[0048] As can be seen from the variation in Fig. 2C, there are different angular displacements Θ at different times. If the optical path of the camera is changed as well as the brush head, the position of the pixel data also changes. The present invention is based on the idea that the displacement due to camera vibration can be calculated and compensated for.

[0049] Referring again to the embodiment of FIG. 1, the electric toothbrush 10 comprises a system 18 for processing the acquired images. The system comprises a processor device 18A configured to process the images acquired from the camera using an image processing algorithm for mapping at least one region of the acquired image to a certain pose of the vibration part by determining the pose of the vibration part when the image of the respective region was acquired. The system also comprises an image processor 18B for calculating a pixel shift for the at least one region based on the pose of the vibration part. Finally, the system comprises an image processor 18C for shifting the pixel data to generate a corrected image. It should be noted that, although all three processors are here configured on the toothbrush, in other embodiments one or more of the processors may be remote processors connected to the toothbrush, for example by wireless connection.

[0050] Specifically, the processor unit 18A obtains the time at which an area of ​​an image is acquired and maps this time to the pose of the brush head 14 at the time the image of that area is acquired. The oscillatory motion of the brush head 14 is known and can therefore be easily mapped to the time at which the image of the area is acquired.

[0051] Using the example of the oscillatory motion depicted in Figure 2A, the angle varies during oscillation according to the curve depicted in Figure 2C. However, if the time is known, the angle Θ from the central axis of the brush head at the time the image of the area is taken can be calculated by the angle Θ from the central axis of the brush head at the time the image of the area is taken. vib It is known that.

[0052] Similarly, using the example of oscillatory motion depicted in FIG. 2B, the angle of rotation is varied, but at a given time of image acquisition, the rotation angle Θ rot may be required.

[0053] In the example depicted in Figure 3, the different regions are formed by a group of rows of pixels. As can be seen, the different regions can be easily mapped to vibration patterns by comparing the deviations to the vibration waveform.

[0054] The image processor 18B is then adapted to calculate the pixel shift for the region based on the brush head's pose. An example of this for the oscillatory motion of Figure 2A is depicted in Figure 4A. The angle Θ at which the image of the region was captured. vib is known, and the distance between the brush head and the imaging surface or object is d, which in this example is assumed to be the length of the brush hair. Figure 4A illustrates how the displacement can be calculated. Thus, the displacement is Displacement = d.tan Θ vib is given by:

[0055] Once the displacement is known, the pixel density can be used to determine how much the pixel data for the region should be shifted or moved within the image.

[0056] An example image frame is 620 columns and 480 rows of pixels capturing an area of ​​10 mm by 10 mm. If the displacement due to the vibration motion at the time of image capture (calculated above) is 2 mm, the pixel data will be shifted by 20% of the image. If the vibration is in the column direction, a shift of 124 pixels (20% of 620) will result. If the vibration is in the row direction, a shift of 96 pixels (20% of 480) will result. If the vibration makes an angle β with the column direction, the shift will be a vector of length 2 mm containing 96.sinβ rows and 124.cosβ columns.

[0057] The image shift is the minimum pixel shift r and the rotation angle Θ of the vibration. rot , as a function of the distance of the feature in the image from the region of the image. Now, an example of the further step of converting the pixel displacement from the rotating coordinate system to the Cartesian system of the camera image will be described. In this example, the angle Θ for the rotational motion depicted in FIG. 2B is rot The rotation of the brush head is calculated by the rotation angle Θ rot, resulting in an orientation shift that depends on the pixel's distance from the image center r, where the pixel shift is minimum. Here, the rotation angle Θ rot The part of the image along the image horizon, which is at the height of the center of the image that is rotated by d, is taken into account. As shown in FIG. 4B, the displacement d in the x direction x and the displacement in the y direction, d y is expressed by the following equation: d x =r.sinΘ rot .sinΘ rot d y =r.sinΘ rot .cosΘ rot It can be calculated by:

[0058] Once the displacement is known, it can be translated into pixels in the image in a similar manner as above.

[0059] Although two different vibration motions are described in connection with Figures 2A-2B and 4A-4B, other vibration motions may also be used and the displacements may be calculated using known techniques. For example, Figures 2A-4A relate to rotations about the y-axis, but rotations about the x-axis may exist as well. Furthermore, combinations of vibration motions may exist. For example, combinations of the vibration motions of Figures 2A-2B may exist.

[0060] Image processor 18C then shifts or translates the pixel data to produce a modified image. In one embodiment, the pixel data is shifted by the distance calculated by processor 18B.

[0061] The present invention uses the known geometry and vibration motion of the camera to correct for different orientations of the camera during image acquisition.

[0062] In some embodiments, multiple regions of the image are processed by processors 18A, 18B, and 18C so that for each of the multiple regions, the pose of the vibrator is mapped, a pixel shift is calculated, and the pixels are shifted by the calculated distance. In some embodiments, each region of the image is processed by processors 18A, 18B, and 18C so that for each region, the pose of the vibrator is mapped, a pixel shift is calculated, and the pixels are shifted by the calculated distance.

[0063] To assist in reducing the exposure period required to obtain a sufficiently exposed image, the camera 16 of the embodiment of Figure 1 includes an illumination device 19 (e.g., an LED) configured to illuminate a portion of a user's oral cavity during use. That is, where the sensitivity (e.g., ISO) and / or lens aperture of the camera 16 is fixed and / or limited, a reduction in the length of exposure required to obtain a sufficient amount of light can be obtained by illuminating the user's oral cavity with the illumination device 19.

[0064] Also, while the embodiment of Figure 1 employs the concept of shifting the pixel data of the region by an amount calculated by processor 18B to return it to a center position, other embodiments may calculate the displacement to a different position, for example, a displacement to another fixed position, such as an antinode of the vibration.

[0065] Additionally, the above embodiments shift pixel data in the region for which the displacement is calculated. Alternatively, pixel data from other regions may be shifted to correct for the displacement.

[0066] 5, there is shown a simplified schematic block diagram of a mouthpiece 40 according to another embodiment. The mouthpiece 40 is adapted to be inserted into a user's mouth and to vibrate (in use) for oral cleansing purposes.

[0067] The mouthpiece 40 is provided with vibration means 42 (in particular an electric motor) adapted to vibrate the mouthpiece, in use, with a periodic vibration waveform (having a frequency in the range 100Hz to 500Hz).

[0068] The mouthpiece 40 also comprises a camera 44 arranged in a mouthpiece tray and adapted to capture images of one or more oral features of a user in use. A flash LED 46 is further provided in the mouthpiece tray to illuminate the user's oral cavity in use. The camera 44 and LED 46 are configured to be controlled by a control unit 48 (i.e., a controller) of the mouthpiece.

[0069] A system 49 for processing images acquired from the camera 44 is integrated into the control unit 48. The system 49 is configured to map at least one region of the acquired image to a certain pose of the camera. To assist the system 50, the mouthpiece 40 further comprises a sensor device 50 adapted to sense operating parameters of the mouthpiece 40. In particular, in this example the sensor device 50 comprises an accelerometer and a gyroscope for sensing variations in the displacement and velocity of the mouthpiece 40. Information about the sensed variations in the displacement and velocity of the mouthpiece 40 is provided to the system 50, and based on this information the system 49 maps a region of the acquired image to the pose of the camera or to the optical path of the camera.

[0070] As already described above with reference to the embodiment of Figure 1, the system 49 also calculates a pixel shift for said at least one region based on the pose of the vibrating part and shifts the pixel data to generate a rectified image. Advantageously, this process is repeated multiple times, or for all regions of the image.

[0071] It should be noted that the cameras in the above embodiments include rolling shutter cameras configured to operate at the image capture frame rate. For such rolling shutter cameras, tradeoffs in performance characteristics are required. For example, rolling shutter cameras are more cost effective (e.g., cheaper) but take longer to capture a complete image depending on acquisition settings and / or performance.

[0072] For rolling shutter cameras, the shutter and capture is typically line by line, so different lines will have different displacements due to the vibration motion. Each line or region is displaced by a different amount, resulting in a distorted image due to the "wobble" effect. An example of such an image is shown in FIG. 6A. Referring to FIG. 6B, the results of a simulation of captured image frames with a camera vibrating at 30 frames per second to vary the camera's exposure time, vibration frequency, and vibration period are shown. Thus, an embodiment employs a line scanner to place captured lines in the image location by location.

[0073] It should therefore be understood that the rolling shutter camera can be free-running and not synchronized with the vibration (and thus image data acquisition frequency) of the personal care device, where a complete image is reconstructed from image data obtained from multiple different frames of the rolling shutter camera.

[0074] A similar principle is also employed in high speed global shutter cameras where individual images are shifted rather than lines being shifted and can be corrected as a whole for motion. For example, if a first image is taken on the left side and a second image on the right side, the pixel shift between both can be calculated and these two images can then be corrected for location and added together to increase the signal to noise ratio (i.e. a wide angle image created by using a precise portion of each image), as can be done line by line in a rolling shutter camera implementation.

[0075] For example, Figure 7A illustrates mitigating distortion according to the proposed embodiment, showing first, second and third images 51, 52, 53 acquired successively from a rolling shutter camera 16 (i.e., first through third consecutive frames from the rolling shutter camera).

[0076] 7A, a first region 551 of the image 51 is acquired when the brush head is displaced from the central axis for a first captured image 51. Also, a second region 552 of the second image 51 is acquired a second time when the brush head is at its central axis. Finally, a third region 553 of a third captured image 53 is acquired a third time when the brush head is in a third pose.

[0077] The brush head pose for each of the first, second, and third passes is determined by processor 18A, and the displacement is calculated by processor 18B for each of regions 551, 552, and 553. As depicted in Figure 7B, the pixel data from each of the regions may be shifted to a central position to reduce distortion and construct an improved image.

[0078] It has been recognised that the cause of the wobble is related to the precise angle of the camera (or more specifically the camera's focus lens) relative to the teeth, which can result in a wobble image. Examples of scenarios that can result in a wobble image include: a) A free-running rolling shutter camera in constant, i.e. non-pulsed illumination. b) With pulsed illumination, the phase of the light pulse during the oscillation changes from image to image. With a rolling shutter camera, some lines of the image are acquired while the camera is pointed in one direction, while other lines are acquired while the camera is pointed in the other direction. A wobbling image is created as the camera passes over the lines of the image frame during the oscillation motion. c) For short shutter times, the phase of the shutter release during oscillation varies from image to image. With a rolling shutter camera, some lines of the image are acquired while the camera is pointed in one direction, while other lines are acquired while the camera is pointed in the other direction. As the oscillatory motion moves the camera, it passes through the lines of the image frame, creating a jittery image. d) When multiple lines or areas are acquired in one frame or image, the lines are offset by "pixels" depending on the instant / stage of line acquisition and their respective speeds. e) In a rolling shutter camera, when multiple images are taken during the vibration, the images are merged. f) In a global shutter camera where the entire image is acquired simultaneously, the entire image is displaced by the oscillatory motion.

[0079] It should therefore be understood that the present invention can be applied to the above situations and scenarios. After correction of the displaced regions, an image of improved quality can result.

[0080] It should therefore be appreciated that embodiments are configured to determine the orientation of the vibrating portion and then correct the image data for variations from a particular point. The image data may be corrected to a center point or to another particular point. By adjusting or correcting the image data to a single point in the vibration cycle, distortion due to vibration may be reduced or eliminated.

[0081] By removing the distortion, an improved image of the subject can be obtained, and therefore improved oral care can be provided.

[0082] In the example of Fig. 1, the camera 16 is located in the brush head 14. However, it may equally be located elsewhere, for example in the body of an electric toothbrush. However, the optical path passes through the brush head 14 and is therefore subject to the same vibrations and movements as the brush head. The resulting image is therefore also subject to the same types of distortions. The invention is applicable to personal care devices in which the focus lens of the camera is located in the brush head or in the vibrating part. Thus, the invention may be applied to personal care devices in which the camera is located away from the brush head but the optical path of the camera passes through the brush head.

[0083] By way of example, referring now to FIG. 8, a flow diagram of a method 600 for processing images acquired from a personal care device according to a proposed embodiment is depicted.

[0084] For this exemplary embodiment, the personal care device comprises a vibrating portion and a camera adapted to capture images of one or more oral features of a user in use, the camera having an optical path passing through the vibrating portion, each image having one or more regions.

[0085] The method 600 begins with initialization 610 of a reconstructed image. For example, the reconstructed image is initialized with the first raw image acquired from a camera.

[0086] Next, in step 620, the captured image from the camera image is processed by an image processing algorithm to map at least one region of the captured image to a certain pose of the vibrating part. Such a region may be as small as a single pixel. Specifically, the image processing algorithm compares the pose of the vibrating part when the image of the region was captured.

[0087] Then, in step 630, the pixel shift for the region is calculated.

[0088] In step 640, the pixel data is shifted. In one embodiment, the pixel data from the region is shifted by the amount calculated in step 630. A modified image is then generated using the shifted pixel data. As indicated by the arrow labeled "650," the process of mapping the regions may be repeated for each region of the image.

[0089] From the above description of various concepts and embodiments, it should be understood that a method is proposed for processing images acquired from a personal care device, such method being employed in a processing system or computer, such system / computer being integrated with a vibrating personal care device.

[0090] 9 illustrates an example of a computer 70 in which one or more components of an embodiment are employed. The various operations described above utilize the capabilities of the computer 70. For example, one or more portions of the system for providing a subject-specific user interface may be incorporated into any element, module, application, and / or component described herein. In this regard, it should be understood that the system functional blocks may be executed on a single computer or distributed across several computers and locations (e.g., connected through the Internet), such as a cloud-based computing infrastructure.

[0091] The computer 70 includes, but is not limited to, a PC, a workstation, a laptop, a PDA, a Palm device, a server, storage, and the like. In general, with respect to the hardware architecture, the computer 70 comprises one or more processors 71, a memory 72, and one or more I / O devices 73 communicatively coupled via a local interface (not shown). The local interface may be, for example, but is not limited to, one or more buses or other wired or wireless connections as known in the art. The local interface comprises additional elements to enable communication, such as controllers, buffers (caches), drivers, repeaters, and receivers. Additionally, the local interface includes address, control, and / or data connections to enable appropriate communication between the aforementioned components.

[0092] Processor 71 is a hardware device for executing software that may be stored in memory 72. Processor 71 may be virtually any custom or commercially available processor, central processing unit (CPU), digital signal processor (DSP), or coprocessor among several processors associated with computer 70, and processor 71 may be a semiconductor-based microprocessor (in the form of a microchip) or a microprocessor.

[0093] The memory 72 may comprise any one or combination of volatile memory elements (e.g., random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM), etc.) and non-volatile memory elements (e.g., ROM, erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), tape, compact disk read only memory (CD-ROM), disk, diskette, cartridge, cassette, etc.). Furthermore, the memory 72 incorporates electronic, magnetic, optical, and / or other types of storage media. It should be noted that the memory 72 may comprise a distributed architecture, in which various components are located remotely from one another but may be accessed by the processor 71.

[0094] The software in memory 72 includes one or more separate programs, each of which includes an ordered list of executable instructions for implementing logical functions. The software in memory 72 includes a suitable operating system (O / S) 74, a compiler 76, source code 75, and one or more applications 77, according to an exemplary embodiment. As shown, application 77 includes a number of functional components for implementing the features and operations of the exemplary embodiments. The application 77 of computer 70 represents various applications, computing units, logic, functional units, processes, operations, virtual entities, and / or modules, according to an exemplary embodiment, but application 77 is not intended to be limiting.

[0095] Operating system 74 controls the execution of other computer programs and provides scheduling, input / output control, file and data management, memory management, and communication control and related services. It is contemplated by the inventors that application 77 for implementing the exemplary embodiment is applicable to all commercially available operating systems.

[0096] The application 77 may be a source program, an executable program (object code), a script, or any other entity that includes a set of instructions to be executed. If it is a source program, it is typically translated through a compiler (such as compiler 76), an assembler, an interpreter, etc., which may or may not be present in memory 72, so that it operates properly in conjunction with the O / S 74. Additionally, the application 77 may be written in an object-oriented programming language having classes of data and methods, or a procedural programming language having routines, subroutines, and / or functions, such as, but not limited to, C, C++, C#, Pascal, BASIC, API calls, HTML, XHTML, XML, ASP scripts, JavaScript, FORTRAN, COBOL, Perl, Java, ADA, .NET, etc.

[0097] The I / O devices 73 include input devices such as, but not limited to, a mouse, keyboard, scanner, microphone, camera, etc. Additionally, the I / O devices 73 also include output devices such as, but not limited to, a printer, display, etc. Finally, the I / O devices 73 further include devices that communicate with both input and output such as, but not limited to, a NIC or modulator / demodulator (for accessing remote devices, other files, devices, systems, or networks), radio frequency (RF) or other transceivers, telephone interfaces, bridges, routers, etc. The I / O devices 73 also include components for communicating across various networks such as the Internet or an intranet.

[0098] If computer 70 is a PC, workstation, intelligent device, etc., the software in memory 72 further includes a Basic Input / Output System (BIOS) (omitted for simplicity). The BIOS is a set of essential software routines that initializes and tests the hardware at startup, starts O / S 74, and assists in transferring data between hardware devices. The BIOS may be stored in some type of read-only memory, such as a ROM, PROM, EPROM, EEPROM, etc., so that the BIOS may be executed when computer 70 is booted.

[0099] When computer 70 is operating, processor 71 is configured to execute software stored in memory 72, for communicating data to and from memory 72, and for generally controlling the operation of computer 70 and the software accordingly. Applications 77 and O / S 74 are read, in whole or in part, by processor 71, possibly buffered within processor 71, and then executed.

[0100] It should be noted that when the application 77 is implemented in software, the application 77 may be stored on virtually any computer-readable medium for use by or in connection with any computer-related system or method. In the context of this specification, a computer-readable medium is an electronic, magnetic, optical, or other physical device or means that can contain or store a computer program for use by or in connection with a computer-related system or method.

[0101] The application 77 may be embodied in any computer readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other system capable of retrieving and executing instructions from an instruction execution system, apparatus, or device. In the context of this specification, a "computer readable medium" may be any means that can store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.

[0102] The proposed image acquisition and / or processing method may be implemented in hardware, software, or a combination of both (e.g. as firmware executed on a hardware device). To the extent that the embodiment is partially or wholly implemented in software, the functional steps illustrated in the process flow charts are performed by suitably programmed physical computing devices, such as one or more central processing units (CPUs) or graphic processing units (GPUs). Each process, and its individual component steps as illustrated in the flow charts, may be performed by the same or different computing devices. According to an embodiment, a computer-readable storage medium stores a computer program having computer program code configured to cause one or more physical computing devices to perform the encoding or decoding method as described above when the program is executed on the one or more physical computing devices.

[0103] The storage medium includes volatile and non-volatile computer memories such as RAM, PROM, EPROM, and EEPROM (registered trademark), optical disks (CD, DVD, BD, etc.), magnetic storage media (hard disks and tapes, etc.). The various storage media may be fixed or portable within a computing device such that one or more programs stored thereon may be loaded into a processor.

[0104] To the extent that the embodiments are implemented partially or entirely in hardware, the blocks shown in the block diagrams of Figures 1 and 5 may be separate physical components, or logical partitions of a single physical component, or may all be integrated and implemented in one physical component. The functionality of a block shown in the figures may be divided among multiple components in an implementation, or the functionality of multiple blocks shown in the figures may be combined in a single component in an implementation. Hardware components suitable for use in the embodiments of the present invention include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs). One or more blocks may be implemented as a combination of dedicated hardware for performing some functions and one or more programmed microprocessors and associated circuitry for performing other functions.

[0105] Variations of the disclosed embodiments may be understood and effected by those skilled in the art practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and singular elements do not exclude a plurality. A single processor or other unit fulfills the functions of several items recited in the claims. The mere fact that certain means are recited in mutually different dependent claims does not indicate that a combination of these means cannot be used to advantage. Where a computer program is described above, the computer program is stored / distributed on a suitable medium, such as an optical storage medium or a solid medium, provided together with or as part of other hardware, but also distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. It should be noted that when the term "adapted" is used in the claims or the description, the term "adapted" is intended to be equivalent to the term "configured". Any reference signs in the claims should not be interpreted as limiting the scope.

[0106] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer programs according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams represents a module, segment, or portion of instructions that includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions mentioned in the blocks occur out of the order mentioned in the figures. For example, two blocks shown in succession are in fact executed substantially simultaneously, or the blocks are sometimes executed in reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, as well as combinations of blocks in the block diagrams and / or flowchart illustrations, can be realized by a special-purpose hardware-based system that performs the specified functions or actions, or a combination of special-purpose hardware and computer instructions.

Claims

1. 1. A method of processing images acquired from a personal care device, the personal care device comprising vibration means for causing a vibration part of the personal care device to vibrate in use to vibrate the personal care device, and a camera for acquiring images of one or more features of a user in use, the acquired images being affected by the vibration of the personal care device, the method comprising: mapping at least one region of the acquired image to a pose of the vibration part, the pose describing at least one of a location of the vibration part and an orientation of the vibration part; calculating a pixel shift for the at least one region based on the orientation of the vibration unit; shifting pixel data of the acquired image based on the calculated pixel shift for the at least one region to generate a corrected image; The method comprising:

2. The method of claim 1 , wherein the image comprises a plurality of distinct regions, and the mapping step comprises mapping each of the plurality of regions of the acquired image to a respective pose of the vibration part.

3. The method of claim 2 , wherein the calculating step comprises: calculating, for each of the plurality of regions, the pixel shift based on the respective orientation of the vibrating portion mapped to the region.

4. 4. The method of claim 1, wherein the shifting step comprises, for each of the at least one region of the acquired image, shifting the pixel data of the region of the acquired image by the calculated pixel shift for the region to generate the modified image.

5. The method of claim 1 , wherein the pixel shift is proportional to a distance d between the vibration part and an imaging plane of the camera.

6. The angle of the attitude of the vibration part from the central axis is Θ ｖｉｂ where the pixel shift is given by the distance d and the angle Θ ｖｉｂ Based on both, preferably, d.tan Θ is used as the pixel shift distance. ｖｉｂ The method of claim 5 , wherein the calculation is performed by:

7. The method according to claim 1 , further comprising the step of defining an area of ​​minimum pixel deviation within the acquired image.

8. The method of claim 7 , wherein calculating the pixel displacement for the at least one region is further based on a distance of the at least one region from the region of minimum pixel displacement.

9. The step of calculating the pixel shift for the at least one region includes calculating a rotation angle Θ of the oscillation. ｒｏｔ The method of any one of claims 1 to 8, further based on

10. The method of claim 1 , further comprising the step of transforming the calculated pixel shift from a rotated coordinate system to the Cartesian system of the camera image.

11. 11. A method according to any one of claims 1 to 10, wherein the step of mapping the regions of the acquired images to the orientation of the vibration part comprises determining the orientation of the vibration part when the image of the respective region was acquired.

12. The method of claim 1 , wherein calculating the pixel shift for each region is based on pixel density of the acquired image.

13. A computer program comprising computer program code means adapted to perform the method according to any one of claims 1 to 12 when the computer program is run on a computer.

14. 1. A personal care device comprising: a vibration means for vibrating the vibration part of the personal care device when in use to vibrate the personal care device; a camera that captures an image of one or more features of a user in use, the captured image being affected by vibration of the personal care device; 1. A processor unit configured to process the images acquired from the camera using an image processing algorithm, the image processing algorithm comprising: Mapping at least one region of the acquired image to a certain orientation of the vibration unit; calculating a pixel shift for the at least one region based on the orientation of the vibration unit; shifting pixel data of the acquired image based on the calculated pixel shift for the at least one region to generate a corrected image; A processor device having A personal care device comprising:

15. The personal care device of claim 14 , wherein the camera is located on the vibration portion.

Images