Providing feedback regarding a treatment operation performed on a body part of a subject

Abstract

According to an aspect, there is provided an apparatus for use with a treatment device for providing feedback to a user about a treatment operation performed on a body part of a subject, wherein the treatment device is configured to apply light pulses to skin of the body part to perform the treatment operation, wherein the light pulses applied to the skin treat an area of the skin. The apparatus comprises a processing unit configured to receive a first measurement signal from a first sensor, the first measurement signal comprising information about a position and / or motion of the treatment device over time; process the first measurement signal to estimate a previous treatment position as a position of the treatment device relative to the body part when a light pulse was generated for a light pulse previously applied to the body part by the treatment device during the treatment operation; estimate a previous treatment area for the previously applied light pulse based on the estimated previous treatment position, the previous treatment area corresponding to an area of the skin of the body part to which the light pulse was applied when the treatment device was at the previous treatment position; process the first measurement signal to estimate a current position of the treatment device relative to the body part; estimate a current treatment area based on the estimated current position of the treatment device, the current treatment area corresponding to an area of the skin to which a light pulse will be applied when the treatment device is at the current position; and generate a feedback control signal for a feedback unit, wherein the feedback control signal is configured to cause the feedback unit to generate feedback indicating whether the current treatment area corresponds to or substantially corresponds to the previous treatment area.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to providing feedback on a treatment operation performed on a body part of a subject, and in particular to providing feedback on a treatment operation in which light pulses are applied to the skin on a body part. BACKGROUND

[0002] Techniques for removing unwanted hair include shaving, electrolysis, plucking, laser and light therapy (known as photoepilation) and injection of a treatment androgen. Light-based techniques are also used in other types of dermatological treatments, including reducing hair growth and treating acne.

[0003] By using an appropriate configuration of light energy, i.e. in terms of wavelength, intensity and / or pulse duration if the light is pulsed, selective heating of the hair root and subsequent temporary or permanent damage of the hair follicle can be achieved. Domestic photoepilation devices, such as the Philips Lumea device, use intense pulsed light (IPL) from a high intensity light source, such as a high output pulsed xenon flash lamp that generates broad spectrum light.

[0004] Photoepilation treatments are characterized in that a user of a photoepilation device treats a relatively small skin area for the purpose of removing hair. Photoepilation treatments use strong light to heat the melanin in the hair and hair root, which puts the hair follicle in a resting phase, preventing hair regrowth. In order to use this technique effectively for hair removal, the user must treat the skin completely without leaving any gaps. Since the duration of this effect is limited, the treatment must be repeated regularly: conventionally, once every 4 to 8 weeks in a maintenance period after an initial period of about two months (the treatment is performed every two weeks).

[0005] WO 2019 / 224273 Al discloses a device for treating a body part with light pulses. The device comprises a handpiece for applying light pulses to the body part. The handpiece is movable relative to the body part and comprises an exit window via which the light pulses are applied to the body part. The handpiece further has an optical, capacitive or inertial motion detector. The device further comprises a system for controlling the light pulse emission during the handpiece motion at a rate determined based on information provided by the motion detector. The device can further have a display configured to show one or more of the following features: a position of the handpiece on the body part provided by a camera; an area of the body part being treated and / or an area of the body part not being treated; and a historical temperature of the skin based on information provided by a temperature sensor.

[0006] In a conventional photoepilation treatment, a user of a photoepilation device has to repeatedly manually position the photoepilation device on the skin and trigger light pulses to cover an entire body part (e.g. arm, leg, etc.). However, since the photoepilation device typically does not provide any feedback to the user about the areas that have already been treated, and there is no or little user-perceptible change to the skin or hair at or shortly after the application of a light pulse, it is difficult for the user to fully cover the body part and / or to avoid over-treating certain areas of the body part. Furthermore, some types of treatment devices can utilize different accessories to treat different parts of the body, and these accessories can change the size of the area of skin treated with each light pulse. This makes it even more difficult for the user to determine which parts of the skin have already been treated.

[0007] It is therefore desirable to be able to determine and provide feedback to a user of a treatment device about a treatment operation performed on a body part of a subject. SUMMARY

[0008] According to a first particular aspect, there is provided an apparatus for use with a treatment device for providing feedback to a user about a treatment operation performed on a body part of a subject, wherein the treatment device is configured to apply light pulses to skin of the body part to perform the treatment operation, wherein a light pulse applied to the skin treats an area of the skin. The apparatus comprises a processing unit configured to receive a first measurement signal from a first sensor, the first measurement signal comprising information about a position and / or motion of the treatment device over time; process the first measurement signal to estimate, for a light pulse previously applied to the body part by the treatment device during the treatment operation, a previous treatment position as a position of the treatment device relative to the body part when the light pulse was generated; estimate, for the previously applied light pulse, a previous treatment area corresponding to an area of the skin of the body part to which the light pulse was applied when the treatment device was at the previous treatment position; process the first measurement signal to estimate a current position of the treatment device relative to the body part; estimate, based on the estimated current position of the treatment device, a current treatment area corresponding to an area of the skin to which a light pulse will be applied when the treatment device is at the current position; and generate a feedback control signal for a feedback unit, wherein the feedback control signal is configured to cause the feedback unit to generate feedback indicating whether the current treatment area corresponds to or substantially corresponds to the previous treatment area.

[0009] In some embodiments, the first sensor is one of: a motion sensor, an accelerometer, a gyroscope, a magnetometer, a barometric pressure sensor, and an imaging unit.

[0010] In some embodiments, the sensor is an imaging unit, the first measurement signal is a plurality of images or a video sequence, and the plurality of images or video sequence is processed to identify a position and / or motion of the treatment device over time. In these embodiments, the processing unit can be configured to: identify skin features and / or body part features in the images or video sequence; monitor motion of those skin features and / or body part features in the images or video sequence over time; and determine a position and / or motion of the treatment device relative to the body part from the motion of those skin features and / or body part features in the images or video sequence. In these embodiments, the processing unit can also be configured to receive a second measurement signal from a motion sensor, the second measurement signal comprising information about a position and / or motion of the treatment device over time; the processing unit can be configured to process the first measurement signal and the second measurement signal to estimate a previous treatment position and a current position.

[0011] In some embodiments, the feedback control signal is a display control signal, and the feedback unit is a display unit. In these embodiments, the display control signal can be configured to cause the display unit to generate a graphical display comprising a graphical representation of the previous treatment area and the current treatment area overlaid on the body part. In these embodiments, the display unit can be a display screen, and the display control signal can comprise a plurality of images or a video sequence of the body part with the graphical representation superimposed thereon. In alternative embodiments, the display unit can be a projector, and the display control signal can be configured to cause the projector to project the graphical representation onto the body part of the subject in accordance with the estimated previous treatment area and the current treatment area. In other alternative embodiments, the display unit can be configured to display the graphical representation as part of an augmented reality display of the body part.

[0012] In some embodiments, the processing unit is configured to estimate the previous treatment area and estimate the current treatment area based on: (i) a size of an aperture in the treatment device through which the light pulse passes; and (ii) a size of an aperture in an attachment that is attached to the treatment device.

[0013] In some embodiments, the processing unit is further configured to identify when the light pulse is applied to the skin of the body part. In these embodiments, the processing unit is configured to identify when the light pulse is applied to the skin of the body part by processing the first measurement signal or the received images or video sequence to identify an increase in brightness due to the light pulse.

[0014] In some embodiments, the processing unit is further configured to generate an actuation signal that will cause the light source in the treatment device to produce a light pulse if the determined current treatment area does not overlap or does not substantially overlap with any of the previous treatment areas.

[0015] In some embodiments, the processing unit is further configured to estimate a previous treatment area and a current treatment area, respectively, based on a predetermined relationship between a position of an orifice of the processing device through which a light pulse in the processing device is applied to the skin and an estimated previous treatment position and an estimated current position of the processing device, respectively.

[0016] In some embodiments, the apparatus is part of a processing device. In alternative embodiments, the apparatus is separate from the processing device.

[0017] In some embodiments, the apparatus comprises a first sensor. In alternative embodiments, the apparatus is separate from the first sensor.

[0018] According to a second aspect, there is provided a system comprising: a processing device comprising one or more light sources for performing a treatment operation on a skin; a first sensor for outputting a first measurement signal comprising information about a position and / or motion of the processing device over time; and an apparatus according to the first aspect or any embodiment thereof.

[0019] According to a third particular aspect, there is provided a computer-implemented method for providing feedback to a user about a treatment operation performed by a processing device on a body part of a subject, wherein the processing device is configured to apply light pulses to a skin of the body part to perform the treatment operation, wherein a light pulse applied to the skin treats an area of the skin. The method comprises receiving a first measurement signal from a first sensor, the first measurement signal comprising information about a position and / or motion of the processing device over time; processing the first measurement signal to estimate a previous treatment position for a light pulse previously applied to the body part by the processing device when the light pulse was generated, for the light pulse previously applied; estimating a previous treatment area for the light pulse based on the estimated previous treatment position, the previous treatment area corresponding to an area of the skin of the body part to which the light pulse was applied when the processing device was at the previous treatment position; processing the first measurement signal to estimate a current position of the processing device relative to the body part; estimating a current treatment area based on the estimated current position of the processing device, the current treatment area corresponding to an area of the skin to which a light pulse will be applied when the processing device is at the current position; and generating a feedback control signal for a feedback unit, wherein the feedback control signal is configured to cause the feedback unit to generate feedback indicating whether the current treatment area corresponds to or substantially corresponds to the previous treatment area.

[0020] In some embodiments, the first sensor is one of: a motion sensor, an accelerometer, a gyroscope, a magnetometer, a barometric pressure sensor, and an imaging unit.

[0021] In some embodiments, the sensor is an imaging unit, the first measurement signal is a plurality of images or a video sequence, and the plurality of images or video sequence is processed to identify a position and / or motion of the treatment device over time. In these embodiments, the method can further comprise: identifying skin features and / or body part features in the images or video sequence; monitoring motion of those skin features and / or body part features in the images or video sequence over time; and determining a position and / or motion of the treatment device relative to the body part from the motion of those skin features and / or body part features in the images or video sequence. In these embodiments, the method can further comprise receiving a second measurement signal from a motion sensor, the second measurement signal comprising information about a position and / or motion of the treatment device over time; the steps of processing the first measurement signal to estimate a previous treatment position and processing the first measurement signal to estimate a current position can comprise processing the first measurement signal and the second measurement signal to estimate the previous treatment position and the current position.

[0022] In some embodiments, the feedback control signal is a display control signal, and the feedback unit is a display unit. In these embodiments, the display control signal can be configured to cause the display unit to generate a graphical display comprising a graphical representation of the previous treatment area and the current treatment area overlaid on the body part. In these embodiments, the display unit can be a display screen, and the display control signal can comprise a plurality of images or a video sequence of the body part with the graphical representation superimposed thereon. In alternative embodiments, the display unit can be a projector, and the display control signal can be configured to cause the projector to project the graphical representation onto the body part of the subject in accordance with the estimated previous treatment area and the current treatment area. In other alternative embodiments, the display unit can be configured to display the graphical representation as part of an augmented reality display of the body part.

[0023] In some embodiments, the steps of estimating the previous treatment area and estimating the current treatment area can comprise estimating the previous treatment area and estimating the current treatment area based on: (i) a size of an aperture in the treatment device through which the light pulse passes; and (ii) a size of an aperture in an attachment that is attached to the treatment device.

[0024] In some embodiments, the method further comprises identifying when the light pulse is applied to the skin of the body part. In these embodiments, the step of identifying when the light pulse is applied to the skin can comprise identifying when the light pulse is applied to the skin of the body part by processing the first measurement signal or the received images or video sequence to identify an increase in brightness due to the light pulse.

[0025] In some embodiments, the method further includes generating an actuation signal such that a light source in the processing device generates a light pulse if the determined current processing area does not overlap or substantially does not overlap with any area in the previously processed area.

[0026] According to a fourth aspect, a computer program product including a computer-readable medium is provided, the computer-readable medium having computer-readable code implemented therein, the computer-readable code being configured to, when executed by a suitable computer or processing unit, cause the computer or processing unit to perform the method according to the third aspect or any embodiment thereof.

[0027] These and the described embodiments, these and other aspects will be obvious and will be explained. Attached Figure Description

[0028] Figure 1 This is an illustration of an exemplary processing device that can be used with the present invention;

[0029] Figure 2 This is a block diagram of an exemplary system including an imaging unit and apparatus according to various embodiments;

[0030] Figure 3 Infrared images of the arm after multiple light pulses were applied are shown;

[0031] Figure 4 This is a flowchart illustrating an exemplary method according to the technology described herein;

[0032] Figure 5 shows two images obtained by the imaging unit;

[0033] Figure 6 It can be an illustration of visual feedback provided according to embodiments of the technology described herein; and

[0034] Figure 7 These are images illustrating exemplary visual feedback provided by embodiments of the technology described herein. Detailed Implementation

[0035] As described above, the techniques described herein can be used to determine and provide feedback on treatment procedures performed on a subject's body parts. Specifically, the feedback can indicate whether the current location of the treatment device on the skin of the body part is skin that has already been treated (i.e., light pulses have been applied). In a particular embodiment, the feedback can be a graphical display that includes a graphical representation of any previously treated skin areas and an indication of the skin area to be treated at the current location of the treatment device.

[0036] The techniques described herein can be implemented by a processing device (e.g., by a processing unit within a processing device) or by a processing unit in a separate device (e.g., a smartphone, tablet, smartwatch, laptop, computer, server, etc.). One or more sensors are required to provide information on the location (including orientation) and / or motion of the processing device over time. In some embodiments, the one or more sensors include one or more motion sensors, such as accelerometers, gyroscopes, etc. In these embodiments, the one or more motion sensors may alternatively be referred to as an inertial measurement unit (IMU). In some embodiments, the one or more sensors include an imaging unit (e.g., a camera) that acquires multiple images or video sequences of a skin region on a subject. In the case of a motion sensor (e.g., an accelerometer), the motion sensor may be part of the processing device. In the case of an imaging unit, the imaging unit may be part of the processing device, part of a device separate from the processing device implementing the techniques described herein, or separate from the processing device and any device implementing the techniques described herein. It should be understood that in some embodiments, the one or more sensors may include one or more motion sensors and an imaging unit.

[0037] Figure 1 This is an illustration of an exemplary processing device 2 that can be used to apply light pulses to a skin area. It should be understood that... Figure 1 The processing device 2 mentioned is merely an example of a handheld processing device 2 that can be used with the present invention, and the processing device 2 is not limited to... Figure 1 The device is shown in the form of a handheld treatment device. The treatment device 2 is used on the body of a subject (e.g., a human or animal) and is held in one or both hands by the user during use. When the treatment device 2 comes into contact with or approaches a part of the subject's body, it performs a treatment on the subject's skin or body using one or more light pulses. The treatment may be the removal of unwanted hair by laser and / or phototherapy (referred to as phototherapy or intense pulsed light (IPL) treatment). Alternatively, the treatment may be laser and / or phototherapy for reasons other than hair removal or prevention, such as dermatological (skin) treatments, acne treatments, phototherapy for other types of skin conditions, skin rejuvenation, skin tightening, or port-wine stain treatment.

[0038] As described herein, the processing device 2 is operated or used by a “user” and is used on the body of a “subject.” In some cases, the user and the subject are the same person, i.e., the processing device 2 is held in the hand and used by the user himself (e.g., on the skin of his / her leg). In other cases, the user and the subject are different people, e.g., the processing device 2 is held in the hand and used by the user on someone else. In either case, due to little or no user-perceptible change in the skin during or shortly after the application of the light pulse, it is difficult for the user to achieve complete coverage of the body part and / or avoid overtreatment of certain areas of the body part.

[0039] An exemplary processing device 2 includes a housing 4, which includes at least a handle portion 5 and a head portion 6. The handle portion 5 is shaped to allow a user to hold the processing device 2 with one hand. The head 6 is located at the head end 8 of the housing 4 and is positioned to contact a subject so as to perform a processing operation on the subject's body or skin at the location where the head 6 contacts the body or skin.

[0040] Processing device 2 is used to perform processing operations using light pulses. Therefore, in Figure 1 In the device 2, the head 6 includes an aperture 10 disposed in or on the housing 4, such that the aperture 10 can be placed adjacent to or on the subject's skin (i.e., in contact with the subject's skin). The processing device 2 includes one or more light sources 12 for generating light pulses, which are applied to the subject's skin via the aperture 10 to perform the processing operation. One or more light sources 12 are disposed in the housing 4 such that light pulses are provided from the one or more light sources 12 through the aperture 10. The aperture 10 may be in the form of an opening at the head end 8 of the housing 4, or it may be in the form of a window (including a waveguide) that is transparent or translucent to the light pulse (i.e., the light pulse can pass through the window).

[0041] exist Figure 1 In the exemplary embodiment shown, the orifice 10 has a generally rectangular shape, which creates a generally rectangular skin treatment area on the skin. It should be understood that the orifice 10 can have any other desired shape. For example, the orifice 10 can be a square, ellipse, circle, or any other polygon.

[0042] Despite Figure 1 Not shown, but the processing device 2 may have one or more detachable attachments for the head 6 of the processing device 2, each attachment including an opening 10. The attachments may be used on different body parts and have different sized openings 10. For example, one attachment may be used on a large body part such as the leg, and therefore has a large opening 10, while another attachment may be used on the skin / hair above the upper lip, and therefore has a small opening 10.

[0043] One or more light sources 12 can generate light pulses of any suitable or desired wavelength (or wavelength range) and / or intensity. For example, light source 12 can generate visible light, infrared (IR) light, and / or ultraviolet (UV) light. Each light source 12 can include any suitable type of light source, such as one or more light-emitting diodes (LEDs), (xenon) flash lamps, one or more lasers, etc. Light source 12 can provide light pulses of approximately 2.5 milliseconds (ms) in the spectral content range of 560–1200 nanometers (nm), because these wavelengths heat the melanin in the hair and hair root through absorption, which puts the hair follicle into a quiescent phase and prevents hair regrowth.

[0044] One or more light sources 12 are configured to provide light pulses. That is, the light source 12 is configured to generate high-intensity light for a short duration (e.g., less than 1 second). The intensity of the light pulses should be high enough to enable processing operations on the skin or body part adjacent to the aperture 10.

[0045] The processing device 2 shown may also optionally include two skin contact sensors 14, 16 located on or within the head 6, for determining whether the head 6 is in contact with the skin. The skin contact sensors 14, 16 measure parameters indicating whether the head 6 is in contact with the skin and generate corresponding measurement signals including a time series of parameter measurements. The measurement signals can be processed to determine whether the head 6 is in contact with the skin. Typically, skin contact sensors are used in the processing device 2, particularly in phototherapy hair removal devices, to ensure that the processing device 2 is properly in contact with the skin before generating light pulses, thus preventing the light pulses from being directed into the eyes of the user or subject.

[0046] In some embodiments, the parameter may be capacitance, and thus the skin contact sensors 14, 16 may measure capacitance via a corresponding pair of electrical contacts or electrodes on the surface of the head 6, wherein the measured capacitance indicates the presence of skin contact. In an alternative embodiment, the parameter may be light intensity or level, and thus the skin contact sensors 14, 16 may be light sensors that measure the intensity or level of light incident on the light sensor, wherein the measured intensity or level indicates the presence of skin contact (e.g., less light / no light can indicate skin contact because skin blurs the light sensors 14, 16, and vice versa). In other alternative embodiments, the parameter may be a measurement of contact pressure, and thus the skin contact sensors 14, 16 may measure contact pressure via a corresponding pressure sensor or mechanical switch, wherein the measured contact pressure indicates the presence of skin contact.

[0047] The processing device 2 shown also includes an optional skin tone sensor 18 located on or within the head 6, which can be used to determine the skin tone of the skin contacted by the head 6. The skin tone sensor 18 can measure parameters indicating skin tone and generate a measurement signal including a time series of parameter measurements. The measurement signal can be processed to determine the skin tone of the skin contacted by the head 6. Typically, skin tone sensors are used in the processing device 2, particularly in phototherapy hair removal devices, to ensure that the light pulses have an intensity suitable for the skin type being treated, or even to prevent the generation of light pulses if the skin type is unsuitable for them (e.g., darker skin with a much higher melanin content).

[0048] In some embodiments, the optional skin color sensor 18 may be a light sensor, and the parameter measured by the light sensor may be the intensity or level of light of a specific wavelength or multiple wavelengths reflected from the skin. The measured intensity or level of reflected light at a specific wavelength can indicate skin color. The measured intensity or level of reflected light may be based on the concentration of melanin in the skin, and thus the measured intensity or level can indicate melanin concentration. Melanin concentration can be derived, for example, from measurements of light reflection at wavelengths of 660 nm (red) and 880 nm (infrared).

[0049] The processing device 2 shown also includes a user controller 20, which can be operated by a user to activate the processing device 2, causing the head 6 to perform the required processing operations on the subject's body (e.g., generating one or more light pulses by one or more light sources 12). The user controller 20 may be in the form of a switch, button, touchpad, etc.

[0050] As described above, one or more sensors are needed to provide information about the position (including orientation) and / or motion of the processing device over time, so that appropriate feedback can be determined, and in some embodiments, the one or more sensors may be part of or within the processing device 2. Where the sensors are one or more motion sensors (e.g., accelerometers, gyroscopes, etc.), the one or more motion sensors may be part of the processing device 2, for example, inside the processing device 2 (and therefore within...). Figure 1 (Not shown in the image). In embodiments where one or more sensors include an imaging unit (e.g., a camera), the imaging unit may be on or within the processing device 2. Figure 1As shown, the imaging unit 22 can be arranged on the housing 4 of the processing device 2, enabling it to acquire images or video sequences of skin or body parts located in front of the processing device 2. That is, the imaging unit 22 is arranged on the housing 4 of the processing device 2 such that it acquires images or video sequences from a direction generally parallel to the direction in which the light source 12 emits light pulses through the aperture 10. In some implementations, the imaging unit 22 is arranged on the housing 4 such that when the head 6 is positioned in contact with the subject, allowing light pulses to be applied to the skin portion visible to the light source 12 through the aperture 10, the imaging unit 22 cannot see the skin portion in contact with the aperture 10 and the head end 6. In embodiments where the imaging unit 22 is arranged on or within the housing 4 of the processing device 2, the imaging unit 22 is in a fixed arrangement relative to the rest of the processing device 2.

[0051] Figure 2 This is a block diagram of an exemplary system 40, which includes means 42 for providing feedback on skin areas that have been treated with light pulses and those that have not. System 40 also includes: one or more sensors 44 for providing measurement signals including information about the position and / or movement of the processing device 2 over time; and a feedback unit 45 for providing feedback to the user and / or subject on skin areas that have been / have not been treated with light pulses. In some implementations, the processing device 2 may be considered part of system 40, although in Figure 2 Processing device 2 is not shown. As described above, in some embodiments, device 42 may be a separate device from processing device 2, and therefore device 42 may be in the form of an electronic device, such as a smartphone, smartwatch, tablet computer, personal digital assistant (PDA), laptop computer, desktop computer, remote server, smart mirror, etc. In other embodiments, device 42, and in particular the functions provided by device 42 according to the invention, are part of processing device 2.

[0052] As described above, one or more sensors 44 may be one or more motion sensors 44, such as accelerometers, gyroscopes, barometers (which can be used to measure changes in altitude), magnetometers, etc., which measure the motion of the processing device 2 and output measurement signals representing these motions. One or more motion sensors 44 may be part of or within the processing device 2. In the case where one or more motion sensors 44 include accelerometers, the accelerometer 44 may generate measurement signals that include multiple acceleration measurement samples representing the motion of the processing device 2 at multiple moments. The accelerometer 44 may be an accelerometer that measures three-dimensional acceleration, and the measurement signals generated by the accelerometer 44 may include individual measurement signals representing three-dimensional acceleration. For example, the accelerometer 44 may output corresponding measurement signals for each of the x-axis, y-axis, and z-axis of a Cartesian coordinate system.

[0053] One or more sensors 44 may also include, or optionally, an imaging unit 44 (e.g., a camera or event camera) that acquires multiple image or video sequences. An event camera is an imaging sensor in which each pixel of the sensor independently indicates a change in brightness at the time an event occurs. Multiple images (including images formed from brightness indications from the event camera) or video sequences are output as measurement signals, and imaging processing techniques can be used to process the measurement signals (image / video sequences) to identify motion of the processing device 2. In some embodiments, the imaging unit 44 is attached to or a portion of the processing device 2, in which case the image / video sequences can be processed to extract motion of the processing device 2 based on motion of the subject (e.g., body parts or skin) visible in the image / video sequences. In other embodiments, the imaging unit 44 may be separate from the processing device 2, such as an imaging unit 44 within a device 44, or an imaging unit 4 in a separate device, such as a smartphone, smartwatch, tablet computer, personal digital assistant (PDA), laptop computer, desktop computer, smart mirror, etc. In this case, the imaging unit 44 is capable of observing the processing device 2 and body parts from a distance. Imaging unit 44 may include any suitable components for capturing images, such as a charge-coupled device (CCD) and one or more lenses and / or mirrors. In some embodiments, imaging unit 44 is a camera, such as a digital camera or an event camera. As described above, in some embodiments, one or more sensors 44 may include one or more motion sensors and imaging units.

[0054] Device 42 includes a processing unit 46, which generally controls the operation of device 42 and enables device 42 to perform the methods and techniques described herein. Processing unit 44 receives measurement signals from sensor 44, processes the measurement signals to determine feedback to be provided regarding the processing operation, and provides feedback control signals to feedback unit 45. Thus, in embodiments where sensor 44 is part of device 42, processing unit 46 may be configured to receive measurement signals directly from one or more sensors 44, or in embodiments where sensor 44 is separate from device 42, processing unit 46 may be configured to receive measurement signals from one or more sensors 44 via another component. In either case, processing unit 46 may include or include one or more input ports or wires for receiving measurement signals. Processing unit 46 may also include or include one or more output ports or cables for outputting feedback control signals.

[0055] Processing unit 46 may be implemented in a variety of ways, using software and / or hardware, to perform the various functions described herein. Processing unit 46 may include one or more microprocessors or digital signal processors (DSPs) that may be programmed using software or computer program code to perform desired functions and / or control components of processing unit 46 to achieve desired functions. Processing unit 46 may be implemented as a combination of dedicated hardware performing certain functions (e.g., amplifiers, preamplifiers, analog-to-digital converters (ADCs), and / or digital-to-analog converters (DACs)) and processors performing other functions (e.g., one or more programmed microprocessors, controllers, DSPs, and associated circuitry). Examples of components that may be employed in various embodiments of the invention include (but are not limited to) conventional microprocessors, DSPs, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), hardware for implementing neural networks and / or so-called artificial intelligence (AI) hardware accelerators (i.e., processors or other hardware specifically designed for AI applications that can be used in conjunction with a main processor) and / or hardware specifically designed for simultaneous localization and mapping (SLAM) techniques.

[0056] Processing unit 46 may include or be associated with memory unit 48. Memory unit 48 may store data, information, and / or signals (including images) for use by processing unit 46 in the operation of control device 42 and / or in performing or carrying out the methods described herein. In some implementations, memory unit 48 stores computer-readable code executable by processing unit 46, causing processing unit 46 to perform one or more functions, including the methods described herein. In certain embodiments, the program code may be in the form of an application for a smartphone, tablet computer, laptop computer, computer, or server. The memory cell 48 may include any type of non-transitory machine-readable medium, such as cache or system memory, including volatile and non-volatile computer memory, such as random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), and electrically erasable PROM (EEPROM), and the memory cell may be implemented in the form of memory chips, optical discs (such as optical discs (CD), digital versatile discs (DVD), or Blu-ray discs), hard disks, magnetic tape storage solutions, or solid-state devices (including memory sticks, solid-state drives (SSDs), memory cards, etc.).

[0057] exist Figure 2In the illustrated embodiment, since device 42 is shown as separate from sensor 44, device 42 also includes interface circuitry 50 to enable device 42 to receive measurement signals from sensor 44. Interface circuitry 50 in device 42 enables data connection and / or data exchange with other devices, including any one or more of sensor 44, processing device 2, servers, and databases. The connection to sensor 44 (or any electronic device, such as processing device 2) can be direct or indirect (e.g., via the Internet), and therefore interface circuitry 50 can enable connection between device 42 and a network via any desired wired or wireless communication protocol, or directly enable connection between device 42 and another device (e.g., sensor 44 and / or processing device 2). For example, interface circuitry 50 can operate using WiFi, Bluetooth, Zigbee, or any cellular communication protocol (including but not limited to Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), Long Term Evolution (LTE), Advanced LTE, etc.). In the case of wireless connectivity, interface circuitry 50 (and therefore device 42) may include one or more suitable antennas for transmitting / receiving via a transmission medium (e.g., air). Alternatively, in the case of a wireless connection, the interface circuitry 50 may include means (e.g., a connector or plug) that enable the interface circuitry 50 to connect to one or more suitable antennas external to the device 42 for transmitting / receiving via a transmission medium (e.g., air). The interface circuitry 50 is connected to the processing unit 46.

[0058] Despite Figure 2 Although not shown, device 42 may include one or more user interface components, which include one or more components that enable a user of device 42 to input information, data, and / or commands into device 42 and / or enable device 42 to output information or data to the user of device 42. The user interface may include any suitable input device, including but not limited to a keyboard, keypad, one or more buttons, switches or dial pads, mouse, trackpad, touchscreen, stylus, camera, microphone, etc., and the user interface may include any suitable output device, including but not limited to a display unit or display screen, one or more lamps or lamp elements, one or more speakers, vibration elements, etc.

[0059] Feedback unit 45 is used to generate feedback indicating whether the currently processed area corresponds to or substantially corresponds to a previously processed area. The feedback can be any one of visual, auditory, and tactile, or any combination thereof. Therefore, feedback unit 45 may include any one or more output components, including but not limited to display units (such as displays, displays / transparent panels for providing augmented reality (AR) displays, or projectors such as picoprojectors or microelectromechanical systems (MEMS) picoprojectors), one or more lamps or lamp elements, one or more speakers, and one or more vibration elements. In a preferred embodiment, feedback unit 45 is a display unit, and the feedback is a graphic display including a graphic representation of the skin area previously processed with light pulses, and a graphic representation of the area processed with light pulses according to the current position of processing device 2.

[0060] In some embodiments, the feedback unit 45 is attached to or a portion of the processing device 2. In other embodiments, the feedback unit 45 is part of the device 42. In these embodiments, the feedback unit 45 may be part of or utilize any output component of the user interface component of the device 42. In further embodiments, the feedback unit 45 may be separate from the processing device 2 and the device 42, for example in a separate device such as a smartphone, smartwatch, tablet computer, personal digital assistant (PDA), laptop computer, desktop computer, smart mirror, etc.

[0061] It should be understood that the actual implementation of device 42 may include Figure 2 Additional components are shown. For example, device 42 may also include a power source, such as a battery, or components for enabling device 42 to be connected to an AC power source.

[0062] As described in the background section, in conventional light-based treatments, the user of the treatment device 2 must repeatedly and manually position the device 2 on the skin and trigger light pulses. However, there are little or no user-perceptible changes to the skin or hair during or shortly after the application of light pulses, making it difficult for the user to achieve complete coverage of the body part and / or avoid overtreatment of certain areas of the body part by repeating light pulses on already treated skin areas. To illustrate the typical treatment coverage obtained without providing any feedback to the user of the treatment device 2, Figure 3 An infrared image of arm 60 after multiple light pulses have been applied is shown. This infrared image was obtained using a thermal imaging camera. It should be understood that this image is included for the purpose of illustrating the problems solved by the disclosed technology, and that processing device 2, apparatus 42, and system 40 do not include a thermal (infrared) camera. Figure 3In the image, the brightness of a portion of the image depends on the temperature of that portion; brighter (whiter) portions correspond to higher temperatures, and darker (blacker) portions correspond to lower temperatures. It can be seen that there are multiple bright regions 62 on arm 60 (in... Figure 3 Only some bright areas are marked in the image; these correspond to regions of higher temperature. These areas have a higher temperature due to the light pulse recently applied by processing device 2. The light pulse will cause a small (temporary) increase in temperature of the skin portion to which the light pulse is applied. Figure 3 As can be seen, many parts of arm 60 have not recently been treated with light pulses, particularly the gaps (labeled 64) between bright areas 62. However, light pulses are unlikely to cause visible changes in the skin, so the user of the processing device 2 will not be able to see which parts of arm 60 have been treated with light pulses. This illustrates the need for feedback from the user of the processing device 2, enabling them to identify which parts of the body have been treated with light pulses.

[0063] In short, the technology described herein allows for the identification or differentiation of skin areas to which light pulses have been applied and areas not to which light pulses have not been applied by tracking the position of the processing device 2 relative to the body part during the application of light pulses, and by tracking the current position of the processing device 2 relative to those locations where light pulses have already been applied. Control signals can be generated for the feedback unit 45 (e.g., the display unit), which will cause the feedback unit 45 to provide feedback on whether the current processing area (i.e., where the processing device 2 is currently positioned) corresponds to or substantially corresponds to a previously processed area. Therefore, the feedback allows the user to better determine where the next light pulse should be applied to the body part.

[0064] In embodiments where the feedback is visual, the visual feedback may be a graphical display that includes a superimposed graphical representation of previously processed and currently processed areas on the body part. This allows the user of the processing device 2 to easily see which parts of the body part have been processed and which still need processing.

[0065] Figure 4 The flowchart illustrates an exemplary method according to the technology described herein. One or more steps of the method can be performed by a processing unit 46 in device 42 in conjunction with memory unit 48, interface circuitry 50, and a suitable user interface. The processing unit 46 can perform one or more steps in response to executing computer program code that can be stored on a computer-readable medium (e.g., memory unit 48).

[0066] In step 101, a first measurement signal is received from sensor 44. The first measurement signal includes information about the position and / or motion of processing device 2 over time. As described above, the first measurement signal may be a signal from motion sensor 44, or it may be a signal representing a sequence of multiple images or videos. In some embodiments, step 101 may include receiving the first measurement signal from a first sensor 44 (e.g., motion sensor 44) and receiving a measurement signal from another sensor 44 (e.g., imaging unit).

[0067] In step 101, the first measurement signal can be received directly from sensor 44, for example, in real time or near real time when the measurement signal is generated by sensor 44. Alternatively, the first measurement signal may have been previously generated by sensor 44 and stored for subsequent analysis, for example, stored in memory unit 48, in a memory unit associated with processing device 2 or sensor 44, or in a remote database. In this case, step 101 may include processing unit 46 obtaining or retrieving the first measurement signal from the storage location (e.g., from memory unit 48, etc.). In these alternative embodiments, in order to provide feedback to the user at an appropriate time so that the user can complete the processing operation, the first measurement signal should be obtained or retrieved shortly after its generation (e.g., no longer than a few seconds).

[0068] In step 103, for a light pulse previously applied to a body part by the processing device 2 during the processing operation, the processing unit 46 processes a first measurement signal to estimate the position of the processing device 2 relative to the body part when the light pulse was generated or applied. This estimated position is referred to as the "previously processed position".

[0069] In embodiments where sensor 44 is one or more motion sensors such as an accelerometer, the first measurement signal is a motion measurement signal, and step 103 includes processing the motion measurement signal to identify the movement and position of the processing device 2 relative to a body part. Techniques for processing measurements from motion sensors to determine the movement and position of a device in which motion sensor 44 is housed are known in the art and will not be described further herein.

[0070] In embodiments where sensor 44 is an imaging unit and the first measurement signal is a sequence of multiple images or videos, step 103 includes processing the image or video sequence to identify the motion and position of the processing device 2 relative to a body part. In embodiments where the imaging unit 44 is attached to the processing device 2, the processing unit 46 is capable of processing the image or video sequence to identify skin features and / or body part features in the image or video sequence and monitoring the motion of those skin features and / or body part features in the image or video sequence. The determined motion of those skin features and / or body part features in the image or video sequence corresponds to the motion of the processing device 2 relative to the body part. The imaging unit 44 is fixed in or on the processing device 2, and the relationship between the aperture 10 and the field of view of the imaging unit 44 will be known, enabling the processing unit 46 to determine the previous processing position, i.e., the position of the processing device 2 relative to the body part when a light pulse is applied. Techniques for processing image or video sequences to identify skin features and / or body part features are generally known in the field of imaging processing, although some further details are provided below.

[0071] In an alternative embodiment where the imaging unit 44 is separated from the processing device 2 and both the processing device 2 and the body part can be observed from a distance, in step 103, the processing unit 46 can process the image or video sequence to identify the processing device 2 and skin features and / or body part features, and monitor their relative positions over time. Techniques for processing image or video sequences to identify the processing device 2 and skin features and / or body part features are generally known in the field of imaging processing, although some further details are provided below.

[0072] In embodiments where one or more sensors 44 include one or more motion sensors and imaging units, step 103 may include processing a first measurement signal from one of the motion sensors and imaging units and a second measurement signal from the other of the motion sensors and imaging units to estimate the position of the processing device 2 relative to a body part.

[0073] In some embodiments, such as in the above-described embodiments where one or more sensors 44 include one or more motion sensors and imaging units, step 103 may include using Simultaneous Localization and Mapping (SLAM) technology to determine the position of the processing device 2 relative to the body part.

[0074] In step 103, after the first measurement signal has been processed to estimate the movement and / or position of the processing device 2 relative to the body part over time, the previously processed position is determined as the estimated position of the processing device 2 relative to the body part when the light pulse is generated or applied.

[0075] In step 105, for the light pulse estimated in step 103 and based on the previous processing position determined in step 103, processing unit 46 estimates the skin area of ​​the body part to which the light pulse is applied. This estimated area is referred to as the "previous processing area" of the light pulse applied when processing device 2 was in the previous processing position. The previous processing position typically has a known relationship with the position of the aperture 10 of processing device 2. This allows step 105 to determine the appropriate area of ​​skin on the body part to which the light pulse is applied. For example, if the previous processing position is estimated to be the geometric center of aperture 10, the previous processing area can be centered on the relevant previous processing position. The orientation of processing device 2 when the light pulse is applied is used to determine the orientation of the previous processing area on the skin. Reference Figure 3 The total previously processed area estimated or determined in step 105 should correspond to the bright area 62.

[0076] The size of the previously processed area will depend on the size of the aperture 10 in the processing device 2. In some embodiments, the size of the aperture 10 is fixed or predetermined, and the processing unit 46 can determine the size of the previously processed area using information about the size of the aperture 10 (e.g., the size of the aperture 10), and optionally other information that may affect the size of the skin area to which the light pulse is applied, such as the relative position of the aperture 10 and the light source 12. Alternatively, the processing unit 46 can determine the size of the aperture 10 by analyzing an image of the aperture 10 (e.g., if an image of the aperture 10 is obtained using an imaging unit prior to use).

[0077] As described above, in some embodiments, the processing device 2 may have multiple different attachments capable of processing different parts of the body. These attachments may have orifices 10 of different sizes to vary the size of the skin area processed with each light pulse. In this case, when determining the size of the previously processed area, the processing unit 46 may utilize information about the size of the orifice 10 (e.g., the size of the orifice 10), as well as optional other information, such as the relative position of the orifice 10 and the light source 12 used when the light pulse is applied. In some embodiments, attachments used on the processing device 2 may be detected by the device 42, for example, by processing one or more image or video sequences of the processing device 2 to identify attachments on the processing device 2. In one example, before using the processing device 2, an image of the orifice 10 is obtained by an imaging unit separate from the processing device 2, and the processing unit 46 may determine the size of the orifice 10 by analyzing the image of the orifice 10. The size of the orifice 10 may be determined by extracting the size of the window / orifice 10 using processing techniques such as edge detection and (rectangular) shape fitting. Alternatively, the device 42 may be able to receive signals indicating attachments attached to the processing device 2. Processing device 2 can provide a signal in response to detecting the attachment type attached to processing device 2. Processing device 2 can detect the attachment type in a variety of different ways, such as automatically based on signals exchanged between the body of processing device 2 and the attachment, or based on user setting input to processing device 2 indicating the type of attachment or the type of body part to be processed. Alternatively, different attachments can have orifices 10 of different colors, and when a light pulse is applied to a body part, device 42 can identify the attachment in use by observing the color of the light pulse. Once the attachment is identified or otherwise known, information about the size of the orifice 10 associated with the identified attachment is obtained, for example, from memory unit 48, and used to determine the size of the previously processed area.

[0078] Steps 103 and 105 can be performed on multiple (or all) previous light pulses during the processing operation. In a preferred embodiment, steps 103 and 105 are performed when or shortly after a light pulse is applied to a body part, while feedback is provided to the user during the processing operation so that the user can determine where to position the processing device 2 for the next light pulse. Thus, steps 103 and 105 can be performed in response to determining that a light pulse has been applied to a body part by the processing device 2. The processing unit 46 can determine that a light pulse has been applied to a body part in one (or more) different ways. In embodiments where the device 42 is part of the processing device 2, the processing unit 46 can receive a signal indicating that a light pulse has been generated from the light source 12 or from the control circuitry of the light source 12. Alternatively, when the device 42 is separate from the processing device 2, the processing unit 46 can receive a signal indicating that a light pulse has been generated from the processing device 2. Alternatively, when the sensor 44 is an imaging unit, or when the imaging unit is otherwise disposed in the system 40, image or video sequences from the imaging unit can be processed to identify the light pulse. For example, Figure 5 shows two images 66 of a body part 68 obtained by the imaging unit. In image 66 (labeled 66-1) of Figure 5(a), a portion of the processing device 2 is visible along with the skin of the body part 68. When this image is acquired, the processing device 2 does not apply a light pulse to the body part 68. When image 66 (labeled 66-2) of Figure 5(b) is acquired, the processing device 2 applies a light pulse to the body part 68, and this is visible through the brighter top portion of image 66-2. Therefore, in an embodiment, the processing unit 46 can be configured to process an image or video sequence to identify the occurrence of a light pulse. The occurrence of a light pulse can be identified by calculating the average brightness of each image (or series of images) and comparing the average brightness to identify whether any image is significantly brighter (e.g., brighter than a threshold amount). Alternatively, the average brightness of each image or series of images can be compared with a threshold amount to determine whether the brightness is consistent with the light pulse that has been applied. In some embodiments, the threshold amount can be adjusted based on ambient lighting conditions. The advantage of these embodiments is that the device 42 does not need to be communicatively connected to the processing device 2, and the device 42 can be used with conventional processing devices 2.

[0079] In step 107, processing unit 46 processes the first measurement signal to estimate the current position of processing device 2 relative to the body part. When a light pulse is generated or applied, the processing in step 107 can be the same as the processing by processing unit 46 in step 103 for estimating the position of processing device 2 relative to the body part.

[0080] In step 109, based on the current position of processing device 2 estimated in step 107, processing unit 46 estimates the skin region of the body part to which the light pulse will be applied at the current position of processing device 2. This estimated region is referred to as the "current processing region". The processing in step 109 can be the same as that used by processing unit 46 in step 105 to estimate the skin region of the body part to which the previous light pulse was applied. As in step 105, in step 109, when estimating the size of the current processing region, the size of the aperture 10 can be taken into account.

[0081] In step 111, processing unit 46 generates a feedback control signal for feedback unit 45. The feedback control signal is generated such that it causes feedback unit 45 to generate feedback indicating whether the current processing region corresponds to or substantially corresponds to a previous processing region. The current processing region corresponding to or substantially corresponding to a previous processing region can be defined as the current processing region overlapping with at least X% of one or more previous processing regions. X can be, for example, 20% (i.e., the current processing region substantially corresponds to a previous processing region when at least 20% of the current processing region overlaps with one or more previous processing regions), 50%, or a value in the range of 20% to 50%. In some embodiments, X can be a small value (e.g., in the range of 1% to 5%) to substantially prevent any overlap between the current processing region and any previous processing region.

[0082] The method may further include providing the generated feedback control signal to the feedback unit 45 to provide feedback to the user. As described above, the feedback unit 45 may be configured to provide any combination of visual, auditory, and tactile feedback to indicate whether the processing device 2 is at a position where a light pulse has previously been applied. The user of the processing device 2 will be able to use these indications to determine whether to trigger a light pulse at the current position of the processing device 2, which can help the user improve the coverage of the processing operation on the subject's skin.

[0083] For example, if it is determined that the current processing area overlaps with or excessively overlaps with a previously processed area, a feedback control signal can illuminate red light (e.g., on processing device 2). Similarly, if it is determined that the processing area does not overlap with or excessively overlaps with a previously processed area, a feedback control signal can illuminate green light (e.g., on processing device 2). As another example, red / green light can be replaced by (or accompanied by) tactile feedback (e.g., vibration of processing device 2) to indicate whether the current processing area overlaps with or excessively overlaps with a previously processed area. As another example, red / green light can be replaced by (or accompanied by) auditory feedback (e.g., noise, auditory messages, etc.) to indicate whether the current processing area overlaps with or excessively overlaps with a previously processed area.

[0084] In a preferred embodiment, the feedback unit 45 is a display unit, such as a display screen, a transparent panel for providing an AR display, or a projector. In these embodiments, the feedback signal is a display control signal that causes the display unit to generate a graphic display including a graphic representation of a previously processed area and a currently processed area superimposed on the body part.

[0085] Figure 6 This is an illustration of visual feedback that can be provided by a display unit according to embodiments of the technology described herein. In these embodiments, the visual feedback 70 includes a first portion 72, which is an image or (live) video sequence of a body part 74 obtained using an imaging unit (which may be a sensor 44 for determining the position and motion of the processing device 2, or a separate imaging unit where one or more sensors 44 are motion sensors). Alternatively, the first portion 72 may include a graphical representation of the body part 74. Figure 6 In this context, body part 74 is the subject's forearm. In the first part 72, one or more graphic representations 76 of the previously processed area are superimposed on the image or video sequence of body part 74. The previously processed area 76 is superimposed on the relevant location on body part 74 where those light pulses were applied. Therefore, when generating the display control signal, the processing unit 46 determines where the previously processed area 76 should be located in the visual feedback 70 based on the estimated current position of the processing device 2 relative to each estimated previously processed location. Figure 6 Several pre-processed regions 76 are marked, specifically pre-processed regions 76-1, 76-2 and 76-3 (which partially overlap each other) and pre-processed region 76-4.

[0086] Figure 6 The visual feedback 70 also includes a second part 76. This is in the case where the imaging unit is located on the processing device 2 (e.g., as...). Figure 1 As shown), when the processing device 2 comes into contact with the skin, the imaging unit will conventionally be unable to observe a specific portion of the skin (and which corresponds to the current processing area), and may observe some adjacent skin depending on the field of view of the imaging unit, including whether a portion of the processing device 2 obscures the view of the imaging unit. The second portion 76 of the visual feedback 70 is used to display a graphical representation 80 of the current processing area, and a graphical representation 76 of any previously processed areas within the obscured area represented by the second portion 78. Therefore, the previously processed areas 76-5 are displayed on the boundary between the first portion 72 and the second portion 78 (i.e., the previously processed areas 76-5 are partially in the first portion 72 and partially in the second portion 78). The previously processed area 76-6, entirely within the second portion 78, is shown.

[0087] Figure 7An exemplary visual feedback 70 is shown, which can be provided according to embodiments of the technology described herein. Figure 7 The images provided in the image are related to Figure 6 The embodiments shown provide consistent feedback, and therefore Figure 7 Visual feedback 7 is shown, which has a first part 72 (video sequence of arm 74), several previously processed areas 76 on arm 74 in the first part 72, and a second part 78 showing a graphical representation 80 of the currently processed area.

[0088] In embodiments where the display unit is a projector rather than a display showing a sequence of images or videos of body parts with graphical representations of previously processed and currently processed areas, the projector can project the graphical representation directly onto the body part at an appropriate location. In this way, the user can see the previously processed area directly on the body part, making it easier for them to determine where the next light pulse should be applied.

[0089] In embodiments where the display unit is a transparent panel for providing an AR display, the transparent panel can be positioned within the user's line of sight, directly between the user and the body part being processed, allowing the user to view the body part through the transparent panel. In some embodiments, the transparent panel is one or both lenses of a pair of glasses. Graphical representations of the previously processed area and the currently processed area can be projected or displayed on the transparent panel such that, from the user's field of vision, they are superimposed on the correct position of the body part. This presents the user with an AR view of the body part, making it easier for the user to see the previously processed areas on the body part.

[0090] In embodiments where feedback is provided during the use of processing device 2 (e.g., when processing device 2 moves over a body part and applies light pulses), steps 101-111 can be performed generally continuously. That is, in step 101, measurement signals can be received generally continuously (e.g., according to the sampling rate of sensor 44, or according to the transmission rate from sensor 44 to device 42 (e.g., packet rate)), and steps 103-111 can be performed generally continuously to update the feedback to be provided to the user.

[0091] In embodiments where sensor 44 is an imaging unit attached to processing device 2 and processing unit 46 processes image or video sequences in steps 103 and 105 to identify skin features and / or body part features in the image or video sequence, processing unit 46 may use one or more of a variety of different techniques to process the image or video sequence. In some embodiments, skin features may be detected using feature detection algorithms such as the Shi-Tomasi angle detection method and / or the scale-invariant feature transform (SIFT) method. Alternatively, AI-based feature detection techniques optimized for skin features may be used. To estimate the motion of processing device 2 over time, the position of detected skin features in an image or video sequence can be tracked over time by matching skin features between consecutive images or consecutive video frames. Homography between two sets of features can be determined by calculating the homography between two sets of features in two consecutive images or video frames. Homography determines the relationship between the positions and orientations of two imaging units when viewed on the same plane.

[0092] Over time, drift will occur in the location calculation at the current processing location due to the accumulation of errors in measurement and / or processing techniques. To reduce or avoid this drift, skin feature detection and motion tracking can be extended beyond two image or video frames, thereby tracking skin features across multiple image or video frames.

[0093] Based on the body parts shown in the image, image processing techniques suitable for tracking the position of skin features on non-planar surfaces, such as Simultaneous Localization and Mapping (SLAM), can be used for homography calculation.

[0094] Features or subjects visible in the background of an image or video sequence may cause errors when tracking skin features or the position of processing device 2. To reduce these errors, images can be processed to identify and segment skin regions from the surrounding background. Image portions associated with background regions are discarded and / or not used for skin feature tracking.

[0095] Rapid movement of the processing device 2 or imaging unit 44 may cause blurring of the acquired image and a reduction in the number of detectable skin features. This may result in the inability to determine and track the position of the processing device 2. Image blurring can be reduced by using imaging unit 44 at a faster / higher frame rate and a lower exposure time.

[0096] In some embodiments, face tracking failures and / or drift errors can be recovered by using loop closure. This can be achieved by identifying skin regions based on previously identified skin features that have moved outside the view of imaging unit 44, thus resetting the drift to its position. Therefore, device 42 can retain information about the skin features identified in the image or video sequence and use this information to determine whether the skin region has been previously identified and whether it has been previously processed with light pulses when identifying skin features in the next image or video sequence.

[0097] In some embodiments, if the current processing position of the processing device 2 does not correspond to or substantially does not correspond to a previous processing area, the processing device 2 can be configured to automatically trigger a light pulse. In this case, the device 42 can be configured to automatically trigger the processing device 2 to generate a light pulse. In some embodiments, one or more additional conditions can be applied to trigger the light pulse, such as the processing device 2 being in contact with skin, and / or the skin tone in contact with the processing device 2 being suitable for receiving a light pulse. If a light pulse can be applied at the current position of the processing device 2, the processing unit 46 can determine an actuation signal to actuate the light source 12, or an actuation signal that causes the processing device 2 to actuate the light source 12.

[0098] In embodiments where one or more images of which light pulses are visible are obtained, processing unit 46 may also be configured to process the images to determine the brightness and / or color of the light pulses. Processing unit 46 may compare the determined brightness and / or color of the light pulses with each other to determine whether the brightness and / or color of the light pulses has changed over time. Any change in the brightness and / or color of the light pulses over time (e.g., a decrease in brightness) can provide an indication of the health and / or decay of light source 12. If the brightness and / or color meet a threshold criterion, an indication may be provided to the user that light source 12 may need to be replaced (if light source 12 is replaceable), or that processing device 2 itself needs to be replaced.

[0099] In some embodiments, the feedback provided to the user may include feedback indicating whether the current processing area corresponds to or substantially corresponds to a previously processed area (including those previously processed areas in previous processing operations). In these embodiments, the previously processed positions and regions of the light pulses applied during the processing operation are estimated following the method described above, and information about these previously processed positions and regions is stored, for example, in memory unit 48. During the next processing operation, this information is retrieved and used to form the feedback control signal in the next processing operation. For example, the feedback may indicate the previously processed region from the previous processing operation and the previously processed region from the current processing operation.

[0100] Therefore, a method and apparatus are provided for providing feedback to the user of the processing device regarding the processing operations performed on a body part of a subject.

[0101] Those skilled in the art, by practicing the principles and techniques described herein and studying the accompanying drawings, disclosure, and appended claims, can understand and implement variations of the disclosed embodiments. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude multiple. A single processor or other unit can implement the functions of several items as described in the claims. The fact that certain measures are recited in mutually different dependent claims does not imply that combinations of these measures cannot be advantageously used. Computer programs may be stored or distributed on suitable media, such as optical storage media or solid-state media provided with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. Any reference numerals in the claims should not be construed as limiting the scope.

Claims

1. An apparatus for use with a processing device, wherein the processing device is configured to apply light pulses to the skin of a user's body part to perform the processing operation, wherein the light pulses applied to the skin process the skin area, the apparatus comprising a processing unit configured to: A first measurement signal is received from a first sensor, the first measurement signal including information about the position and / or movement of the processing device over time; For a light pulse previously applied to the body part by the processing device during the processing operation, the first measurement signal is processed to estimate the previous processing position as the position of the processing device relative to the body part when the light pulse was generated; Based on the estimated previous processing location, a previous processing region is estimated for the previously applied light pulse, the previous processing region corresponding to the skin area of ​​the body part to which the light pulse was applied when the processing device was at the previous processing location; The first measurement signal is processed to estimate the current position of the processing device relative to the body part; Based on the estimated current position of the processing device, a current processing area is estimated, the current processing area corresponding to the skin area where a light pulse will be applied when the processing device is at the current position; as well as Generate a feedback control signal for the feedback unit, wherein the feedback control signal is configured to cause the feedback unit to generate feedback indicating whether the current processing area corresponds to or substantially corresponds to a previous processing area, and to provide the generated feedback to the user during the processing operation.

2. The apparatus of claim 1, wherein the sensor is an imaging unit, the first measurement signal is a plurality of images or video sequences, and the plurality of images or video sequences are processed to identify the position and / or motion of the processing device over time.

3. The apparatus of claim 2, wherein the processing unit is configured to: Identify skin features and / or body part features in the image or video sequence; Monitoring the movement of skin features and / or body part features in the image or video sequence over time; and The position and / or movement of the processing device relative to the body part is determined based on the movement of the skin features and / or body part features in the image or video sequence.

4. The apparatus of claim 2 or 3, wherein the processing unit is further configured to: receive a second measurement signal from a motion sensor, the second measurement signal including information about the position and / or movement of the processing device over time; and The processing unit is configured to process the first measurement signal and the second measurement signal to estimate the previously processed position and the current position.

5. The apparatus according to any one of claims 1-4, wherein the feedback control signal is a display control signal, and the feedback unit is a display unit.

6. The apparatus of claim 5, wherein the display control signal is configured to cause the display unit to generate a graphic display, the graphic display including a graphic representation of the previously processed area and the currently processed area superimposed on the body part.

7. The apparatus of claim 6, wherein any one of the following: The display unit is a display screen, and the display control signal includes multiple image or video sequences superimposed with the graphic representation of body parts; The display unit is a projector, and the display control signal is configured to cause the projector to project the graphic representation onto the user's body part based on the estimated previously processed area and the currently processed area; and The display unit is configured to display the graphic representation as part of an augmented reality display of the body part.

8. The apparatus according to any one of claims 1-7, wherein the processing unit is configured to estimate the previously processed region and the current processed region based on: (i) the size of the aperture in the processing device through which the light pulse passes; and (ii) the size of the aperture in an attachment to the processing device.

9. The apparatus according to any one of claims 1-8, wherein the processing unit is further configured to: If the determined current processing area does not overlap or substantially does not overlap with any of the previous processing areas, an actuation signal is generated, which will cause the light source in the processing device to generate light pulses.

10. The apparatus according to any one of claims 1-9, wherein the processing unit is further configured to: estimate the previous processing region and the current processing region based on a predetermined relationship between the position of the aperture in the processing device and the estimated previous processing position and the estimated current position of the processing device, respectively, wherein the light pulse is applied to the skin through the aperture.

11. A system comprising: A processing device, including one or more light sources, for performing processing operations on the skin; A first sensor is configured to output a first measurement signal, the first measurement signal including information about the position and / or movement of the processing device over time; as well as The apparatus according to any one of claims 1-10.

12. A computer-implemented method for providing feedback to a user, the feedback relating to a processing operation performed by a processing device on a body part of the user, wherein the processing device is configured to apply light pulses to the skin of the body part to perform the processing operation, wherein the light pulses applied to the skin treat an area of ​​the skin, the method comprising: A first measurement signal is received from a first sensor, the first measurement signal including information about the position and / or movement of the processing device over time; For a light pulse previously applied to the body part by the processing device during the processing operation, the first measurement signal is processed to estimate the previous processing position as the position of the processing device relative to the body part when the light pulse was generated; Based on the estimated previous processing location, a previous processing region is estimated for the previously applied light pulse, the previous processing region corresponding to the skin area of ​​the body part to which the light pulse was applied when the processing device was at the previous processing location; The first measurement signal is processed to estimate the current position of the processing device relative to the body part; Based on the estimated current position of the processing device, a current processing area is estimated, the current processing area corresponding to the skin area where a light pulse will be applied when the processing device is at the current position; as well as Generate a feedback control signal for the feedback unit, wherein the feedback control signal is configured to cause the feedback unit to generate feedback, the feedback indicating whether the current processing area corresponds to or substantially corresponds to a previous processing area, and The generated feedback is provided to the user during the processing operation.

13. The method of claim 12, wherein the sensor is an imaging unit, the first measurement signal is a plurality of images or video sequences, and the plurality of images or video sequences are processed to identify the position and / or motion of the processing device over time.

14. The method of claim 13, wherein the method further comprises: Identify skin features and / or body part features in the image or video sequence; Monitoring the movement of skin features and / or body part features in the image or video sequence over time; and The position and / or movement of the processing device relative to the body part is determined based on the movement of the skin features and / or body part features in the image or video sequence.

15. A computer program product comprising a computer-readable medium having computer-readable code implemented therein, the computer-readable code being configured to cause the computer or processing unit, when executed by a suitable computer or processing unit, to perform the method according to any one of claims 12-14.

Images