Assessment of target surfaces and markings thereon

Abstract

A method of assessing concentration of a marking on a target surface, the method comprising: acquiring at least one image of the marking on the target surface; analyzing one or more marking characteristics of a plurality of pixels associated with the marking in the acquired image; determining a concentration score for one or more of the plurality of pixels based on the analyzed one or more marking characteristics; and assessing an overall concentration score of the marking based on the determined concentration score for determining one or more actions to take for removing the marking from the target surface.

Description

ASSESSMENT OF TARGET SURFACES AND MARKINGS THEREONRELATED APPLICATIONS

[0001] This application claims priority to Australian Provisional Patent Application No. 2024901982 filed on 28 June 2024 and Australian Provisional Patent Application No. 2024901983 filed on 28 June 2024, the entire contents of which are incorporated by reference herein in their entireties.TECHNICAL FIELD

[0002] The present disclosure generally relates to assessment of marking(s). More specifically, the present disclosure relates to assessment of a marking on a target surface for determining one or more actions to take for removing the marking from the target surface.

[0003] The present disclosure further generally relates to estimation of skin. More specifically, the present disclosure relates to estimation of skin to determine the type of skin.BACKGROUND

[0004] Embedding a marking on a target surface, for example a tattoo on skin, may be intended to be permanent. However, some owners of the marking (i.e. the person on whose body the marking is embedded) may want to get the marking removed for various reasons such as, but not limited to, regretting / being embarrassed of the presence of the marking, social anxiety, limitations on career progression or private relationships, or a negative impact on the social status of the owner. The procedure to remove markings is non-trivial due to the intended permanency and possible detrimental side effects such as pain, damage to skin health, impact on immune response and / or overall health of the owner etc.

[0005] Currently, assessing concentration of a marking i.e., type of ink particles of which the marking is made up, in the target surface is a manual process. The industry standard “Kirby Desai scale” requires a human assessor to assign points in 6 different categories, where each category can have points ranging from a minimum of 0- 1 to a maximum of 2-6 points. The total number of points assigned across the 6 categories would then become the basis for assessing the concentration of the marking. Such processes suffer from several disadvantages such as, the assessment being manual and therefore highly subjective since different assessors may assign points differently or the same assessor may assign different points at different times for the same marking, receiving an assessment is an involved and non-private process, scaling the execution of initial assessments before makingan appointment for removal of the marking can be cumbersome due to the involvement of human practitioners or assessors.

[0006] Skin of humans, animals etc. that are subjected to laser-based applications may respond differently due to difference in type of the skin. For e.g., one type of skin may be more sensitive than another type of skin when subjected to the same operational parameters of the laser device.

[0007] Laser-based applications for skin may include, but not limited to, removal of hair, tattoo, marks on skin such as birthmarks, wrinkles, keloids, scars, vascular lesions, pigmented lesions, sundamaged skin and so on. Laser-based removal procedures are non-trivial due to several reasons such as the intended permanency of the mark (e.g., tattoo), possible detrimental side effects such as pain, damage to skin health, impact on immune response and / or overall health of the owner etc. due to variation in sensitivity of different types of skin.

[0008] Currently available methods and systems for estimating types of skin suffer from one or more disadvantages such as high cost of specialized equipment, which can make deployment in smaller (low volume) facilities uneconomical, requirement of physical presence of the person whose skin is to be examined to perform estimation of skin type by human assessors, requirement of presence of a trained health practitioner, which may limit the location and time in which an estimate is done, and which is considered privacy invasive.

[0009] It is desirable to overcome or ameliorate one or more of the above-described difficulties, or to at least provide a useful alternative.SUMMARY

[0010] According to an aspect, there is provided a method of assessing concentration of a marking on a target surface, the method comprising: acquiring at least one image of the marking on the target surface; analysing one or more marking characteristics of a plurality of pixels associated with the marking in the acquired image; determining a concentration score for one or more of the plurality of pixels based on the analysed one or more marking characteristics; and assessing an overall concentration score of the marking based on the determined concentration score for determining one or more actions to take for removing the marking from the target surface.

[0011] According to another aspect, there is provided a system for assessing concentration of a marking on a target surface, the system comprising: an imaging device configured to acquire at least one image of the marking on the target surface; and a concentration determination module communicatively connected to the imaging device and configured to: analyze one or more marking characteristics of a plurality of pixels associated with the marking in the acquired image; determinea concentration score for one or more of the plurality of pixels based on the analyzed one or more marking characteristics; and assess an overall concentration score of the marking based on the determined concentration score for determining one or more actions to take for removing the marking from the target surface.

[0012] According to an aspect, there is provided a method of estimating a type of target surface, the method comprising: acquiring at least one image of the target surface; determining a plurality of target surface pixels associated with the target surface in the acquired image; determining a surface characteristic for the target surface based on the determined plurality of target surface pixels; assigning a subgroup-based score to each of the plurality of target surface pixels based on the determined surface characteristic; and estimating a type of the target surface based on the assigned subgroup-based score and a surface context reference model.

[0013] According to another aspect, there is provided a system for estimating a type of target surface, the system comprising: an imaging device configured to acquire at least one image of the target surface; and a type determination module configured to: determine a plurality of target surface pixels associated with the target surface in the acquired image; determine a surface characteristic for the target surface based on the determined plurality of target surface pixels; assign a subgroup - based score to each of the plurality of target surface pixels based on the determined surface characteristic; and estimate a type of the target surface based on the assigned subgroup-based score and a surface context reference model.

[0014] Assessing the overall concentration score of the marking may comprise aggregating the determined concentration score with historical concentration values.

[0015] One or more surface characteristics of a plurality of pixels associated with the target surface in the acquired at least one image may be analysed.

[0016] Assessing the overall concentration score of the marking may further be based on the analysed one or more surface characteristics.

[0017] Analysing one or more surface characteristics may comprise determining a correspondence between the plurality of pixels associated with the target surface and a surface context reference model.

[0018] In some embodiments, pixel dimensions of a reference object in the acquired at least one image may be detected.

[0019] Pixel dimensions of the marking may be converted into actual dimensions of the marking using the detected pixel dimensions of the reference object and actual dimensions of the reference object.

[0020] In some embodiments, assessing the overall concentration score of the marking may further be based on the actual dimensions of the marking.

[0021] The one or more actions to undertake for removal of the marking may be displayed on a user interface.

[0022] The marking may be a tattoo.

[0023] The target surface may be skin of a person or an animal.

[0024] In some embodiments, the marking pixels may be segmented from the surface pixels in the at least one image using the analyzed one or more surface characteristics.

[0025] In some embodiments, the one or more actions may comprise predicting a number of sessions required for removing the marking from the target surface.

[0026] In some embodiments, the one or more actions may comprise predicting a cost estimate for removing the marking from the target surface.

[0027] In some embodiments, the one or more actions may comprise displaying on the user interface one or both of a session time and a location to make an appointment for removing the marking.

[0028] Determining a plurality of target surface pixels may further comprise determining an initial surface score for each of the plurality of pixels in the acquired image.

[0029] Determining a plurality of target surface pixels may further comprise categorizing each of the plurality of pixels of the acquired image having the initial surface score greater than a predetermined threshold as the plurality of target surface pixels.

[0030] A plurality of owner pixels in the acquired image associated with a target owner of the target surface may be determined.

[0031] Determining a plurality of target surface pixels may further comprise determining an initial surface score for each of the plurality of owner pixels in the acquired image.

[0032] Determining a plurality of target surface pixels may further comprise categorizing each of the plurality of owner pixels having the initial surface score greater than a predetermined threshold as the plurality of target surface pixels.

[0033] A surface characterization score based on the determined surface characteristic may be determined.

[0034] Determining the surface characterization score may comprise aggregating the surface characteristic of the plurality of target surface pixels.

[0035] The target surface may be skin.

[0036] The surface context reference model may be Fitzpatrick scale.

[0037] The surface characteristic may be colour distribution.

[0038] The surface characterization score may be Individual Typology Angle (IT A) score.BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Preferred embodiments of the present disclosure are hereafter described, by way of nonlimiting example only, with reference to the accompanying drawings, in which: a. Fig. 1 is a flow diagram of a method of assessing concentration of a marking on a target surface according to an embodiment of the present disclosure; b. Fig. 2 is an exemplary target surface having a marking thereon; c. Fig. 3 is a system for performing the method of Fig. 1 according to an embodiment of the present disclosure; d. Fig. 4 is a view for explaining assessment of concentration according to an embodiment of the present disclosure; e. Fig. 5 is a flow diagram of a method of assessing concentration of a marking on a target surface according to another embodiment of the present disclosure; f. Fig. 6 shows an exemplary imaging guidance view for assisting a user to acquire an image of the marking on the target surface; g. Fig. 7 is a flow diagram of a method of assessing concentration of a marking on a target surface according to yet another embodiment of the present disclosure; h. Fig. 8 shows an exemplary imaging guidance view for assisting a user to acquire an image of the marking on the target surface, the image including a reference object therein; i. Fig. 9A, 9B and 9C show selection of part(s) of the marking to be removed; j. Fig. 10A, 10B are exemplary diagrams showing a marking on skin on upper arm of a human in overview and zoomed-in views respectively; k. Fig. 11 A, 11B are exemplary diagrams showing a marking on skin on ankle of a human in overview and zoomed-in views respectively;l. Fig. 12 is a flow diagram of a method of estimating a type of target surface according to an embodiment of the present disclosure; m. Fig. 13 shows an exemplary target surface of a target owner; n. Fig. 14 is a system for performing the method of Fig. 12 according to an embodiment of the present disclosure; and o. Fig. 15 is a view for explaining estimation of type of target surface according to an embodiment of the present disclosure.DETAILED DESCRIPTION

[0040] An exemplary target surface 204 having a marking 202 thereon is shown in Fig. 2. In one example, the marking 202 may be a tattoo and the target surface 204 may be skin on a body part of a person and the marking 202 may be applied on the target surface 204 by inserting ink pigment into the tissue or dermis of the target surface 204 or by any other suitable application process. In another example, the marking 202 may be a brand or logo mark applied on a target surface 204 such as the hide of cattle and the marking 202 may be applied on the target surface 204 using heat or any other suitable application process.

[0041] A target surface 204 may have one or more markings 202. One or more markings 202 may be applied, embedded or added to the target surface 204 with an intention for the one or more markings 202 to be permanent. One or more embodiments of the present disclosure is directed towards assessing a concentration of the one or more markings 202 on a target surface 204 for determining one or more actions to be taken for removing or diminishing the appearance of the marking 202 to an extent that it is not easily visible to a naked eye. It will be appreciated that a marking 202 may fade over time i.e., the type of the ink particles (concentration) of the marking 202 may reduce or change over time due to various factors such as, but not limited to, exposure of the marking 202 to sun, type of target surface 204, location of the target surface 204 having the marking 202 on the body of the person or being, colour of the ink particles. A marking 202 with a high concentration value may require more time, cost and / or processes for the marking to be removed from the target surface 204. A marking 202 with a lower concentration value on the other hand may require lesser time, cost and / or processes for the marking to be removed from the target surface 204.

[0042] The concentration of the marking 202 may be defined as the type of ink particles (which may vary per ink colour) in the target surface and may impact one or more parameters associated with the removal / reduction process (e.g., number of sessions, cost, time per session, total time overthe total number of sessions etc.) i.e., process required to remove or diminish or reduce the visibility of the marking without causing or reducing detrimental side effects (e.g., pain, change in sensitivity of the target surface 204, health of the target surface 204, impact on immune response of the owner of the marking 202 and / or general health of the owner). It will be appreciated that the terms "removal", "removing" or variations thereof, of the marking 202 may be defined as removing the marking 202 partially or completely or diminishing or reducing the visibility of the marking 202 to a naked eye or diminishing or reducing the visibility of the marking 202 such that the marking can then be covered with a newer marking. It will further be appreciated that "removal", "removing" or variations thereof, of the marking 202 may refer to the whole marking 202 or portions of the marking 202. In another example, where the marking 202 is a skin burn, the concentration of the marking may be defined as a degree of burn, i.e. the extent to which it has affected the epidermis and dermis of the skin. The deeper the affected layer is, and the more the extent of the damage, the higher the concentration may be. In some examples, bum marks may be reduced with spray-on skin cells.

[0043] The marking 202 (e.g., tattoo) may be removed using laser treatment, where one or more parameters of the laser (e.g., wavelength, beam spot diameter, fluence) is set such that the ink molecules may be broken down to be small enough to be disposed of by the human immune system. The required wavelength for the laser may depend on one or more factors such as, but not limited to, a type of ink used, an age of the marking 202 (e.g., how much time has passed since the application of the marking 202 on the target surface 204). The laser fluence may depend on one or more factors such as, but not limited to, a type of target surface 204.

[0044] In one example, a 1064 nanometre (nm) laser wavelength may be safely used on any skin type. This wavelength may be sufficient for removal of darker inks (e.g., black, dark blue), and may diminish lighter-coloured inks as well but to a lesser degree. A 532 nm laser wavelength may be applied to remove red or orange colours (particularly on lighter skin tones). A 730 nm laser wavelength may be applied to remove blue and / or green colours (particularly for medium skin tones). It will be appreciated that suitable adjustments to the laser wavelength may be made based on the type of the target surface 204 to minimize adverse effects. It will be appreciated that the wavelengths of laser to be used for removing the marking 202 may be subject to compliance guidelines determined by the health departments of respective jurisdictions (e.g., according to USA Food & Drug Administration (FDA) guidelines as well as Conformite Europeenne (CE) compliance guidelines). Different skin types, marking embedding techniques and / or ink pigmentation / colours may result in different absorption coefficient (cm1) of the ink of the marking on the target surface. Laser pulses with ultra-short picosecond duration may create a rapid photoacoustic effect within thetarget surface 204 while gently impacting outer layers of the target surface 204. Certain types of laser applicators may target ink particles of the marking 202 while leaving the outermost layer of the target surface 204 intact or hardly damaged.

[0045] The marking removal or reduction operator may consider the type / tone of the target surface 204 of the owner as guidance to set a laser beam spot diameter and select a treatment fluence within a manufacturer prescribed margin for the diameter and the type / tone of the target surface 204 and other indicators such as the heritage of the person. In an example, the operator may select a spot diameter of 6mm for an owner with a skin type in band 2 of a standardized skin type schema. The operator may then pick a treatment fluence of 1.4 Joules per square centimetre (J / cm2) and apply the setting for treatment of a small patch of skin. Depending on the response of the skin, the operator may then increase or decrease the diameter, and consequently decrease or increase the treatment fluence, for another patch of skin. Once a good response is found, the operator may use the setting for the specific reduction iteration or treatment. The spectrum of skin tones may be typically indicated by the dermatological Fitzpatrick scale with six classes from light skin (I) to dark skin (VI). Lighter skin tones may be associated with people from Nordic heritage (often correlating with red hair and freckles), while darker skin tones may be associated with people having heritage from regions near the equator.

[0046] Fig. 1 is a flow diagram of a method 100 of assessing concentration of a marking 202 on a target surface 204 according to an embodiment of the present disclosure. The method 100 comprises acquiring at least one image of the marking 202 on the target surface 204 (step 102), analyzing one or more marking characteristics of a plurality of pixels associated with the marking 202 in the acquired image (step 104), determining a concentration score for one or more of the plurality of pixels based on the analyzed one or more marking characteristics (step 106) and assessing an overall concentration score of the marking 202 based on the determined concentration score for determining one or more actions to take for removing the marking 202 from the target surface 204 (step 108). The steps will be described in more detail below.

[0047] Fig. 3 is a system 300 for performing the method of Fig. 1 according to an embodiment of the present disclosure. The system 300 comprises an imaging device 302 and a concentration determination module 304 as will be described in more detail below.Image Acquisition

[0048] At step 102 of the method 100, at least one image of the marking 202 on the target surface 204 may be acquired using an imaging device 302. In one embodiment, the imaging device 302 maybe a mobile phone. In another embodiment, the imaging device 302 may be a camera. In a further embodiment, the imaging device 302 may be a scanner and the at least on image may be acquired after a body part pertaining to the target surface 204 having the marking 202 thereon is scanned by the scanner.

[0049] The at least one image may be in JPEG (Joint Photographic Experts Group) format, HEIC (High Efficiency Image Container) format or any other suitable format.

[0050] In some embodiments, the step 102 may comprise acquiring one or more image sequences, such as videos (e.g., in H264, MPEG-4, M-JPEG or any other suitable formats). In further embodiments, the step 102 may comprise acquiring a plurality of images (i.e., burst images) in JPEG, HEIC or any other suitable formats. The acquired plurality of images may depict different parts of the marking 202 in which case, the plurality of images may be stitched into a single composite image for subsequent concentration assessment (described below). In an embodiment, the plurality of images may be stitched using scale-invariant feature transform which may characterize local features of an image. The area of each marking 202 (in the plurality of images) may first be detected and segmented, after which the SIFT key points for each segment may be computed. The SIFT key points may then be compared and clustered with a Hough Transform. The transformation matrix between the two image segments may then be computed using linear least squares. With the transformation matrix, the image segments can be represented in the same image space for further processing. In another embodiment, the plurality of images may be stitched into a single composite image using Hough Transform.

[0051] In another embodiment, each of the plurality of acquired images depicting different parts of the marking 202 may be analysed individually, and an overlap between the images may be aggregated (e.g. by taking an average, maximum, minimum, or weighted averaging using suitable weights (e.g., 0.5)) for concentration assessment. It will be appreciated that any amount of overlap that can have a noticeable impact on the end result would be sufficient. In an embodiment where each of the plurality of images depicts the whole marking 202, each image may be analysed individually, and resulting marking characteristics (described in more detail below) may be aggregated (e.g., by taking an average, maximum, minimum, or weighted averaging using suitable weights (e.g., 0.5)). In another embodiment, one of the plurality of images may be selected for concentration assessment (e.g., by measuring blurriness of the plurality of images and selecting the least blurry image). Blurriness may be measured by convolving the image convolved with a Laplacian kernel ([[0,-l,0], [- l,4,-l],[0,-l,0]]), to measure a second derivative of the image. If thevariance of the response is below a threshold (e.g., 100), the image may be considered to be blurred due to an absence of clear edges.

[0052] In some embodiments, at least two images may be acquired at step 102, one of the images being an overview shot of the marking 202 and the target surface 204, the other image being a zoomed in shot of the marking 202 (e.g., including less target surface 204 than in the overview shot). Fig. 10A, 10B are exemplary diagrams showing a marking on skin on upper arm of a human in overview and zoomed-in views respectively. Fig. 11 A, 11B are exemplary diagrams showing a marking on skin on ankle of a human in overview and zoomed-in views respectively. In one or more embodiments, the plurality of images may be captured at the same angle from the marking 202. In some embodiments, the plurality of images may be captured at different angles from the marking 202. In some embodiments, the one or more images may be captured at an angle between 0 degrees and 30 degrees from the marking 202, where 0 degrees is when the imaging device 302 is positioned top-down onto the target surface 204. In one embodiment, the overview image (e.g., 10A, 11A) may be used to determine a size of the marking 202 (described in more detail below) and the zoomed-in image (e.g., 10B, 11B) may be used to analyse one or more marking characteristics of the marking 202. The determined size may then be used to scale the marking characterization. In one embodiment, the overview image may be used to determine the part of the body having the marking. The body part may impact the number of removal sessions or iterations required, since the farther away the marking is from the body’s lymph nodes (i.e., key part of the immune system), the more number of removal sessions that may be required. In an example, a marking on a foot may require 4 removal sessions more to remove than a marking on the neck.

[0053] In an embodiment, prior to acquiring the at least one image (step 102), a person requiring removal of the marking 202 (e.g., the person or owner on whose body the marking 202 is present, the person in charge of cattle from whose hide the marking 202 is to be removed) may be presented with a list, images, 3D model or the like to select the relevant body part from, on a display screen of a user device. In one example, the display screen may be a display screen of a mobile phone, a computing device or the like. In one or more embodiments, the user device may be the same as the imaging device 302. In other embodiments, the user device may be communicatively connected to the imaging device 302. A shape or outline of the body part selected by the person on the user device may then be displayed on the display screen which can provide guidance when acquiring the at least one image using the imaging device 302 (e.g., to check if the relevant body part is centered in the image, sufficient context (e.g., a margin of at least 15% of the image at each of the 4 sides surrounding the marking) is captured, and / or that a distance between imaging device and the personor owner of the marking 202 is acceptable (e.g., for a marking having dimensions of 10cm by 10cm, a distance of 15cm - 30cm may be acceptable).Concentration Assessment

[0054] The steps of analyzing one or more marking characteristics of a plurality of pixels associated with the marking 202 in the acquired image (step 104), determining a concentration score for one or more of the plurality of pixels based on the analyzed one or more marking characteristics (step 106) and assessing an overall concentration score of the marking 202 based on the determined concentration score for determining one or more actions to take for removing the marking 202 from the target surface 204 (step 108) may be performed by a concentration determination module 304 of the system 300 as will be described below.

[0055] In an embodiment, the system 100 may be a mobile phone comprising the imaging device 302 (e.g., a camera of the mobile phone) and the concentration determination module 304. In another embodiment, the imaging device 302 may be a camera or a scanner communicatively connected to a computing device such as a computer, laptop etc. and having a concentration determination module 304 therein. In an embodiment, the concentration determination module 304 may be hosted in a remote location (e.g., server on a cloud) while the imaging device 302 is at the same or nearby location of the owner of the marking 202, and the imaging device 302 and the concentration determination module 304 are configured to communicate with each other using suitable communication protocols.Pixel classification (pixels associated with the marking and pixels associated with the target surface)

[0056] The at least one acquired image is sent from the imaging device 302 to the concentration determination module 304. It will be appreciated that in the scenario where the imaging device 302 and the concentration determination module 304 are in a single device such as a mobile phone, the acquired image may be stored in the same single device. Alternatively, the acquired image may be stored in a remote location (e.g., a persistent storage of a remote system) to which the imaging device 302 and the concentration determination module 304 are communicatively connected e.g., via the Internet.

[0057] The acquired image is made up of a plurality of pixels. One or more of the plurality of pixels of the acquired image may be associated with the marking 202 while the remaining one or more of the plurality of pixels of the acquired image may be associated with the target surface 204. In other words, a marking 202 may be represented by a set of pixels ("marking pixels") while thetarget surface may be represented by a different set of pixels ("target surface pixels"). In some embodiments, one or more pixels of the acquired image may be neither associated with the marking 202 nor the target surface 204 and may be classified as background or belonging to a reference object (described in more detail below). In one embodiment, any pixels detected to be in the target surface area but having colour(s) that do not match a desired colour of target surface area may be classified as marking pixels.

[0058] The concentration determination module 304 may perform semantic segmentation using an artificial neural network (ANN) configured with model parameters and weights pre-trained for marking detection i.e., detection of the marking 202. At inference time, upon receiving the acquired at least one image as an input, the ANN may output coordinates of areas in the acquired image containing the marking 202, at least in part or as a whole and assign the one or more pixels associated with the marking 202 to a class of marking pixels.

[0059] The concentration determination module 304 may further perform semantic segmentation using an artificial neural network (ANN) configured with model parameters and weights pre-trained for target surface detection, i.e., detection of the target surface 204. At inference time, upon receiving the acquired at least one image as an input, the ANN may output coordinates of areas in the acquired image containing the target surface 204, at least in part or as a whole and assign the one or more pixels associated with the target surface 204 to a class of target surface pixels. In an embodiment, an object detection to detect the owner may be performed (described in more detail below) and any pixels in the area associated with the target surface 204 that do not match a target surface (e.g., skin) may be classified as marking pixel. The pixels classified as marking pixels together form the marking 202.

[0060] In an embodiment, all pixels in the acquired image are considered for classification. In another embodiment, only those pixels near the marking 202 are considered for classification as target surface pixels, e.g., pixels with a distance less than 100 pixels to the closest boundary pixel of marking 202. This approach may mitigate the risk of including in the classification process other target surfaces in the image which may not belong to the owner of the marking 202, but to other people who are irrelevant to the assessment.

[0061] In yet another embodiment, object detection (e.g., detecting the owner of the marking 202) may be performed prior to detection and classification of pixels associated with the marking 202 and the target surface 204. An ANN may be configured with model parameters and weights pre-trained for the owner detection. As owner detection is more generic than detection of marking 202 or target surface 204, a larger training data set may be used. At inference time, upon receivingthe acquired at least one image as an input, the ANN may output coordinates of areas in the acquired image containing the owner of the marking 202. In some further embodiments, the ANN may output a more granular classification (such as man, woman, child etc.). This approach may mitigate the risk of including in the classification process elements in the image having similar colour properties to the target surface 204 (e.g., birch coloured furniture having similar colour as skin). Further detection and classification of pixels associated with the target surface 204 may then be limited to the areas identified as being associated with the owner.

[0062] In an embodiment, the one or more pixels associated with the target surface 204 may be transformed from an original colour space (e.g., RGB (red, green, blue)) to Hue-Saturation- Value (HSV) colour space. It will be appreciated that grouping of similar colours may be easier in HSV colour space than in original colour space. A 3D sub-space in the HSV colour space may then be determined. In an example, 3D sub-space having a hue between 5 and 15, a saturation between 32 and 192, and value (similar to intensity) between 48 and 250 may be determined. Pixels with a HSV value in the sub-space may then be assigned to the target surface pixel class.

[0063] In another embodiment, the one or more pixels associated with the target surface 204 may be transformed from an original colour space (e.g., RGB (red, green, blue)) to Luminance / Chrominance-blue / Chrominance-red (YCbCr) colour space. It will be appreciated that considering brightness information in a different way than colour information may be easier in YCbCr colour space than in the original colour space. A 3D sub-space in the YCbCr colour space may then be determined. In an example, 3D sub-space having a Y between 0 and 255, a Cb between 133 and 173, and Cr between 77 and 127 may be determined. Pixels with a YCbCr value in the subspace may then be assigned to the target surface pixel class.Concentration Score DeterminationAnalysis of Marking Characteristics

[0064] Once a plurality of pixels associated with the marking 202 (marking pixels) is determined, the marking pixels are analyzed to determine one or more marking characteristics. Determining a concentration score for one or more of the plurality of pixels associated with the marking 202 (step 106) may be based on a colour of the pixels.

[0065] In one embodiment, there may be three ordinal classes of colour based on historical concentration statistics. In one embodiment, the historical concentration statistics may be provided by human domain experts who have recorded the concentration for the different colour classes. In another embodiment, a data set with single-coloured images (having the same size, skin type, andbody part) and the corresponding number of sessions (for completing the removal) may be provided. The colours found in the data set images having the same number of sessions may then be grouped. The first class may be for dark colours, i.e., pixels having a low intensity (e.g., pixels in L*a*b* colour space with a* and b* values outside the range of -45 and 45, and L* values less than 160). The second class may be for red colours, i.e. pixels which are in the colour space for red matching the historical concentration statistics for red (e.g., those pixels in L*a*b* colour space with a polar angle between a* and b* in the range of 22.5° and 45°, and L* values in the range of 160 and 232. The third class may be for light colours (e.g., those pixels in L*a*b* colour space with a* and b* values in the range of -45 and 45, and L* values greater than 160).

[0066] A first concentration metric Pl may be set to the highest of the three classes which represents more pixels than a threshold T1 or the equivalent of T1 in physical dimensions. In one example, T1 is 500 pixels. In another example, T1 is 25 square millimetres. In an embodiment, if class 3 exceeds the threshold, Pl=3. If none of the classes meet the threshold, Pl is set to 1.

[0067] In another embodiment, the pixels may be transformed from the original colour space, e.g. RGB (red, green, blue), to the CIELAB colour space (L*a*b*) which may have a closer resemblance to human colour perception than other colour spaces. The three components of L*a*b* are L* representing luminance, a* ranging from red to green, and b* ranging from yellow to blue. Each of the marking pixels may be assigned a colour code. The ab-distance from the origin of the colour space to the (a,b) coordinates may be computed as the square root of the sum of a2and b2. A pixel may be considered colourless if:• the ab-distance is below a colour threshold value (e.g., 12.5),• or the luminance L is below a lower luminance threshold (e.g., 20% of 255),• or the luminance L is over an upper luminance threshold (e.g., 80% of 255).

[0068] Pixels considered colourless may be further classified into colour codes as follows:• black, if the luminance L is below a lower grey zone threshold (e.g., 160),• white, if the luminance L is equal to or greater than an upper grey zone threshold (e.g.,232),• otherwise grey.

[0069] The polar angle between the a and b components may be computed. Pixels not considered colourless may be classified into colour codes, where the list of colour codes is provided by a subject matter expert based on the relevance to the concentration of the marking. It will be appreciated that darker colours may already have been classified as colourless previously, as the objective is not to match human perception but the relevance to the concentration of the marking. Colour names in the following table are just to aid human understanding. In one embodiment:

[0070] The first concentration metric Pl may then be set to the highest value, (e.g., if there is enough yellow and the other colours do not matter, the Pl will be set to 5).

[0071] A second concentration metric P2 may be set based on a physical estimated size ES of the marking 202 and a predefined size S2 in square millimetres. In an example, S2=12903 square millimetres. It will be appreciated that the predefined size may be a comparison value representing a maximum ink surface that can be treated in a single session (to avoid pain, skin issues, and / or immune system overload).P2 = ceiling(ES / S2), (1) where ceiling rounds up to the closest integer, such that the lowest possible value of P2 is 1.

[0072] A third concentration metric P3 may be set as the surface characteristic of the target surface 204 (e.g., a value in the range 1 to 6). The value may be normalised further based on a risk ratio. In an embodiment, the risk ratio may reflect the impact of getting the type of target surface estimate wrong. In one example, a 5% safety margin may be added.

[0073] In another example, the normalization is non-linear, and the value of P3 may be adjusted as follows: P3 * 1.02AP3. In one embodiment, the normalization may be linear by adding a fixedamount H3 (e.g., 1.05), based on historical concentration statistics. Normalization may be done such that the normalized value is still within an expected range (e.g., minimum of 1 and maximum of 6).

[0074] In one embodiment, a concentration score P may be determined as follows:P = ((P3 * H3) * R3 + Pl * Hl) * P2 (2) where Hl and H3 are constants derived from historical concentration statistics, and R3 is a risk factor which can be set to optimistic (<1) or pessimistic (>1). In an embodiment, Hl=2, H3=1.05, R3=l.

[0075] In another embodiment, a concentration score P may be determined as follows:P = ((P3 + H3) * R3 + P1 * Hl) * P2 (3) where Hl and H3 are constants derived from historical concentration statistics, and R3 is a risk factor which can be set to optimistic (<1) or pessimistic (>1). In an embodiment, Hl=2, H3=1.05, R3=l.

[0076] A risk factor may reflect whether the estimate should be biased to underestimating or overestimating, and may be a commercially-driven decision. If volume (i.e. attracting many owners) is important, the risk factor may be set to optimistic. If a unit margin (that may not consider overheads) is important, the risk factor may be set to pessimistic.

[0077] In another embodiment, Hl may be obtained from a lookup table that depends on the value of Pl. In an example, Hl=l if Pl<=2 and Hl=1.5 if Pl>2.Analysis of Target Surface Characteristics

[0078] Once a plurality of pixels associated with the target surface 204 is determined, the pixels are analyzed to determine one or more target surface characteristics.

[0079] Analyzing one or more surface characteristics of the target surface 204 comprises determining a correspondence between the target surface pixels and a surface context reference model. In an example, the surface characteristic is the skin type, and the surface context reference model is the dermatological Fitzpatrick scale with 6 classes from light skin to dark skin.

[0080] In one embodiment, for each target surface pixel, an Individual Typology Angle (ITA) score may be computed. The pixels may be transformed from the original colour space, e.g. RGB (red, green, blue), to the CIELAB colour space (L*a*b*) which may have a closer resemblance to human colour perception than other colour spaces. The three components of Lab are L* representing luminance, a* ranging from red to green, and b* ranging from yellow to blue.ITA = arctan((L*-50) / b*) x 180 / K, (4)i.e. the ITA may be computed by dividing L* minus 50 by b* and then multiplying with 180 / K to normalise the angle. The ITAs for all relevant target surface pixels may form a distribution. In one embodiment, the ITA may be converted into an ITA band before further processing (e.g., band 1 for scores greater than 55, band 2 for scores greater than 41 and below band 1, band 3 for scores greater than 28 and below band 2, band 4 for scores greater than 10 and below band 3, band 5 for scores greater than -30 and below band 4 and band 6 for the remaining. The surface characteristic may then be computed from the ITA band distribution.

[0081] In one embodiment, the surface characteristic may be the mean of the ITA band distribution. In another embodiment, the surface characteristic may be the highest value of the ITA band distribution. In yet another embodiment, the surface characteristic may be the mean value of the highest value of the ITA band distribution (e.g., the mean of the upper quartile ITA bands).

[0082] The assessment of the concentration may be based on the resulting surface characteristic(s). In one embodiment, the band number minus 1 may be the number of additional sessions based on the skin type. For example, an estimation of band 3 skin type may indicate that two extra removal sessions are required compared to band 1 skin type.

[0083] In one or more embodiments, the steps of determining one or more pixels associated with the target surface 204 and analyzing one or more surface characteristics based on the determined target surface pixels may be performed before, after or in parallel with the steps of assessing concentration of the marking 202. In one embodiment, analysis of the surface characteristics may be done prior to pixel concentration score determination of the marking pixels such that the surface characteristic may then be used in the pixel concentration score determination step. This may help in further disambiguating marking pixels from target surface pixels, since they may often be intertwined. In one embodiment, the marking pixels with an ITA band similar to the surfacecharacteristic (e.g., + / - 1 band) may be excluded from further calculations in pixel concentration score determination step.

[0084] Fig. 5 is a flow diagram of a method 500 of assessing concentration of a marking on a target surface according to another embodiment of the present disclosure. The method 500 comprises acquiring at least one image of the marking 202 on the target surface 204 (step 502) and determining a plurality of pixels associated with the marking 202 (marking pixels) in the acquired image (step 504). The method 500 further comprises determining a plurality of pixels associated with the target surface (target surface pixels) in the acquired image (step 506). It will be appreciated that step 506 may be performed prior to, along with or after step 504 as described above. From step 506, the method 500 proceeds to analyse one or more surface characteristics of the plurality of pixels associated with the target surface in the acquired image (step 510) as described above. Segmenting the marking pixels from the surface pixels in the acquired at least one image may be further refined using the analyzed one or more surface characteristics (step 516). In one embodiment, the method 500 may proceed from step 502 to step 504 (first pass) without knowledge about the surface characteristic (e.g., skin type of the owner). Once the surface characteristic is estimated (step 510), the results may used to further refine (step 516) segmentation of the marking pixels, i.e. a second pass by using a more narrow definition of skin based on the skin type. In an example, since tattoo ink is not binary (i.e. it flows into just skin rather than there being a hard edge), such further refinement may help in improving segmentation of the pixels associated with the tattoo.

[0085] Upon segmenting the marking pixels at step 504, the method 500 proceeds to analyse one or more marking characteristics of the plurality of pixels associated with the marking in the acquired image (step 508) as described above. The method 500 then proceeds to determine a concentration score for one or more of the plurality of marking pixels based on the analysed one or more marking characteristics (step 512) as described above, and assesses an overall concentration score of the marking based on the determined concentration score for determining one or more actions to take for removing the marking from the target surface (step 514). The analysed one or more surface characteristics of the plurality of target surface pixels at step 510 may be used (as shown by 518) in the assessment of the overall concentration score at step 514.

[0086] The determined concentration score P may be aggregated with historical concentration values to generate an overall concentration score Q. The score determination calculations are applied to images in a historical database which includes estimates by subject matter experts or actual concentration metrics of the removal process. The calculations described above may be compared to the database values on an aggregate basis, e.g. the average. The average may then becompared and the difference may serve as a correction factor. In one example, +0.52 may be added to the determined concentration score. In another example, the determined concentration score may be multiplied by 1.18.

[0087] In one embodiment, the historical concentration values may be obtained from concentration assessment results derived by subject matter experts on the image or observations of the marking 202 and the target surface 204 made in-person by a human assessor. In a further embodiment, the historical concentration scores may be updated by comparing the outcome of the actual removal with the analyzed marking characteristics. In one or more embodiments, the assessment results derived by subject matter experts and / or observations made by human assessors may be stored in a remote storage location and accessed by the concentration determination module 304.

[0088] In various embodiments, one or both the determined concentration score P and the overall concentration score Q may be presented to the owner or other stakeholders (e.g., on a display screen of a user device. In some embodiments, the assessment of concentration may be based on additional practical and commercial considerations. In one embodiment, the overall concentration score Q is multiplied with a price to determine an impact score I.I = Q * price, (5) where the price may defined by an operator. In one example, price may be $200.

[0089] In another embodiment, higher weight may be given to the size of the marking 202I = Q * pricel + ceiling(P2 / S) * price2, (6) where S is a predefined size, say S=2580 square millimetres and pricel and price2 are defined by the operator. In an example, pricel=$150 and price2=$50.Size

[0090] The size of the marking 202 may affect the concentration of the marking due to limitations of human immune system. After determining the marking pixels, the marking size may be determined in terms of pixels. The pixel dimensions of the marking pixels may then be converted to physical dimensions of the marking. In one embodiment, a reference object with a known physical dimension may be detected in the acquired image as a reference object instance, and a conversion metric (e.g., millimetres per pixel) may be created by dividing the known physical dimension by the dimension of detected image pixels corresponding to the reference object instance. The marking pixel dimensions may then be multiplied with the conversion metric to get an estimate of the physical dimensions of the marking.

[0091] Fig. 6 shows an exemplary imaging guidance view for assisting a user to acquire an image of the marking 202 on the target surface 204. In an embodiment, a preview and dashed box 602 may be presented on a display screen of the imaging device or a suitable user device such as a mobile phone 604 to guide the user to frame the marking 202 (e.g., to guide the user to leave some margin around the marking).

[0092] Fig. 8 shows an exemplary imaging guidance view for assisting a user to acquire an image of the marking 202 on the target surface 204 wherein the image includes a reference object 802. In one embodiment, the reference object 802 is a rounded rectangle such as a credit card or loyalty card or any card of a standardized size (e.g., business card, driver's license etc.). In an embodiment, standardized size of a reference object 802 may have dimensions of 85.6mm by 53.98mm and a diagonal of 101.20mm. In an embodiment, a preview and dashed box 804 may be presented on a display screen of the imaging device or a suitable user device such as a mobile phone 806 to guide the user to frame the marking 202 (e.g., to guide the user to leave some margin around the marking).

[0093] In an embodiment, the user may be guided to take an overview image including a reference object 802 and a zoomed-in image without the reference object 802. The dimensions of the marking detected in the overview image may then be applied to the zoomed-in image.

[0094] In one embodiment, the foreign reference object 802 (e.g., a coin, credit card, ruler, match box etc.) may be directed to be part of the acquired image such that the reference object 802 is in a similar plane (relative to the imaging device 302 as the marking 202. Pixels corresponding to the reference object 802 may then be detected through an ANN configured with model parameters and weights pre-trained for reference object detection. At inference time, the ANN may receive the acquired image as input and output coordinates of areas in the imager containing instances of the reference object 802. If there are multiple instances, the instance with the highest confidence score may be selected. In an embodiment, if there is a detection with a confidence of 0.8 and there is a detection with a confidence of 0.6, then the detection with a confidence of 0.8 may be selected. The pixel dimensions of the detected reference object instance may then be measured. In an embodiment, the pixel dimensions may be measured by calculating the diameter of the detected circle if the reference object 802 is a coin. In another embodiment, the pixel dimensions may be measured by calculating the diagonal of a detected rectangle if the reference object 802 is a credit card. The diagonal of a standard size credit card or business card is approximately 101 mm. The known reference object physical dimensions may then be divided by the pixel dimensions to get the conversion metric. In an example, if the detected diagonal is 420 pixels, the conversion metric maybe 101 / 420 = 0.24 mm per pixel. For a detected marking measuring 500 by 300 pixels, the estimated area = (500 * 0.24) * (300 * 0.24) = 8640 mm2.

[0095] Fig. 7 is a flow diagram of a method 700 of assessing concentration of a marking 202 on a target surface 204 according to yet another embodiment of the present disclosure. The method 700 comprises acquiring at least one image of the marking 202 on the target surface 204 (step 702), wherein the image includes a reference object 802, and determining a plurality of pixels associated with the marking 202 (marking pixels) in the acquired image (step 704). The method 700 further comprises detecting pixel dimensions and actual dimensions of the reference object 802 in the acquired image (step 706). The dimensions of the marking pixels determined at step 704 may then be converted into actual dimensions at step 708 using (shown as step 716) the detected pixel dimensions and actual dimensions of the reference object. Once the actual dimensions of the marking 202 are determined, the method 700 proceeds to analyse one or more marking characteristics of the plurality of pixels associated with the marking in the acquired image (step 710) as described above. The method 700 then proceeds to determine a concentration score for one or more of the plurality of marking pixels based on the analysed one or more marking characteristics (step 712) as described above, and assesses an overall concentration score of the marking based on the determined concentration score for determining one or more actions to take for removing the marking from the target surface (step 714).

[0096] In another embodiment, the owner or the marking 202 may indicate the body part to which the marking relates. In an example where the indicated body part is a hand, a related body part (e.g., underarm) or a component of the body part (e.g., knuckles) may then be used as the reference object, where the physical dimensions of the reference object are averages of population statistics, that may be further refined (e.g., by using the gender, age, weight or height of the owner). In one embodiment, person detection (described above) may be employed to get the boundaries of an underarm. Two long and approximately parallel boundaries may be considered to be the length of the underarm, which may then be used to compute a conversion metric in a similar manner to the approach described above for the foreign reference object. In a further embodiment, further analysis may be performed to determine the smallest distance between the two lengths of the underarm, which may be assumed to be the location of the wrist. Wrist width may then be used as the reference object.

[0097] In another embodiment, the body part to which the marking 202 relates is labeled by the user or a body part labelling image processing module, or a combination thereof. Examples of user labelling are: the user types in the name of the body part; the user selects the body part from a listof labels; the user selects the body part by clicking on a 2D or 3D diagram of the human body. In an example of a body part labelling image processing module, semantic segmentation using an artificial neural network (ANN) configured with model parameters and weights pre-trained for body part segmentation may be deployed, where a labeled training dataset includes labels such as, but not limited to, “foot”, “ankle”, “lower leg”, “knee”, etc.. At inference time, upon receiving the acquired at least one image as an input, the ANN may output coordinates of areas in the acquired image containing the body parts, at least in part or as a whole and assign the one or more pixels associated with the body part to a class of body part labels. If multiple body part labels are segmented in the input image, one body part may be selected, e.g.: the biggest body part segment in terms of pixel count; the body part segment of which the centre pixel is the closest to the centre of the image; the body part segment which overlaps the area of the marking most. In an example of combination with user labelling, the user may confirm or correct the outcome of the body part labelling image processing module into a selected body part label. This embodiment may require less effort from user.

[0098] The selected body part label may be looked up in a body part conversion table which records, for each body part label, the expected pixels per millimetre resolution in the image. While the resolution may not be directly dependent on the body part, in practice, users may frame the image differently depending on the body part. For example, when photographing an ankle, a user may hold the camera approximately 5 times as close to the body part compared to when photographing the torso. The expected pixels per millimetre resolution may be experimentally determined from a population of sample images, where test subjects emulate the photo-taking of various body parts. In one embodiment, the expected pixels per millimetre resolution is the average of the pixels per millimetre resolutions found in the population for the body part label. In another embodiment, the expected pixels per millimetre resolution is the median of the resolutions found in the population for the body part label.

[0099] The looked-up pixels per millimetre resolution (LUPPMR) may then be applied to the area of the detected marking and converted to millimetres: area_m_mm = area_in_pixels / LUPPMR. For example, for a detected marking measuring 500 by 300 pixels, and the body part label “ankle”, the LUPPMR may be between 22 and 28 pixels per millimetre. The estimated area at the maximum may be = (500 * 300) / 22 = 6,818.18mm2and at the minimum may be = (500 * 300) / 28 = 5,357.14mm2.

[0100] In one embodiment, measurements may be corrected for image resolutions. The look up table values relate to 4000x3000 pixels image resolutions, taken under controlled circumstances.For an input image with a different image resolution, an adjusted pixels per millimetre resolution (APPMR) may be computed as follows: APPMR = ((input_image_width * input_image_height) / (4000 * 3000)) * LUPPMR. The conversion is then: area_m_mm = area_in_pixels / APPMR. For example, for a detected marking measuring 500 by 300 pixels, and the body part label “ankle”, the LUPPMR may be at a minimum 22 pixels per millimetre and at a maximum 28 pixels per millimetre. In one embodiment, the example input image is taken on a tablet and has resolution 2592x1944. APPMR (when LUPPMR is at the minimum) = ((2592* 1944) / (4000*3000)) * 22 = 9.24 pixels per millimetre. The estimated area at the maximum may be = (500 * 300) / 9.24 = 16,233.77 mm2. In another embodiment, the example input image is taken on a tablet and has resolution 2592x1944. APPMR (when LUPPMR is at the maximum) = ((2592*1944) / (4000*3000)) * 28 = 11.76 pixels per millimetre. The estimated area at the minimum may be = (500 * 300) / 11.76 = 12,755.1 mm2. Colour Correction

[0101] In one or more embodiments, colour correction may be applied to the acquired image prior to further processing by the concentration determination module 304.

[0102] In an embodiment, a colour reference marker may be included in the image (e.g., a white sticker). A transformation from the pixel values of the colour reference marker to the known colours of the colour reference marker may then be determined, and the transformation may be applied to the image.

[0103] In another embodiment, scene illumination may be estimated as input to a chromatic adoption transform (CAT). Such an approach may be advantageous when it is not practical to have a colour reference marker in the image.Downscaling

[0104] In some embodiments, the acquired image may be downscaled prior to processing by the concentration determination module 304 and the downscaled version may be used for the various steps / processes performed by the concentration determination module 304. Doing so may reduce computational load.

[0105] In other embodiments, only some steps or processes may use a downscaled version of the acquired image while other steps or processes may use the original version. In an example, person detection process may use a downscaled version of the acquired image, while marking detection process may use the original version.

[0106] In one embodiment, downscaling may involve averaging the pixel values in a window (e.g., 2x2 pixels) to a new pixel value.

[0107] In another embodiment, downscaling may involve maxpooling the pixel values in a window (e.g., 2x2 pixels) to a new pixel value. In other words, the highest value of the pixels in a window may be taken as a representative for the downscaling.

[0108] It will be appreciated that a plurality of artificial neural networks (ANN) may be employed in enabling the disclosure. In one embodiment, each detection (e.g., marking detection, target surface detection, reference object detection) may be performed done by a separate ANN. In another embodiment, a single ANN may perform the different detections within one inference, using the same pre-trained weights and parameters.Actions to take for removal of marking from target surface

[0109] Following the assessment of concentration of the marking 202, one or more actions to be taken for removing the marking 202 from the target surface 204 may be determined. One such action may be to predict an estimation of a number of sessions required to remove the marking 202 from the target surface 204. Another action may be to generate an estimation of cost i.e., quote for removing the marking 202 from the target surface 204.

[0110] In an embodiment, removing a tattoo (marking) from a skin (target surface) may involve applying laser light beams to the skin. Ink particles that make up the tattoo in the skin may absorb the laser and disintegrate. The disintegrated ink particles would then be small enough for the body ’ s immune system to encapsulate and dispose. The removal of the tattoo may be spread out over several sessions to minimize short-term pain and / or maximise long-term skin health and / or to ensure the immune system of the body is under acceptable levels and not overloaded. In one example, a tattoo with pastel colours may require more number of sessions for removal than a tattoo with black and / or dark green colours (for removing pastel-coloured tattoo than for black and / or green coloured tattoo). This may be due to the different molecular structures of the inks in the tattoo (e.g., molecular structure of pastel colours may be harder to break down compared to the molecular structure of the black and / or green colours and therefore 2 additional sessions may be required for removing pastelcoloured tattoo than for a tattoo having black and / or green colours). Similarly, as red ink typically uses a different material, a further 2 sessions may be required to break down the red ink molecules / particles without overloading immune response of the person. The size of the tattoo (in terms of ink) may also affect the total number of sessions required to remove the tattoo, since the tattoo may have to be removed in parts to avoid pain and / or immune system overloading. For example, if a tattoo needs to be removed in two parts, it may take twice the number of sessions. However, the time between sessions may be shorter, since the skin for one part may heal while the skin for the other part is treated. The skin type of a person may determine the intensity of the laserthat can be used without affecting skin health. For example, people with a very white skin tone (typically having red hair and sensitive to sun burns) may be able to handle a strong laser and require fewer sessions than people with a very dark skin (typically having black hair and brown eyes and not very sensitive to sun bums). In one example, people with a very white skin tone may require four sessions less than people with a very dark skin for removing the same tattoo (same size and colour palette) on the same location on their body.

[0111] Fig. 4 is a view for explaining assessment of concentration according to an embodiment of the present disclosure. An owner 402 may have a marking 404 on a target surface 406 at some part of the body. An imaging device 408 such as a mobile phone camera may be used to acquire at least one image of the marking 404 and the target surface 406. The acquired image may then be transmitted over a communication network 410 to a remote system 412 including a concentration determination module (as described above).

[0112] In one embodiment, the acquired image may be uploaded to the remote system 412 via the communication network 410, wherein the concentration determination module 304 performs detection of marking areas such as the marking 404, target surface 406 and any other relevant areas for subsequent classification and processing of all the detected target areas as described above.

[0113] In another embodiment, the acquired image may be uploaded from the imaging device 408 to the remote system 412, wherein detection of the marking areas takes place. The detected areas may then be transmitted back to the imaging device 408 such as a mobile phone, where the owner or a relevant operator such as a practitioner can select, remove, or adjust the detected areas. In an embodiment where the imaging device 408 may not have a display screen, the detected areas may be transmitted from the remote system 412 to a user device comprising a display screen (not shown), where the owner or a relevant operator such as a practitioner can select, remove, or adjust the detected areas. The new set of detected areas may then be uploaded to the remote system 412 via the communication network 410 and analysis may proceed on the new set of detected areas only. In one embodiment, analysis components that do not depend on the definition of the marking areas (e.g., analysis of surface characteristic(s), determination of dimensions of reference object) may be executed in parallel while user interaction takes place. In another embodiment, execution is paused until user interaction is complete.

[0114] In yet another embodiment, detection of target areas may be performed locally on the imaging device 408 such as a mobile phone or a user device to which the imaging device 408 is connected if the imaging device 408 does not have a display screen. The detected areas may then be displayed on the display screen of the imaging device 408 or the user device, where the owneror relevant operator can select, remove, or adjust areas. The new set of areas may then be uploaded to the remote system 412 via the communication network 410 and analysis may proceed on the new set of areas only.

[0115] In yet another embodiment, the acquired image may be presented on the display screen of imaging device 408, where the user may create areas (e.g., the user may define the areas of a marking from scratch, such as by clicking on the image to define points to form a closed polygon). The new set of areas may then be uploaded to the remote system 412 via the communication network 410 and analysis may proceed on the new set of areas only.

[0116] Fig. 9A shows an acquired image of a marking 202 on a target surface 204 according to an embodiment of the present disclosure. Fig. 9B shows detection of an outline 902 of the marking 202 according to an embodiment of the present disclosure. The detection of the outline 902 may be done by semantic segmentation of the marking pixels and target surface pixels and analysis of the marking characteristics as described above. In some embodiments, such detection of the outline 902 of the marking 202 may be done on a mobile phone or any suitable user device such as but not limited to a computer, laptop, etc. The outline 902 may be in an edit mode and a user may move, delete or add edit points to change the outline in order to select relevant part(s) of the marking 202 to be removed. Fig. 9C shows an outcome of user interaction according to an embodiment of the present disclosure. The user may only want the stem and leaves removed while wanting to retain the remaining part of the marking 202. It will be appreciated that the acquired image may contain multiple markings or the owner may want to remove part of the marking only. Once the user has selected a part of the single marking or multiple markings, the selected part may be further processed and analysed in a similar fashion to the whole marking as described above.

[0117] If the user has selected part of a marking 404, the selected part may be further processed and analyzed in a similar manner to the whole marking as described above. Similarly, if the acquired image contains multiple markings, the user can select a plurality of markings or desired sections of the multiple markings to then be processed and analyzed in a similar manner as a single whole marking as described above.

[0118] In various embodiments, the determined concentration scores P and / or the assessed overall concentration score Q and / or the impact score I (e.g., the price per removal or reduction iteration and / or total price for reducing the appearance of the marking to a desired level and / or) and / or a number of reduction iterations or removal iterations that may be required to reduce the appearance of the marking to a desired level may be (a) displayed on a display screen of the imaging device 408 or a display screen of a user device such as a mobile phone or a computer, tablet, laptopetc. (e.g., if different to the imaging device 408), (b) polled for by a software application running on the imaging device 408 or a user device to the remote system 412, and upon receiving the determined concentration scores and / or the assessed overall concentration score from the remote system 412, displaying the determined concentration scores and / or the assessed overall concentration score on a display screen of the imaging device 408 or a display screen of the user device such as a mobile phone or a computer, tablet, laptop etc. (e.g., if different to the imaging device 408), (c) presented in an email or as an audio text to the owner on the display screen of the imaging device 408 or a display screen of a user device such as a mobile phone or a computer, tablet, laptop etc. (e.g., if different to the imaging device 408). In one embodiment, the one or more actions to be taken in for removing the marking 404 based on the assessed concentration may be displayed on the display screen, e.g., 8 sessions over a period of 6 months. In another embodiment, $30 per week for a period of 6 months. The one or more actions may include predicting a number of sessions required to remove the marking 404, predicting a quote of the cost for removing the marking, making an appointment with a marking removal operator. In some embodiments, one or more of a session time and location of centres where the marking can be removed may be displayed or presented to the user on a display screen or user device to select from and make an appointment.

[0119] Upon completion of assessment of concentration of the marking 404, one or more settings of a marking removal device 414 may be adjusted for removing the marking 404 from the target surface 406 of the owner 402. In one example, the marking removal device 414 is a laser tattoo removal device and the frequency, intensity of the laser can be adjusted based on the assessed overall concentration score of the marking 404. In one embodiment, prevention or reduction of side effects may be considered in adjusting the settings of the marking removal device 414. In one example, for tattoo removal, the laser intensity may be adjusted to the skin type of the target surface to prevent or reduce hypopigmentation and hyperpigmentation.

[0120] Many modifications will be apparent to those skilled in the art without departing from the scope of the present invention.

[0121] The present disclosure may provide an automated assessment of concentration of a marking 202 on a target surface 204 based on image analysis as described above. In contrast to concentration assessment of markings 202 and target surfaces 204 through direct observation by a human practitioner, the present disclosure may provide improved scalability, may be more interactive, provide a way for assessment to be done at any time, in a more secure or private environment (such as the home of the owner of the marking 202) and may provide feedback capability.

[0122] One or more embodiments of the present disclosure may provide a marking concentration assessment method and system that is more objective, private, scalable and consistent. The assessment may also be performed at a location remote from the owner of the marking and / or the marking removal device in some embodiments.

[0123] Further, selecting parameters of the device (e.g., laser tattoo removal device) used to remove the marking 202 based on the assessment of concentration of the marking as described above may assist with providing one or more iterations of removal or reduction sessions with low thermal impact, and may be suitable for a wide range of types of target surface 204.

[0124] The present disclosure may assist an owner of the marking in performing an initial assessment of the marking and getting an estimation of the cost, time, number of sessions etc. that may be required to remove the marking without needing another human practitioner or assessor to perform the initial assessment and estimation. This may be advantageous since marking removal may become more accessible to owners since they can avoid social anxiety and / or embarrassment that may be involved in getting a human assessor to perform the initial assessment and estimation. The owner may then, based on the assessment of concentration of the marking, choose to make an appointment with a professional marking removalist (human operator) at a convenient time and / or location. The professional marking removalist may or may not use the assessment of the marking concentration in determining one or more parameters of the removal device. Using the abovedescribed concentration assessment may reduce or eliminate adverse events due to wrong or improper selection of the one or more parameters of the removal device or choice of removal device by the human operator. Marking removal options may be more objective and auditable due to the present disclosure (e.g., when the human operator's view differs from the assessed overall concentration score, determination of removal device and / or parameters of the removal device may be determined in consultation with a dermatologist, the assessment may assist in training operators who may otherwise go ahead with non-ideal selection of removal device and / or parameters of the removal device such as selecting parameters that may either damage the skin of the owner, or not be sufficient to obtain a desired level of removal or fading as discussed above).

[0125] An exemplary target surface 1304 on a target owner 1302 is shown in Fig. 13. In various embodiments, the target owner 1302 may be a human being, an animal etc. In various embodiments, the target surface 1304 may be skin of the target owner at any part of the body of the target owner 1302.

[0126] One or more embodiments of the present disclosure is directed towards estimating a type of target surface 1304. The type of target surface 1304 may impact one or more parametersassociated with laser-based applications on the target surface. In one example, one or more parameters associated with laser-based removal of hair or a marking on the target surface such as a tattoo, birthmarks, wrinkles, keloids, scars, vascular lesions, pigmented lesions, sun-damaged skin (e.g., number of sessions, cost, time per session, total time over the total number of sessions etc.) i.e., process required to remove or reduce the number of hairs on the target surface (e.g., skin) or to remove or diminish the visibility of the marking without causing or reducing detrimental side effects (e.g., pain, change in sensitivity of the target surface 1304, health of the target surface 1304, impact on immune response of the target owner 1302 and / or general health of the target owner 1302).

[0127] It will be appreciated that the terms "removal", "removing" or variations thereof, of the marking may be defined as removing the marking partially or completely or diminishing the visibility of the marking to a naked eye or diminishing the visibility of the marking such that the marking can then be covered with a newer marking. It will further be appreciated that "removal", "removing" or variations thereof, of the marking may refer to the whole marking or portions of the marking. In another example, the marking (e.g., tattoo) may be removed using laser treatment, where one or more parameters of the laser (e.g., wavelength, beam spot diameter, fluence) is set such that the ink molecules may be broken down to be small enough to be disposed of by the human immune system. In the case of hair removal, the one or more parameters of the laser may be set to achieve the desired hair removal. The required wavelength for the laser may depend on one or more factors such as, but not limited to, a type of ink used, an age of the hair / marking (e.g., how much time has passed since the application of the marking on the target surface 1304). The laser fluence may depend on one or more factors such as, but not limited to, a type of target surface 1304.

[0128] It will be appreciated that the wavelengths of laser to be used for removing the hair or marking from the target surface 1304 of the target owner 1302 may be subject to compliance guidelines determined by the health departments of respective jurisdictions (e.g., according to USA Food & Drug Administration (FDA) guidelines as well as Conformite Europeenne (CE) compliance guidelines). Laser pulses with ultra-short picosecond duration may create a rapid photoacoustic effect within the target surface 1304 while gently impacting outer layers of the target surface 1304. Certain types of laser applicators may target particles of the marking while leaving the outermost layer of the target surface 1304 intact or hardly damaged.

[0129] In one example, a 1064 nanometre (nm) laser wavelength may be safely used on any skin type. This wavelength may be sufficient for removal of darker inks (e.g., black, dark blue), and may diminishes lighter-coloured inks as well but to a lesser degree. A 532 nm laser wavelength may be applied to remove red or orange colours (particularly on lighter skin tones). A 730 nm laserwavelength may be applied to remove blue and / or green colours (particularly for medium skin tones). It will be appreciated that suitable adjustments to the laser wavelength may be made based on the type of the target surface 1304 to minimize adverse effects. It will be appreciated that the wavelengths of laser to be used for removing the marking may be subject to compliance guidelines determined by the health departments of respective jurisdictions (e.g., according to USA Food & Drug Administration (FDA) guidelines as well as Conformite Europeenne (CE) compliance guidelines). Different skin types, marking embedding techniques and / or ink pigmentation / colours may result in different absorption coefficient (cm1) of the ink of the marking on the target surface. Laser pulses with ultra-short picosecond duration may create a rapid photoacoustic effect within the target surface 1304 while gently impacting outer layers of the target surface 1304. Certain types of laser applicators may target ink particles of the marking while leaving the outermost layer of the target surface 1304 intact or hardly damaged.

[0130] The marking removal or reduction operator may consider the type / tone of the target surface 1304 of the owner as guidance to set a laser beam spot diameter and select a treatment fluence within a manufacturer prescribed margin for the diameter and the type / tone of the target surface 1304 and other indicators such as the heritage of the person. In an example, the operator may select a spot diameter of 6mm for an owner with a skin type in band 2 of a standardized skin type schema. The operator may then pick a treatment fluence of 1.4 Joules per square centimetre (J / cm2) and apply the setting for treatment of a small patch of skin. Depending on the response of the skin, the operator may then increase or decrease the diameter, and consequently decrease or increase the treatment fluence, for another patch of skin. Once a good response is found, the operator may use the setting for the specific reduction iteration or treatment. The spectrum of skin tones may be typically indicated by the dermatological Fitzpatrick scale with six classes from light skin (I) to dark skin (VI). Lighter skin tones may be associated with people from Nordic heritage (often correlating with red hair and freckles), while darker skin tones may be associated with people having heritage from regions near the equator.

[0131] In a similar manner, other skin transformations may be achieved using laser treatment. For example, for one laser applicator several skin conditions such as melasma, benign pigmented lesions, and wrinkles are FDA approved for skin types LIV only on the dermatological Fitzpatrick scale with 6 classes from light skin (I) to dark skin (VI). For the same applicator, acne scars are FDA approved for skin types ILV only, and hyperpigmentation Nevus of Ota for skin types III-IV only. In another example, for hair removal, a skin tone in a lower class may allow for a higher fluence setting. However, for skin type I on the Fitzpatrick scale (which is correlated with the hairtype) the likelihood of successful hair removal may be low as there may not be sufficient pigment in the hair to attract the laser.

[0132] Fig. 12 is a flow diagram of a method 1200 of estimating a type of target surface 1304 according to an embodiment of the present disclosure. The method 1200 comprises acquiring at least one image of the target surface 1304 (step 1202), determining a plurality of target surface pixels associated with the target surface 1304 in the acquired image (step 1204), determining a surface characteristic for the target surface 1304 based on the determined plurality of target surface pixels (step 1206), assigning a subgroup-based score to each of the plurality of target surface pixels based on the determined surface characteristic (step 1208) and estimating a type of the target surface 1304 based on the assigned subgroup-based score and a surface context reference model (step 1210). The steps will be described in more detail below.

[0133] Fig. 14 is a system 1400 for performing the method of Fig. 12 according to an embodiment of the present disclosure. The system 1400 comprises an imaging device 1402 and a type determination module 1404 as will be described in more detail below.Image Acquisition

[0134] At step 1202 of the method 1200, at least one image of the target surface 1304 may be acquired using an imaging device 1402. In one embodiment, the imaging device 1402 may be a mobile phone. In another embodiment, the imaging device 1402 may be a camera. In a further embodiment, the imaging device 1402 may be a scanner and the at least on image may be acquired after a body part pertaining to the target surface 1304 whose type is to be estimated is scanned by the scanner.

[0135] The at least one image may be in JPEG (Joint Photographic Experts Group) format, HEIC (High Efficiency Image Container) format or any other suitable format.

[0136] In some embodiments, the step 1202 may comprise acquiring one or more image sequences, such as videos (e.g., in H264, MPEG-4, M-JPEG or any other suitable formats). In further embodiments, the step 1202 may comprise acquiring a plurality of images (i.e., burst images) in JPEG, HEIC or any other suitable formats. The acquired plurality of images may depict the target surface 1304 in different angles in which case, the plurality of images may be stitched into a single composite image for subsequent type estimation (described below). In an embodiment, the plurality of images may be stitched using scale-invariant feature transform which may characterize local features of an image. The area of the target surface 1304 may first be detected and segmented, after which the SIFT key points for each segment may be computed. The SIFT key points may then becompared and clustered with a Hough Transform. The transformation matrix between the two image segments may then be computed using linear least squares. With the transformation matrix, the image segments can be represented in the same image space for further processing. In another embodiment, the plurality of images may be stitched into a single composite image using Hough Transform.

[0137] In another embodiment, each of the plurality of acquired images depicting different parts of the desired target surface 1304 may be analysed individually, and an overlap between the images may be aggregated (e.g. by taking an average, maximum, minimum, or weighted averaging using suitable weights (e.g., 0.5)) for type estimation. It will be appreciated that any amount of overlap that can have a noticeable impact on the end result would be sufficient. In another embodiment, one of the plurality of images may be selected for type estimation (e.g., by measuring blurriness of the plurality of images and selecting the least blurry image). Blurriness may be measured by convolving the image convolved with a Laplacian kernel ([[0,-l,0], [-l,4,-l],[0,-l,0]]), to measure a second derivative of the image. If the variance of the response is below a threshold (e.g., 100), the image may be considered to be blurred due to an absence of clear edges.

[0138] In one or more embodiments, the plurality of images may be captured at the same angle from the target surface 1304. In some embodiments, the plurality of images may be captured at different angles from the target surface 1304. In some embodiments, the one or more images may be captured at an angle between 0 degrees and 30 degrees from the target surface 1304, where 0 degrees is when the imaging device 1402 is positioned top-down onto the target surface 1304. In one embodiment, the overview image may be used to determine the part of the body pertaining to the desired target surface 1304 whose type is to be estimated. The body part may impact the number of sessions or iterations required for the laser-based application, since the farther away the target surface 1304 is from the body’s lymph nodes (i.e., key part of the immune system), the more number of sessions that may be required. In an example, a marking on the skin of a foot may require 4 laserbased removal sessions more to remove than a marking on the skin of the neck.

[0139] In an embodiment, prior to acquiring the at least one image (step 1202), a person requiring the laser-based application (e.g., the person or target owner 1302 on whose body the marking is present, the person in charge of cattle from whose hide the marking is to be removed) may be presented with a list, images, 3D model or the like to select the relevant body part from, on a display screen of a user device. In one example, the display screen may be a display screen of a mobile phone, a computing device or the like. In one or more embodiments, the user device may be the same as the imaging device 1402. In other embodiments, the user device may be communicativelyconnected to the imaging device 1402. A shape or outline of the body part selected by the person on the user device may then be displayed on the display screen which can provide guidance when acquiring the at least one image using the imaging device 1402 (e.g., to check if the relevant body part is centered in the image, sufficient context (e.g., a margin of at least 15% of the image at each of the 4 sides surrounding the marking) is captured, and / or that a distance between imaging device and the person or owner 1302 is acceptable (e.g., for a marking having dimensions of 10cm by 10cm, a distance of 15cm - 30cm may be acceptable)).Estimation of target surface types

[0140] The steps of determining a plurality of target surface pixels associated with the target surface 1304 in the acquired image (step 1204), determining a surface characteristic for the target surface 1304 based on the determined plurality of target surface pixels (step 1206), assigning a subgroup-based score to each of the plurality of target surface pixels based on the determined surface characteristic (step 1208) and estimating a type of the target surface 1304 based on the assigned subgroup-based score and a surface context reference model (step 1210) may be performed by a type determination module 1404 of the system 1400 as will be described below.

[0141] In an embodiment, the system 1400 may be a mobile phone comprising the imaging device 1402 (e.g., a camera of the mobile phone) and the type determination module 1404. In another embodiment, the imaging device 1402 may be a camera or a scanner communicatively connected to a computing device such as a computer, laptop etc. and having a type determination module 1404 therein. In an embodiment, the type determination module 1404 may be hosted in a remote location (e.g., server on a cloud) while the imaging device 1402 is at the same or nearby location of the owner 1302 of the target surface 1304, and the imaging device 1402 and the type determination module 1404 are configured to communicate with each other using suitable communication protocols.Pixel classification

[0142] The at least one acquired image is sent from the imaging device 1402 to the type determination module 1404. It will be appreciated that in the scenario where the imaging device 1402 and the type determination module 1404 are in a single device such as a mobile phone, the acquired image may be stored in the same single device. Alternatively, the acquired image may be stored in a remote location (e.g., a persistent storage of a remote system) to which the imaging device 1402 and the type determination module 1404 are communicatively connected e.g., via the Internet.

[0143] The acquired image is made up of a plurality of pixels. One or more of the plurality of pixels of the acquired image may be associated with the target surface 1304 ("target surface pixels") while the remaining one or more of the plurality of pixels of the acquired image may be associated with other objects or the background. In other words, a target surface 1304 may be represented by a set of pixels ("target surface pixels"). In some embodiments, one or more pixels of the acquired image that may not be associated with the target surface 1304 may be classified as background or belonging to other objects ("background pixels"). In one embodiment, any pixels detected to be in the target surface area but having colour(s) that do not match a desired colour of target surface area may be classified as background pixels.

[0144] The type determination module 1404 may perform semantic segmentation using an artificial neural network (ANN) configured with model parameters and weights pre-trained for target surface detection, i.e., detection of the target surface 1304. At inference time, upon receiving the acquired at least one image as an input, the ANN may output coordinates of areas in the acquired image containing the target surface 1304, at least in part or as a whole and assign the one or more pixels associated with the target surface 1304 to a class of target surface pixels.

[0145] In an embodiment, an object detection to detect the target owner 1302 may be performed and any pixels in the area associated with the target surface 1304 that do not match a target surface (e.g., skin) may be classified as background pixels. The pixels classified as target surface pixels together form the target surface 1304.

[0146] In an embodiment, all pixels in the acquired image are considered for classification. In another embodiment, only those pixels near the hair and / or a marking are considered for classification as target surface pixels, e.g., pixels with a distance less than 100 pixels to the closest boundary pixel of the hair and / or marking. This approach may mitigate the risk of including in the classification process other target surfaces in the image that may not belong to the owner 1302 of the target surface 1304, but to other people who are irrelevant to the assessment.

[0147] In yet another embodiment, object detection (e.g., detecting the target owner 1302 of the target surface 1304) may be performed prior to detection and classification of pixels associated with the target surface 1304. An ANN may be configured with model parameters and weights pre-trained for the target owner detection. As owner detection is more generic than detection of target surface 1304, a larger training data set may be used. At inference time, upon receiving the acquired at least one image as an input, the ANN may output coordinates of areas in the acquired image containing the owner 1302 of the target surface 1304. The plurality of pixels associated with the detected target owner may be referred to as owner pixels. In some further embodiments, the ANN may output amore granular classification (such as man, woman, child etc.). This approach may mitigate the risk of including in the classification process elements in the image having similar colour properties to the target surface 1304 (e.g., birch coloured furniture having similar colour as skin). Further detection and classification of pixels associated with the target surface 1304 may then be limited to the areas identified as being associated with the owner 1302.

[0148] The method 1200 may then proceed to determine a surface characteristic of the target surface 1304 based on the determined plurality of target surface pixels. In an embodiment, the surface characteristic is a colour distribution of the target surface pixels. In an embodiment, the one or more pixels associated with the target surface 1304 may be transformed from an original colour space (e.g., RGB (red, green, blue)) to Hue-Saturation-Value (HSV) colour space. It will be appreciated that grouping of similar colours may be easier in HSV colour space than in original colour space. A 3D sub-space in the HSV colour space may then be determined. In an example, 3D sub-space having a hue between 5 and 15, a saturation between 32 and 192, and value (similar to intensity) between 48 and 250 may be determined. Pixels with a HSV value in the sub-space may then be assigned to the target surface pixel class.

[0149] In another embodiment, the one or more pixels associated with the target surface 1304 may be transformed from an original colour space (e.g., RGB (red, green, blue)) to Luminance / Chrominance-blue / Chrominance-red (YCbCr) colour space. It will be appreciated that considering brightness information in a different way than colour information may be easier in YCbCr colour space than in the original colour space. A 3D sub-space in the YCbCr colour space may then be determined. In an example, 3D sub-space having a Y between 0 and 255, a Cb between 133 and 173, and Cr between 77 and 127 may be determined. Pixels with a YCbCr value in the subspace may then be assigned to the target surface pixel class.

[0150] In an embodiment, an initial surface score for each of the plurality of pixels in the acquired image may be determined. In an embodiment, an initial surface score may be determined for each of the plurality of owner pixels associated with the detected owner in the acquired image. In an embodiment, an initial surface score may be determined for each of the plurality of target surface pixels associated with the target owner 1302.

[0151] In one embodiment, the initial surface score for a pixel is set to 1 if the pixel’s colour value is in the 3D sub-space in the YCbCr colour space, and set to 0 otherwise. In another embodiment, the initial surface score for a pixel is the Euclidean distance between the colour coordinates of the pixel and the centre of the 3D sub-space. In another embodiment, the initial surface score for a pixel is the Euclidean distance between the colour coordinates of the pixel andthe centre of the 3D sub-space if the colour coordinates of the pixel are within the 3D sub-space, and 0 otherwise.

[0152] In one embodiment, the initial surface score for a pixel is the result of the target surface detection ANN. In one embodiment, the initial surface score is 1 if the pixel is in the segmentation result. In another embodiment, the initial surface score is the probability provided by a SoftMax activation function in the final layer of the target surface detection ANN.

[0153] In one embodiment, pixels are categorized as target surface pixels if their initial surface score exceeds a predetermined threshold, e.g. greater than 0. In an embodiment, each of the plurality of owner pixels having the initial surface score greater than a predetermined threshold are categorized as target surface pixels. The predetermined threshold may be experimentally determined by finding a trade-off between false positives and false negatives. In one example, false negatives may be minimised, and consequently the number of false positives may go up. In another example, an equal error rate may be used, determining that the false positive and false negative rates are the same on a dataset.Determination of Surface Characteristic, Surface Characterization Score

[0154] Once a plurality of pixels associated with the target surface 1304 ('target surface pixels") is determined and their initial surface scores are determined, the pixels are analyzed to determine a surface characteristic (step 1206).

[0155] For each target surface pixel, a surface characteristic may be determined. In an embodiment, the surface characteristic may be an Individual Typology Angle (IT A) score. The target surface pixels may be transformed from the original colour space, e.g. RGB (red, green, blue), to the CIELAB colour space (L*a*b*) which may have a closer resemblance to human colour perception than other colour spaces. The three components of Lab are L* representing luminance, a* ranging from red to green, and b* ranging from yellow to blue.ITA = arctan((L*-50) / b*) x 180 / K, (1) i.e. the ITA may be computed by dividing L* minus 50 by b* and then multiplying with 180 / K to normalise the angle. The ITAs for all relevant target surface pixels may form a distribution. In one embodiment, the ITA may be converted into an ITA band before further processing using a surface context reference model (e.g., band 1 for scores greater than 55, band 2 for scores greater than 41 and below band 1, band 3 for scores greater than 28 and below band 2, band 4 for scores greater than 10 and below band 3, band 5 for scores greater than -30 and below band 4 and band 6 for theremaining. The surface characteristic may then be computed as a band number from the ITA band distribution.

[0156] In one embodiment, the surface characterization score may be the mean of the ITA band distribution. In another embodiment, the surface characterization score may be the highest value of the ITA band distribution. In yet another embodiment, the surface characterization score may be the mean value of the highest value of the ITA band distribution (e.g., the mean of the upper quartile ITA bands).

[0157] Segmenting the target surface pixels in the acquired at least one image may be further refined using the determined surface characteristic. In one embodiment, a first pass of target surface pixel segmentation may take place at step 1204 without knowledge about the surface characteristic (e.g., skin type of the owner). Once the surface characteristic is determined (step 1206), the results may used to further refine (step 1204) segmentation of the target surface pixels, i.e. a second pass by using a more narrow definition of the target surface based on the target surface characteristic.Assignment of subgroup-based score

[0158] The subgroup-based score may be assigned based on the surface characterization score, to generate a score that is more relevant to the individual.

[0159] In one embodiment, where the surface characterization score is an aggregate of surface characteristic (e.g., ITA values). The ITA value for each pixel may be compared to the surface characterization score. If the pixel’s ITA is similar, the ITA may be added to a subgroup-based ITA distribution. In one embodiment, the pixel’s ITA is considered similar if the ITA is in the same band or 1 band higher or 1 band lower as the surface characterization score. In another embodiment, the pixel’s ITA is considered similar if the difference between the ITA and the surface characterization score is less than a threshold, say 1.0.

[0160] In another embodiment, the band of the surface characterization score is used to select a method for determining the subgroup-based score. For example, if the surface characterization score is in bands 1-4, pixels are added to the subgroup-based ITA distribution if the pixel is in or near the band of the surface characterization score, in a similar fashion as described above. However, for bands 5-6, the pixels are transformed to the YCgCr colour space, as the detection performance is expected to be better for darker skin tones. Pixels within a subspace of the YCgCr colour space (e.g., Cg>76 and Cg<125 and Cr>136 and Cr<202) may be added to the subgroup-based ITA distribution.

[0161] The subgroup-based score may then be computed from the subgroup-based ITA distribution. In one embodiment, the subgroup-based score may be the mean of the subgroup-based ITA distribution. In another embodiment, the subgroup-based score may be the highest value of the subgroup-based ITA distribution. In yet another embodiment, the subgroup-based score may be the mean value of the highest value of the subgroup-based ITA distribution (e.g., the mean of the upper quartile ITA bands).

[0162] In some embodiments, "subgroup" may refer to a zoomed-in version of the target type detection. In some embodiments, the subgroup may be associated with colour and / or texture of the target surface 1304.

[0163] The subgroup-based score may be normalised based on a risk ratio. In an embodiment, the risk ratio may reflect the impact of getting the type of target surface estimate wrong when subjecting the target surface 1304 to a laser-based application. In one example, a 5% safety margin may be added.

[0164] In another example, the normalization is non-linear, and the subgroup-based score may be adjusted as follows: score * 1.02Ascore. In one embodiment, the normalization may be linear by adding a fixed amount (e.g., 1.05), based on historical type estimation statistics. Normalization may be done such that the normalized value is still within an expected range (e.g., minimum of 1 and maximum of 6).

[0165] A risk factor may reflect whether the estimate should be biased to underestimating or overestimating, and may be a commercially -driven decision. If volume (i.e. attracting many owners) is important, the risk factor may be set to optimistic. If a unit margin is important, the risk factor may be set to pessimistic.

[0166] Estimation of type of target surface

[0167] Estimating a type of the target surface 1304 may may be based on the assigned subgroupbased score and a surface context reference model. In an embodiment, estimating a type of the target surface 1304 may comprise determining a correspondence between the subgroup-based score and a surface context reference model. In an example, the surface context reference model is the dermatological Fitzpatrick scale with 6 classes from light skin to dark skin. The subgroup-based score may be converted to estimate a type of the target surface by looking up the corresponding band in the surface context reference model table described above.

[0168] The estimated type of target surface may be used to estimate the required treatment fluence. E.g., for a beam spot of 6mm and a laser wavelength of 1064nm, target surface types in bands 1-4 result in a treatment fluence of 1.25, while target surface types in bands 5-6 result in a treatment fluence of 0.925.

[0169] The score determination calculations may be applied to images in a historical database which includes target surface type estimates by subject matter experts or actual type estimation metrics of the removal process. The calculations described above may be compared to the database values on an aggregate basis, e.g. the average. The average may then be compared and the difference may serve as a correction factor. In one example, +0.52 may be added to the determined score. In another example, the determined score may be multiplied by 1.18.

[0170] In one embodiment, the historical estimated types may be obtained from type estimation assessment results derived by subject matter experts on the image or observations of the target surface 1304 made in-person by a human assessor. In a further embodiment, the historical scores may be updated by comparing the outcome of the actual removal with the analyzed surface characteristic. In one or more embodiments, the assessment results derived by subject matter experts and / or observations made by human assessors may be stored in a remote storage location and accessed by the type determination module 1404.

[0171] In various embodiments, one or both the subgroup-based score and the estimated type of target surface 1304 may be presented to the owner or other stakeholders (e.g., on a display screen of a user device. In some embodiments, the estimation of the type of target surface 1304 may be based on additional practical and commercial considerations.

[0172] In another embodiment, the owner or the marking may indicate the body part to which the target surface 1304 relates.Colour Correction

[0173] In one or more embodiments, colour correction may be applied to the acquired image prior to further processing by the type determination module 1404.

[0174] In an embodiment, a colour reference marker may be included in the image (e.g., a white sticker). A transformation from the pixel values of the colour reference marker to the known colours of the colour reference marker may then be determined, and the transformation may be applied to the image.

[0175] In another embodiment, scene illumination may be estimated as input to a chromatic adoption transform (CAT). Such an approach may be advantageous when it is not practical to have a colour reference marker in the image.Downscaling.

[0176] In some embodiments, the acquired image may be downscaled prior to processing by the type determination module 1404 and the downscaled version may be used for the various steps / processes performed by the type determination module 1404. Doing so may reduce computational load.

[0177] In other embodiments, only some steps or processes may use a downscaled version of the acquired image while other steps or processes may use the original version. In an example, person detection process may use a downscaled version of the acquired image, while marking detection process may use the original version.

[0178] In one embodiment, downscaling may involve averaging the pixel values in a window (e.g., 2x2 pixels) to a new pixel value.

[0179] In another embodiment, downscaling may involve maxpooling the pixel values in a window (e.g., 2x2 pixels) to a new pixel value. In other words, the highest value of the pixels in a window may be taken as a representative for the downscaling.

[0180] It will be appreciated that a plurality of artificial neural networks (ANN) may be employed in enabling the disclosure. In one embodiment, each detection (e.g., marking detection, target surface detection, reference object detection) may be performed done by a separate ANN. In another embodiment, a single ANN may perform the different detections within one inference, using the same pre-trained weights and parameters.

[0181] Fig. 15 is a view 1500 for explaining estimation of type of target surface according to an embodiment of the present disclosure. A target owner 1502 may have a target surface 1504 at various parts of the body. An imaging device 1506 such as a mobile phone camera may be used to acquire at least one image of the target surface 1504. The acquired image may then be transmittedover a communication network 1508 to a remote system 1510 including a type determination module (as described above).

[0182] In one embodiment, the target surface 1304 may cover a large part of the body, e.g. for hair removal. Image(s) may be acquired corresponding to several parts of the body. Although the images may be aligned by stitching as described elsewhere, for determining the type of target surface, alignment may not be necessary. In one example, there may be disjoint photos of a leg and of an arm. If the body parts have had different exposure to sun light, i.e. different levels of tanning, the estimate of the target surface type may be different for each of the photos. In one embodiment, the target surface type estimations may be performed independently for the different body parts, then: the maximum of the multiple estimations may be used as the overall estimation, or the minimum and the maximum of the multiple estimations may be given, to allow for seasonal adjustments. In an example, the outcome may be “this person has a skin type 2, but their arms have a tanned skin type 3 currently”. This may allow anticipation of considering the skin type 2 for the arms in a later season.

[0183] In one embodiment, the discrepancy between the minimum and maximum may be checked against a historical database for the pair of body parts. If the difference is similar, i.e. within the range of differences observed for the same body parts, tanning may be taken as a reasonable explanation. If the difference is outside the range, it is possible that something went wrong in the process, e.g. wrong lighting conditions. An escalation may then take place, e.g. notifying the photographer to take new images, or to do a different type of examination, e.g. by physical inspection.

[0184] In one embodiment, the acquired image may be uploaded to the remote system 1510 via the communication network 1508, wherein the type determination module 1404 performs detection of target surface areas such as the target surface 1504 and any other relevant areas for subsequent classification and processing of all the detected target areas as described above.

[0185] In another embodiment, the acquired image may be uploaded from the imaging device 1506 to the remote system 1510, wherein detection of the target surface areas takes place. The detected areas may then be transmitted back to the imaging device 1506 such as a mobile phone, where the owner or a relevant operator such as a practitioner can select, remove, or adjust the detected areas. In an embodiment where the imaging device 1506 may not have a display screen, the detected areas may be transmitted from the remote system 1510 to a user device comprising a display screen (not shown), where the owner or a relevant operator such as a practitioner can select,remove, or adjust the detected areas. The new set of detected areas may then be uploaded to the remote system 1510 via the communication network 1508 and analysis may proceed on the new set of detected areas only.

[0186] In yet another embodiment, detection of target surface areas may be performed locally on the imaging device 1506 such as a mobile phone or a user device to which the imaging device 1508 is connected if the imaging device 1506 does not have a display screen. The detected areas may then be displayed on the display screen of the imaging device 1506 or the user device, where the owner or relevant operator can select, remove, or adjust areas. The new set of areas may then be uploaded to the remote system 1510 via the communication network 1508 and analysis may proceed on the new set of areas only.

[0187] In various embodiments, the determined surface characterization score(s), subgroupbased score(s) and / or the estimated type of target surface may be (a) displayed on a display screen of the imaging device 1506 or a display screen of a user device such as a mobile phone or a computer, tablet, laptop etc. (e.g., if different to the imaging device 1506), (b) polled for by a software application running on the imaging device 1506 or a user device to the remote system 1510, and upon receiving the determined surface characterization score(s), subgroup-based score(s) and / or the estimated type of target surface from the remote system 1510, displaying the determined surface characterization score(s), subgroup-based score(s) and / or the estimated type of target surface on a display screen of the imaging device 1506 or a display screen of the user device such as a mobile phone or a computer, tablet, laptop etc. (e.g., if different to the imaging device 1506), (c) presented in an email or as an audio text to the owner on the display screen of the imaging device 1506 or a display screen of a user device such as a mobile phone or a computer, tablet, laptop etc. (e.g., if different to the imaging device 1506). In one embodiment, one or more actions to be taken (e.g., removing hair and / or marking(s) on the target surface based on the estimated type of the target surface) may be displayed on the display screen, e.g., 8 sessions over a period of 6 months. In another embodiment, $30 per week for a period of 6 months. The one or more actions may include predicting a number of sessions required for the removal, predicting a quote of the cost for the removal, making an appointment with a removal operator. In some embodiments, one or more of a session time and location of centres where the hair and / or marking can be removed may be displayed or presented to the user on a display screen or user device to select from and make an appointment.

[0188] Upon completion of estimation of target surface type, one or more settings of a relevant removal device 1512 may be adjusted for removing the hair and / or marking from the target surface 1504 of the target owner 1502. In one example, the removal device 1512 is a laser hair removaldevice or a laser tattoo removal device and the frequency, intensity, wavelength of the laser can be adjusted based on the estimated type of the target surface. In one embodiment, prevention or reduction of side effects may be considered in adjusting the settings of the removal device 1512. In one example, the laser intensity may be adjusted to the skin type of the target surface to prevent or reduce hypopigmentation and hyperpigmentation. In various embodiments, GentleMax Pro may have a wavelength set to 755 nm which may be optimal for skin types I-III and to 1064 nm which may be optimal for skin types IV- VI.

[0189] The present disclosure may provide an automated estimation of type of target surface 1504 based on image analysis as described above. In contrast to estimation of target surface type through direct observation by a human practitioner, the present disclosure may provide improved scalability, may be more interactive, provide a way for estimation to be done at any time, in a more secure or private environment (such as the home of the owner of the marking) and may provide feedback capability, and may improve consistency of estimates.

[0190] One or more embodiments of the present disclosure may provide a target surface type estimation method and system that is more objective, private, scalable and consistent. The estimation or assessment may also be performed at a location remote from the target owner of the target surface and / or the removal device in some embodiments.

[0191] Further, selecting parameters of the removal device (e.g., laser tattoo removal device) used to remove the hair and / or marking on the target surface 1504 based on the estimation of the type of target surface as described above may result in minimal downtime following the one or more iterations of removal session with low thermal impact, and may be suitable for a wide range of types of target surface 1304.

[0192] The present disclosure may assist a target owner of the target surface in performing an initial estimation and getting an estimation of the cost, time, number of sessions etc. that may be required to remove hair and / or marking(s) on the desired target surface without needing another human practitioner or assessor to perform the initial estimations. The target owner may then, based on the estimation of skin type of the target surface, choose to make an appointment with a professional hair / marking removalist (human operator) at a convenient time and / or location. The professional hair / marking removalist may or may not use the estimation of the target surface type in determining one or more parameters of the removal device. Using the above-described target surface type estimation may reduce or eliminate adverse events due to wrong or improper selection of the one or more parameters of the removal device or choice of removal device by the human operator. Marking removal options may be more objective and auditable due to the presentdisclosure (e.g., when the human operator's view differs from the estimated target surface type, determination of removal device and / or parameters of the removal device may be determined in consultation with a dermatologist, the estimation may assist in training operators who may otherwise go ahead with non-ideal selection of removal device and / or parameters of the removal device such as selecting parameters that may either damage the skin of the owner, or not be sufficient to obtain a desired level of removal or hair and / or removal or fading of the marking as discussed above).

Claims

CLAIMS1. A method of assessing concentration of a marking on a target surface, the method comprising: acquiring at least one image of the marking on the target surface; analyzing one or more marking characteristics of a plurality of pixels associated with the marking in the acquired image; determining a concentration score for one or more of the plurality of pixels based on the analyzed one or more marking characteristics; and assessing an overall concentration score of the marking based on the determined concentration score for determining one or more actions to take for removing the marking from the target surface.

2. The method of claim 1, wherein assessing the overall concentration score of the marking further comprises aggregating the determined concentration score with historical concentration values.

3. The method of claim 1, further comprising analysing one or more surface characteristics of a plurality of pixels associated with the target surface in the acquired at least one image.

4. The method of claim 3, wherein assessing the overall concentration score of the marking is further based on the analysed one or more surface characteristics.

5. The method of claim 3, wherein the analysing one or more surface characteristics comprises determining a correspondence between the plurality of pixels associated with the target surface and a surface context reference model.

6. The method of claim 1, further comprising detecting pixel dimensions of a reference object in the acquired at least one image.

7. The method of claim 6, further comprising converting pixel dimensions of the marking into actual dimensions of the marking using the detected pixel dimensions of the reference object and actual dimensions of the reference object.

8. The method of claim 7, wherein assessing the overall concentration score of the marking is further based on the actual dimensions of the marking.

9. The method of claim 3, further comprising segmenting the marking pixels from the surface pixels in the at least one image using the analyzed one or more surface characteristics.

10. The method of claim 1, wherein the one or more actions to take for removal of the marking is displayed on a user interface.

11. The method of claim 1, wherein the marking is a tattoo.

12. The method of claim 1, wherein the target surface is skin.

13. The method of claim 10, wherein the one or more actions comprises determining an impact score based on the overall concentration score.

14. The method of claim 10, wherein the one or more actions comprises predicting a number of sessions required for removing the marking from the target surface.

15. The method of claim 10, wherein the one or more actions comprises displaying on the user interface one or both of a session time and a location to make an appointment for removing the marking.

16. The method of claim 13, wherein the impact score is further based on a cost estimate for removing the marking from the target surface.

17. A system for assessing concentration of a marking on a target surface, the system comprising: an imaging device configured to acquire at least one image of the marking on the target surface; and a concentration determination module communicatively connected to the imaging device and configured to: analyze one or more marking characteristics of a plurality of pixels associated with the marking in the acquired image; determine a concentration score for one or more of the plurality of pixels based on the analyzed one or more marking characteristics; and assess an overall concentration score of the marking based on the determined concentration score for determining one or more actions to take for removing the marking from the target surface.

18. The system of claim 17, wherein the concentration determination module is further configured to assess the overall concentration score of the marking by aggregating the determined concentration score with historical concentration values.

19. The system of claim 17, the concentration determination module is further configured to analyse one or more surface characteristics of a plurality of pixels associated with the target surface in the acquired at least one image.

20. The system of claim 19, wherein assessment of the overall concentration score of the marking is further based on the analysed one or more surface characteristics.

21. The system of claim 19, wherein the concentration determination module is configured to analyse one or more surface characteristics by determining a correspondence between the plurality of pixels associated with the target surface and a surface context reference model.

22. The system of claim 17, wherein the concentration determination module is further configured to detect pixel dimensions of a reference object in the acquired at least one image.

23. The system of claim 22, wherein the concentration determination module is further configured to convert pixel dimensions of the marking into actual dimensions of the marking using the detected pixel dimensions of the reference object and actual dimensions of the reference object.

24. The system of claim 23, wherein the concentration determination module is configured to assess the overall concentration score of the marking based on the actual dimensions of the marking.

25. The system of claim 19, wherein the concentration determination module is further configured to segment the marking pixels from the surface pixels in the at least one image using the analyzed one or more surface characteristics.

26. The system of claim 17, wherein the one or more actions to take for removal of the marking is displayed on a user interface.

27. The system of claim 17, wherein the marking is a tattoo.

28. The system of claim 17, wherein the target surface is skin.

29. The system of claim 26, wherein the one or more actions comprises determining an impact score based on the overall concentration score.

30. The system of claim 26, wherein the one or more actions comprises predicting a number of sessions required for removing the marking from the target surface.

31. The system of claim 26, wherein the one or more actions comprises displaying on the user interface one or both of a session time and a location to make an appointment for removing the marking.

32. The system of claim 29, wherein the impact score is further based on a cost estimate for removing the marking from the target surface.

33. A method of estimating a type of target surface, the method comprising: acquiring at least one image of the target surface; determining a plurality of target surface pixels associated with the target surface in the acquired image;determining a surface characteristic for the target surface based on the determined plurality of target surface pixels; assigning a subgroup-based score to each of the plurality of target surface pixels based on the determined surface characteristic; and estimating a type of the target surface based on the assigned subgroup-based score and a surface context reference model.

34. The method of claim 33, wherein determining a plurality of target surface pixels further comprises determining an initial surface score for each of the plurality of pixels in the acquired image.

35. The method of claim 34, wherein determining a plurality of target surface pixels further comprises categorizing each of the plurality of pixels of the acquired image having the initial surface score greater than a predetermined threshold as the plurality of target surface pixels.

36. The method of claim 33, further comprising determining a plurality of owner pixels in the acquired image associated with a target owner of the target surface.

37. The method of claim 36, wherein determining a plurality of target surface pixels further comprises determining an initial surface score for each of the plurality of owner pixels in the acquired image.

38. The method of claim 37, wherein determining a plurality of target surface pixels further comprises categorizing each of the plurality of owner pixels having the initial surface score greater than a predetermined threshold as the plurality of target surface pixels.

39. The method of claim 33, further comprising determining a surface characterization score based on the determined surface characteristic.

40. The method of claim 39, wherein determining the surface characterization score comprises aggregating the surface characteristic of the plurality of target surface pixels.

41. The method of claim 33, wherein the target surface is skin.

42. The method of claim 33, wherein the surface context reference model is Fitzpatrick scale.

43. The method of claim 33, wherein the surface characteristic is colour distribution.

44. The method of claim 39, wherein the surface characterization score is Individual TypologyAngle (ITA) score.

45. A system for estimating a type of target surface, the system comprising: an imaging device configured to acquire at least one image of the target surface; and a type determination module configured to:determine a plurality of target surface pixels associated with the target surface in the acquired image; determine a surface characteristic for the target surface based on the determined plurality of target surface pixels; assign a subgroup-based score to each of the plurality of target surface pixels based on the determined surface characteristic; and estimate a type of the target surface based on the assigned subgroup-based score and a surface context reference model.

46. The system of claim 45, wherein determining a plurality of target surface pixels further comprises determining an initial surface score for each of the plurality of pixels in the acquired image.

47. The system of claim 46, wherein determining a plurality of target surface pixels further comprises categorizing each of the plurality of pixels of the acquired image having the initial surface score greater than a predetermined threshold as the plurality of target surface pixels.

48. The system of claim 45, further comprising determining a plurality of owner pixels in the acquired image associated with a target owner of the target surface.

49. The system of claim 48, wherein determining a plurality of target surface pixels further comprises determining an initial surface score for each of the plurality of owner pixels in the acquired image.

50. The system of claim 49, wherein determining a plurality of target surface pixels further comprises categorizing each of the plurality of owner pixels having the initial surface score greater than a predetermined threshold as the plurality of target surface pixels.

51. The system of claim 45, further comprising determining a surface characterization score based on the determined surface characteristic.

52. The system of claim 51, wherein determining the surface characterization score comprises aggregating the surface characteristic of the plurality of target surface pixels.

53. The system of claim 45, wherein the target surface is skin.

54. The system of claim 45, wherein the surface context reference model is Fitzpatrick scale.

55. The system of claim 45, wherein the surface characteristic is colour distribution.

56. The system of claim 51, wherein the surface characterization score is Individual TypologyAngle (ITA) score.

Images