Apparatus and method for determining skin parameters within a region of interest on a subject's body - Patents.com

Abstract

An apparatus for determining skin parameters within a region of interest on a subject's body is presented. The apparatus comprises a camera (1) positioned and configured to record images of the region of interest. A control unit (3) of the apparatus is configured to determine the skin parameters within the region of interest by analyzing one or more images of the region of interest recorded by the camera (1) while the region of interest is being manipulated.

Description

[Technical Field]

[0001] The present invention relates to an apparatus and method for determining skin parameters within a region of interest of a person's body. [Background technology]

[0002] The aesthetic outcome of plastic and curative surgery on a patient's face is highly dependent on the surgeon's skill. Plastic surgery is considered to include the fields of aesthetic surgery, reconstructive surgery, and burn surgery, while curative surgery is considered to include oncology surgery, etc. Skin parameters often affect the postoperative aesthetic outcome of such surgery, as well as other anatomical characteristics of the face. Traditionally, surgeons manually inspect the skin before the planned surgery to ultimately define the incision pattern. Skin evaluation and the determination of the incision pattern are highly dependent on the surgeon's experience and knowledge. Defining the incision pattern significantly affects the aesthetic outcome of such surgical interventions. Summary of the Invention [Problem to be solved by the invention]

[0003] It is a general object of the present invention to provide an aid for assisting a surgeon in planning and / or preparing for a surgical intervention. [Means for solving the problem]

[0004] According to a first aspect of the present invention, there is provided an apparatus for determining parameters of skin in a region of interest on a subject's body, the apparatus comprising a camera positioned and configured to record images of the region of interest, and a control unit configured to determine parameters of the skin in the region of interest by analyzing one or more images of the region of interest recorded by the camera while the region of interest is manipulated.

[0005] As a result, preparation and / or planning for a surgical intervention does not only rely on the skill of the surgeon, but is also performed at least in part by the device. This allows for a more standardized assessment of the subject's / patient's skin parameters, ultimately resulting in a more predictable aesthetic outcome of the surgery. Data obtained by such semi-automated or fully automated inspection by the device can further be used to simulate future aesthetic outcomes and demonstrate the simulated results to the subject before surgery. However, not only planning but also the evaluation of whether or not to perform a surgical intervention is supported by the present invention. For example, the demonstration of future aesthetic outcomes can lead to a decision between the surgeon and the subject to forgo a planned surgical intervention.

[0006] Thus, images taken by the camera while the subject's skin in the area of ​​interest is being manipulated contribute to an analysis of the subject's skin in the area of ​​interest. One or more images are analyzed by the control unit, preferably by pattern recognition software and / or computer vision executed by and / or operated on the control unit. This constitutes an entirely new approach for analyzing parameters with the support of computer processing means, thereby allowing for a more standardized categorization of skin characteristics.

[0007] The apparatus includes at least a control unit, e.g., embodied in a computer such as a laptop, and a camera. To enhance the examination, the camera is preferably a 3D camera external to but connected to the computer. The camera may be mounted on a stand, which is part of the apparatus and can be considered an examination stand for examining the subject.

[0008] The purpose of the present device and the method described below is to investigate parameters of a subject's skin within a region of interest. Knowledge of the parameters can subsequently be used for different purposes. The parameters can be used to prepare surgery within the region of interest and help surgeons more clearly evaluate skin characteristics. The parameters can be used to determine an incision pattern within the region of interest. Such an incision pattern can be used in a computer simulation that predicts surgery along the incision pattern and demonstrates a simulated aesthetic result, and / or in computer-controlled automatic incisions along the incision pattern. Various incision patterns can be proposed, resulting in different results in the simulation, which can help the subject and / or surgeon reach a conclusion about a preferred incision pattern. The results of the proposed method can be used for plastic and / or curative surgery. The resulting parameters can also be used for cosmetic advice and / or advice on other skin treatments.

[0009] In a highly preferred embodiment, the parameter is the elasticity of the skin, and therefore in particular the elasticity of the skin in the area of ​​interest. Elasticity in this context may indicate the result of the deformation in response to an operation, relative to some predetermined reference deformation. Elasticity may therefore be measured in the form of an index. In an alternative form, one or more elastic moduli, i.e., Young's modulus, shear modulus, bulk modulus, are determined, for example in units of [Pa]. It is therefore preferable to examine the skin in the area of ​​interest in the same area that will later be treated surgically or in another way. In particular, in the case of a surgical intervention, the surgeon is interested in knowing the parameters of the skin in the area of ​​the intervention. The device allows for a more standardized examination of such parameter(s).

[0010] Skin elasticity can preferably be supported by manipulating the skin. Manipulation preferably refers to physically influencing the skin, i.e., applying a force to the skin, thereby displacing the skin from its state in the absence of such force. Such manipulation is particularly applied to areas of excess skin that may be removed. Manipulation may be performed by a manipulation tool, as described in more detail below, or may be performed manually by a person, such as a surgeon, other medical professional, or other support person. Manipulation preferably includes one or more of shifting, stretching, tightening, twisting, and pushing together the skin in the area of ​​interest, preferably in various directions.

[0011] In another embodiment, the skin parameter is the location of adhesion. In some areas, the skin is not overly firmly attached to the underlying tissue and may be rather mobile, while in other areas or spots (collectively referred to as "sites"), the skin may be fixedly attached to the underlying tissue, bone, etc., and may be largely immobile. The latter sites may be identified in response to manipulation of the skin. For upcoming surgery, the surgeon may also be interested in knowing about any such adhesion sites within the region of interest, the area to be surgically treated. Again, the present device allows for a more standardized examination of such parameters.

[0012] Such attachment sites, also referred to as anchorage locations, can be identified manually or by manipulation tools, particularly by displacing the skin with them. Again, the manipulation preferably includes one or more of shearing, stretching, tightening, twisting, and pressing the skin in the region of interest, preferably in various directions. For example, when a certain manipulation force is applied, the skin may be displaced farther if the manipulation involves an area of ​​the skin far from the anchorage site. Second, the skin may be displaced less if the manipulation involves an area of ​​the skin close to or at the anchorage site. In such a scenario, the direction of the manipulation may contribute to determining the anchorage site. In addition to skin elasticity, the attachment / anchorage site of the skin may also be determined.

[0013] The region of interest may be any area of ​​the subject's body where an incision is desired for plastic surgery or other procedures, and therefore any subsequent procedures may be applied to any such area of ​​the subject's body. However, in a preferred embodiment, the specific region of interest is the subject's face or a specific subregion of the face, or the subject's neck. In such areas of the subject's skin, an incision deviation of even 1 mm can significantly affect the aesthetic outcome. Facial expression is important to human appearance, and although the area of ​​skin on the human face is limited, the skin and its elasticity on the face and / or neck can be highly dependent on the underlying tissue and its elasticity. Tissues may include bone, muscle, fat, etc. Furthermore, skin elasticity may also depend on the skin's adhesion to the underlying tissue, which may be low in the eyelid area and significantly higher in the cheek area, for example.

[0014] In a highly preferred embodiment, the region of interest includes the eyelids of the subject's face. Eyelid surgery has been popular for many years. In eyelid surgery, surgical intervention removes excess skin to tighten the eyelids. In extreme cases, excess skin can affect vision. In other cases, eyelid surgery is performed for aesthetic reasons. Preferably, the region of interest is represented by the upper eyelid. However, the lower eyelid can also represent a region of interest in an alternative approach. Surgical intervention in the lower eyelid can remove excess skin from the lower eyelid, thereby removing the lacrimal sac.

[0015] In various embodiments, the area of ​​interest may include one or more of the cheek, the area between the cheek and the ear, and the area behind the ear, with an incision pattern in one of these areas of interest being preferred for a facelift.

[0016] In other embodiments, the region of interest is defined by an area containing either a malignant skin lesion to be removed, such as a melanomatous spot, or a benign lesion, such as a birthmark or liver spot.

[0017] As described above, the device may preferably include a manipulation tool arranged and configured to manipulate the subject's skin within the region of interest. The manipulation tool preferably produces one or more of the following effects on the skin within the region of interest: shifting, stretching, tightening, twisting, and compressing. Preferably, the manipulation tool allows for manipulation in several of the above-mentioned ways. Preferably, the manipulation tool is configured and arranged to shift, stretch, tighten, twist, and compress the skin in various directions. The manipulation tool preferably manipulates the skin within the region of interest in an automated manner. Additionally, the use of a manipulation tool, which is typically electrically powered, allows for more standardized manipulation of the subject's skin. Furthermore, examination of skin parameters may be performed without contact between the subject and a surgeon or other examiner.

[0018] The operating tool is preferably connected to the control unit via an interface such as a USB and is preferably controlled by the control unit.

[0019] In a preferred embodiment, the manipulation tool includes a mechanical structure for manipulating the subject's skin in a contact state within the region of interest. Such a mechanical structure may be, for example, a pin or other mechanical structure such as a cotton swab or tweezers. In a highly preferred embodiment, the manipulation tool includes a blower for generating an air jet directed toward the region of interest to manipulate the subject's skin. The blower is a simple means for manipulating the subject's skin without contact. The blower preferably includes a fan driven by an electric motor. The fan may be provided within a housing and draws in ambient air from one open end of the housing. Preferably, a nozzle is attached to the other end of the housing for shaping and discharging the air jet. In the present application, it is particularly preferred to generate a small-diameter air jet to blow a focused air jet onto the region of interest. Therefore, the diameter of the nozzle is preferably 0.5 cm or less. With such a small diameter and an appropriate distance between the nozzle and the region of interest, it is even possible to stimulate various sites within the region of interest.

[0020] In a preferred embodiment, the manipulation tool can be operated at various actuation levels. This can be achieved by applying various power levels to the drive of the manipulation tool. As a result, the manipulation tool can be configured to apply various force levels to the region of interest for manipulation purposes. In the case of a blower, the control unit can be configured to cause the blower to eject air at various pressures. In that case, parameters can be determined by the control unit according to the manipulation of the skin in response to the application of various actuation levels.

[0021] The camera is preferably a 3D camera comprising at least two 2D cameras spaced apart from one another and recording 2D images, and a processor configured to calculate a 3D image from the two recorded 2D images. With a 3D camera, the operator / surgeon not only receives a planar 2D image, but also receives information about the third dimension. The processor may be internal to the camera, external to the camera, or carried by a control unit.

[0022] The camera and, if present, the manipulation tool are electrically connected to a control unit. The control unit may be embodied by a computer, such as a laptop. The electrical connection may be wired or wireless. Images captured by the camera are transmitted to the control unit via a suitable interface, such as USBx for wired communication or IEEE 802.11 (WLAN) for wireless communication. With regard to the manipulation tool, the control unit is preferably configured to activate the manipulation tool via a suitable interface, such as one of those mentioned above. In a first embodiment, the camera is activated by the control unit to provide digital data in the form of digital images to the control unit, while the manipulation tool is controlled by the control unit to be switched on and off.

[0023] In a preferred embodiment, the control unit is configured to determine the parameters of the skin in the region of interest not only by analyzing one or more images of the region of interest recorded by the camera while the region of interest is being manipulated, preferably but possibly by a manipulation tool. One or more images of the region of interest taken while the skin in this region is not being manipulated are also analyzed. These images are also called reference images. They allow for a better assessment of the effect of the skin manipulation. The reference image or images can be provided to the control unit from elsewhere or stored therein. This includes one or more reference images taken by a different camera than the camera of the present device. On the other hand, the reference image or images are preferably taken by the camera of the present device. In that case, the camera's resolution, aperture, and other parameters, preferably also the camera's position, are the same for all images. Regardless of the origin of the reference image, the control unit preferably performs a comparison of the one or more images recorded by the camera with the one or more reference images.

[0024] In this embodiment, the region of interest is recorded both while being manipulated and while not being manipulated. All obtained images are captured by the camera of the device. One or more images (abbreviated as "recorded images") recorded while the skin is being manipulated may be taken before or after one or more reference images (abbreviated as "reference images") are taken. Furthermore, images may be recorded for various types of manipulation, particularly for manipulations at various activation levels and / or various directions of manipulation. Note that the scenes of the recorded and reference images are distinguished by the state of manipulation, but additional images may be recorded for various states of the subject's facial expression. For example, if the region of interest is the upper eyelid and the parameter to be determined is the elasticity of the skin of the upper eyelid, the recorded and reference images are preferably taken while the subject has their eyes closed, because it may be difficult to evaluate the elasticity of the manipulated skin when the eyes are open, and the eyelid skin may already be folded. More generally, the facial expression state may include one or more predetermined states, and a person may change between the states by, for example, activating muscles.

[0025] Some image recordings may also include video recordings, whether in an operational or non-operational state.

[0026] When the comparison between the reference image and the recorded image is performed by a computerized method that represents a comparison between the non-manipulated and manipulated states of the skin, it is preferable to have the control unit identify one or more features in the reference image and identify the same one or more features in the recorded image. The feature identification and comparison may be performed by computer vision or pattern recognition software executed by the control unit. In an alternative sequence, the features are first identified in the recorded image and then searched for in the reference image.

[0027] A feature is considered a characteristic in an image that can be computationally detected based on color variation, 3D height or depression, and / or based on one or more other parameters. In real life, a person's facial features may correspond to anatomical features such as eyebrows, eyelid gaps, eyelid contours (particularly the lower contour of the upper eyelid and the upper contour of the lower eyelid), or lid folds. Further features may be individual human features such as birthmarks or liver spots, rather than regular features of the eye and / or facial anatomy. Any feature within the region of interest that is detectable by pattern recognition in one of the recorded image and the reference image may be of interest for analysis.

[0028] In the above example of the eyelid as the region of interest, the primary features extend horizontally, whether linear or curved. To manipulate such features, the skin is preferably manipulated vertically, i.e., the manipulation tool preferably applies a force having at least some vertical component. If the manipulation tool is an air jet, it may be preferably directed at the region of interest at an angle such that the skin is moved vertically either upward or downward, and even more preferably subsequently upward and downward, or vice versa. Therefore, directing the air jet directly, i.e., perpendicular to the region of interest, is less preferred. In general, it is preferred to manipulate essentially horizontally extending features vertically and / or essentially vertically extending features horizontally. Features that extend primarily vertically may include wrinkles near the crow's feet.

[0029] In a next step, the control unit is preferably configured to determine the distance between one or more features (hereinafter collectively referred to as "features") identified in the reference image and the same features identified in the recorded image. To this end, the control unit may first align and scale the reference image with / against the recorded image, e.g., by image processing, thereby ensuring that the areas to be compared with each other are of the same size, resolution, brightness, etc., at least within a tolerance, which facilitates the identification of features and their subsequent tracking in other images. This also makes it possible to use a common scale for identifying the positions of features in the images and thus determine the distance between the positions of the features according to this scale. Thus, the distance between the position of the feature in the non-operated state and the position of the same feature in the operated state is determined.

[0030] In a next step, the control unit is preferably configured to determine a parameter depending on the determined distance. For example, the greater the distance between features, the higher the elasticity of the skin in this region of interest. The smaller this distance, the lower the elasticity of the skin. If two or more features are identified to be displaced in the same direction in response to the manipulation, an average distance can be calculated and can be considered or contribute to a measure of the elasticity of the skin in the region of interest.

[0031] While the above embodiments focus on features that change in response to manipulation, other features may also contribute to parameter determination. For example, features may not necessarily change in response to manipulation. And / or features may disappear in response to manipulation, at least in the recorded image. And / or new features may appear in response to manipulation.

[0032] Preferably, any such classes of features may contribute to the determination of the parameter, and each class is preferably weighted in its contribution to the parameter. This may be done by a formula. In one embodiment, the formula may be as follows:

[0033] ε=w1*F gr1 +w2*F gr2 +w3*F gr3 +w4*F gr4

[0034] where: ε is an index representing the elasticity or resilience of the skin. F grx are characteristics of the same class. gr1 is the class of displaced features. gr2 is the class of undisplaced features. gr3 is an emerging class of features. gr4 is the class of missing features. w1 to w4 are weighting coefficients.

[0035] F gr1 The features of may be determined individually. For example, in the case of gr1, F gr1 may represent the average distance for all displaced features. For example, in the case of gr2, F gr2 may represent the number of non-displaced features. For example, for gr3 and gr4, the corresponding F gr3 and F gr4 may represent the number of newly appeared or disappeared features, respectively. By adjusting the weights w1 to w4, we can determine how the class affects the resilience measure. For example, the class F of displaced features gr1 may be the most weighted, i.e., w1>w2, w3, w4.

[0036] While the determination of the parameters so far relies on analysis of recorded images and possibly reference images, additional inputs and corresponding images can be taken into account. For example, if the manipulation tool is configured to function at various actuation levels, images recorded at various actuation levels are taken into account when determining the parameters. The higher the actuation level of the manipulation tool, the greater the displacement of the features can be expected, and vice versa.

[0037] In the case of a blower, the distance between the nozzle and the region of interest, specifically the distance between the nozzle and the point where the air jet meets the region of interest, may be relevant to determining the parameter. For example, the smaller the distance, the greater the expected displacement of the feature. And the greater the distance, the smaller the expected displacement. In a preferred embodiment, a distance sensor is provided to measure such distance, for example, to measure the distance between the blower nozzle and the skin.

[0038] Other data may contribute to the determination of the parameters, in particular personal data including one or more of the subject's age, sex and place of origin. If the parameter is elasticity, there may be a relationship between the elasticity and the subject's age and / or sex and / or place of origin. Such data may be collected in advance and provided to the control unit so as to be available for the calculation of the parameters.

[0039] In the above embodiment, the determination of the parameters is performed by a computer by an algorithm implemented in software executed by the control unit after the recorded images and possibly the reference images are available, but other steps can also be performed in an automatic manner by a computer, such as:

[0040] The camera is activated to record a reference image of the region of interest while the manipulation tool is deactivated, for example, in response to deactivation of the manipulation tool, or at the start of the inspection method. For example, when the subject is in a reference position relative to the camera and manipulation tool, the operator can activate a program for determining parameters, and in a first step, activate the manipulation tool to manipulate the skin in the region of interest. In response to the activation, for example, immediately or with a predetermined delay, the camera can be controlled to record an image during the manipulation. In response to the image being recorded, the manipulation tool is automatically stopped. In response to the stopping of the manipulation tool, a reference image is recorded by the camera and provided to the control unit. In response to receiving the reference image, the control unit can automatically start determining the parameters.

[0041] Regarding the hardware equipment of the present device, a stand is preferably provided, and the camera and / or operation tool are attached to the stand. The stand may be configured to correspond to a stand used for ophthalmic examinations at an ophthalmologist, and any device can be swung into position in front of a subject seated in a chair. Thus, the stand may include a chair or bed on which the subject can sit or lie to take an examination position facing the camera and / or operation tool. The camera and / or operation tool are preferably attached to a rack in a position that allows the subject to record or operate them when in the examination position.

[0042] The manipulation tool is preferably slidably mounted on a stand or rack. In particular, if the manipulation tool is a blower, the air jet can be adjusted to suit subjects of different heights by adopting different positions. Preferably, the position of the blower is electrically adjustable, and the control unit is configured to position the blower at a predetermined position.

[0043] Additionally or alternatively, the manipulation tool, e.g., the blower, is pivotally mounted on a stand to allow tilting of the blower, where the tilt may be electrically adjustable, in particular the control unit being configured to position the blower in a predetermined tilt position allowing pushing up or down on the skin.

[0044] All the above observations also apply to the camera, which may be slidably mounted on a stand or rack so that it can be adjusted to suit subjects of different heights. Preferably, the position of the camera is electrically adjustable, and the control unit is configured to position the camera at a predetermined position. It is also preferred to position the camera at a predetermined distance relative to the subject's face, more preferably relative to the region of interest. Furthermore, it is preferred that the one or more images are recorded with a standardized background light that allows good illumination of the scene and the region of interest, respectively.

[0045] This positioning step may be performed before recording an image. To that end, the control unit preferably controls the camera in one or both of positioning the camera and recording one or more images. The control unit also preferably controls the manipulation tool in one or more of positioning the manipulation tool, actuating the manipulation tool with a predetermined power, and switching the manipulation tool between various actuation levels.

[0046] In another embodiment of the hardware equipment of the device, the device may include a manipulation table with a camera and / or manipulation tool, which is either directly attached to the table or attached elsewhere but aligned towards the table.

[0047] Preferably, the apparatus comprises an output unit for outputting the determined parameters. The output unit may be a display for showing the determined parameters. In particular, the control unit may be configured to automatically display the parameters once determined. Furthermore, the display may be controlled by the control unit to present the recorded image and / or the reference image. In such an embodiment, the determined parameters may be color-coded on the display, in particular color-coded within a region of interest of the displayed recorded image and / or the displayed reference image.

[0048] In a further embodiment, the parameters, possibly in combination with other parameters, serve as a basis for generating an incision pattern for the region of interest for surgical intervention. In one embodiment, the incision pattern is generated from scratch by the control unit.

[0049] In order to generate a subject-specific incision pattern, a model of the subject's region of interest is preferably generated by the camera and control unit. In a simple approach, a reference image can serve as the model, and / or a 3D scan may be performed by the camera, resulting in a 3D model of at least the region of interest, which also serves as the basis for the coordinates describing the incision pattern to be generated or selected.

[0050] In another embodiment, a pre-existing incision pattern may be proposed based on at least the determined parameters. One or more pre-existing incision patterns, which may be considered as reference incision patterns, are preferably stored in the control unit or in a database of the surgeon or hospital, and / or in another remote location, such as a cloud, accessible to at least the control unit. Such a set of incision patterns may constitute proposed reference incision patterns for various values ​​of the at least one determined parameter. The stored reference incision patterns may also vary depending on other parameters, such as eye width. Eye width may be determined, for example, by the distance between the medial and lateral eyelid commissures.

[0051] The set of reference incision patterns may be selected in an automated manner by the control unit, or may be selected from a database or cloud initiated by the control unit. In such a scenario, the control unit may input the determined parameters and possibly geometric dimensions within the region of interest and / or one or more other parameters such as age, gender, etc. In that case, the set of reference incision patterns assigned to the input parameters is automatically selected.

[0052] In a possible next step, the selected or pre-selected incision pattern may be modified taking into account the determined parameters. For example, individual incisions of the incision pattern may be extended and / or shortened in length and / or individual incisions of the incision pattern may be offset relative to other incision(s) of the incision pattern and / or relative to one or more features of the subject within the region of interest. When adapting an existing incision pattern, the control unit may preferably computer-generate the adapted incision pattern from the existing incision pattern and / or computer-suggest two or more adapted incision patterns for the surgeon or subject to choose from.

[0053] If the region of interest is the eyelid, the incision pattern may include at least two incisions, an upper incision line and a lower incision line. However, additional incision lines may be envisioned. In embodiments where the computer-aided generation of the incision pattern depends on the determined parameters, a program running on the control unit or cloud may, for example, propose or set the upper incision line, determine and set the number of any additional incision lines, and propose and set the lower incision line. The setting of the incision lines preferably includes their unique definition, for example, by coordinates, preferably 3D coordinates.

[0054] For example, the upper and / or lower incision lines of the selected or preselected incision pattern may be raised to account for the determined higher elasticity, or alternatively, the upper and / or lower incision lines of the reference incision pattern may be lowered to account for the determined lower elasticity, all in the vertical dimension.

[0055] Preferably, the control unit is configured to simulate facial expressions at least within the region of interest, taking into account the generated and / or selected incision pattern, allowing the aesthetic results to be demonstrated to the subject. Once the incision pattern is determined, the control unit is configured to apply an image editing routine that simulates the effect of a virtual surgery with the determined incision pattern based on one or more reference images taken without skin manipulation. Such image editing routines are known and can be adapted and / or adjusted for specific purposes. The edited images are preferably displayed to the subject to provide an idea of ​​the aesthetic facial appearance after surgery with each incision pattern. Such simulations can be performed for various states of facial expression, such as with eyes closed and eyes open. Thus, two images taken of a subject with two different facial expressions are edited to simulate the subject with eyes open and closed after surgery. In particular, the edited image with the eyes open is useful for demonstrating the eyelid gap resulting from surgery. Surgery typically affects eyelid gap, i.e., eye opening, which in turn affects the overall facial expression (tired / attentional), and therefore such simulations may further assist the subject and / or surgeon to assess the effectiveness of surgery.

[0056] Preferably, taking into account the preferred 3D characteristics of the recorded image, any simulated condition represented by the edited image may be viewed by the patient in 3D. Preferably, the subject can modify and / or shift individual incision lines in the GUI of the display and immediately obtain the aesthetic result displayed on the screen. In such a case, it may be preferable that the program defines possible incision lines in response to the determined elasticity, and only incision patterns that fit the determined elasticity are provided for selection.

[0057] In another embodiment, the selection and / or determination of the incision pattern is assisted by an ANN (artificial neural network), thus preferably a cloud, i.e. an artificial intelligence component located remotely from the device.

[0058] Machine learning algorithms, such as those included in ANNs, build models based on sample data, known as training data, to make predictions or decisions for new input vectors. Thus, models built from training data are configured to perform accurately on new, unknown input data after experiencing the training data.

[0059] In such embodiments, the ANN preferably comprises a supervised learning algorithm that is trained by providing training data in combination with outputs associated with such training data, and such supervised learning algorithm may include one of a support vector machine, linear regression, logistic regression, naive Bayes, linear discriminant analysis, similarity learning decision tree, K-nearest neighbor algorithm, or neural network.

[0060] Such an ANN is preferably supplied / input with at least preoperative images of the subject's region of interest. Such preoperative images may be captured by a camera of the device. Preferably, the preoperative images are 3D images. For example, one or more reference images captured for parameter determination can be used. Images of the subject in various situations, such as with the eyes open and closed (in the case of eyelid surgery), can be used. In addition to such images, the incision pattern is used as input for the ANN. Additional parameters, such as gender and age, can be used as input. In response to such input, the ANN provides an output in the form of an expected postoperative result, preferably in the form of an image depicting the subject if surgery according to the incision pattern is performed on the subject.

[0061] Such an ANN is preferably trained with corresponding input-output training data representing training examples in which a surgical procedure involving a predetermined incision pattern is performed on a subject, resulting in the postoperative appearance of the region of interest affected by the surgical procedure. Thus, the training data obtained from the training examples includes an input vector V± = {reference preoperative image; incision pattern; ...}, to which an output including at least a postoperative image of the subject (which also includes at least a postoperative image of the region of interest) is assigned. To train the ANN, as many surgeons or hospitals as possible preferably provide the ANN with preoperative and postoperative images in combination with the incision patterns applied during surgery. Through this training, the learning algorithm builds a model capable of predicting output data, here the expected visual appearance of the region of interest of a subject after surgery is performed with the specified incision pattern.

[0062] Preferably, the control unit is configured to connect to the ANN to utilize the ANN and / or train such ANN by providing training data including input vectors and corresponding outputs. The latter can be performed only after surgery, considering that the results of the surgery represent the output data. The former can be used in preparation for surgery, for example, by providing the ANN with patient input vectors including patient images and one or more determined parameters and / or other parameters. As an additional input, the surgeon defines or selects an incision pattern to be applied to the patient. The ANN may return the expected postoperative result, i.e., the appearance of the subject's region of interest after surgery. By presenting various incision patterns, various possible surgical results can be displayed to the patient in order to select the most preferred one.

[0063] The incision patterns themselves may be stored at the hospital or in a general database of incision patterns, for example, in the cloud. In such a database, preferred and / or individual incision patterns may be stored and tagged by individual surgeons with preferences for particular incision patterns. Such a database may also hold incision patterns based on fashion trends, such as those that result in shallow or deep eyelid creases.

[0064] Preferably, in the ANN, visual feature extraction is applied. Features are extracted from the input image. For example, given an input vector V i The extracted vector representing a portion of is an m-dimensional vector, where m is preferably greater than 32 (e.g., m=64 or 128), and typically less than or equal to 512 or 1024. ANN extraction produces vectors with a small memory footprint. The extraction can be constrained to ensure that all vectors are normalized to the same base length, thus enabling more meaningful comparisons between vectors. Each vector may be obtained, for example, at the output of an L2 normalization layer. Each vector is preferably a 128D vector normalized using the L2 norm. Thus, each vector component value is less than or equal to 1. Other normalization schemes are also contemplated. Nevertheless, normalizing the vectors to all the same length allows for more meaningful comparisons.

[0065] The incision pattern automatically generated by the program / control unit or selected by the subject and / or surgeon in response to various automatically preselected suggestions by the control unit can be further used in various ways. Preferably, the device includes an incision tool, preferably an optical incision tool such as a laser, most preferably a carbon dioxide (CO2) laser, provided for non-contact incising the lines of the incision pattern into the subject's skin, thus automatically performing at least the incision following the incision pattern with little or no intervention from the surgeon. Preferably, the incision tool is attached to a robotic arm, the movement of which is preferably computer-controlled, e.g., via a control unit. In this case, the control unit not only assists in determining the incision pattern but also assists in or performs the execution of the incision pattern on the patient. This, of course, requires accurate and safe positioning of the patient. Such incision with a laser has the advantage over manual incision with a scalpel in that the interaction is non-contact. Furthermore, when using a scalpel, placing the scalpel directly on the skin to prepare the incision can displace the skin and potentially deviate from the intended incision path. In a similar manner, the actual incision made by the laser does not apply force to the skin. Instead, when using a scalpel, contact incision and the force applied during incision can cause unintended displacement of the skin, leading to possible deviation from the intended incision path.

[0066] In another embodiment, the device comprises a marking tool for writing or ablation, preferably an optical marking tool such as a laser, most preferably a carbon dioxide (CO2) laser.

[0067] In other embodiments, the device comprises an illumination tool configured to project an incision pattern onto a region of interest of a subject seated on a stand. This may indicate to the subject where the intended incision will be made. In another variation, the control unit is configured to superimpose the incision pattern onto the region of interest of the displayed recorded image and / or reference image. Such superimposed images may assist the surgeon in subsequent surgical interventions. Finally, incision lines may also be projected onto the region of interest before or during a surgical intervention to guide the surgeon.

[0068] According to a second aspect of the present invention there is provided a use of an apparatus according to any of the previous embodiments for preparing a surgical intervention in a region of interest of a subject.

[0069] According to a third aspect of the present invention, there is provided a method for determining parameters of skin within a region of interest on a subject's body, wherein the subject's skin is manipulated within the region of interest, one or more images of the region of interest are recorded while the region of interest is being manipulated, and the parameters of the skin within the region of interest are then determined by computer processing by analyzing the one or more recorded images.

[0070] Preferably, the parameter is elasticity and the region of interest includes the eyelids of the subject's face.

[0071] Preferably, the manipulation of the subject's skin within the region of interest is computer-activated, which may include computer-positioning a manipulation tool, such as a blower, and / or computer-activating a manipulation tool, such as a blower, to generate a jet of air directed at the region of interest, thereby causing one or more of shifting, stretching, tightening, and compressing the skin within the region of interest.

[0072] Preferably, the recording of the one or more images may be initiated by the computer by activating the camera to record the one or more images. Further, the camera may be positioned by the computer.

[0073] In a preferred embodiment, one or more reference images of the region of interest are taken, preferably by the same camera, without manipulating the subject's skin within the region of interest, and the parameters are determined based on a comparison of the one or more reference images with the one or more recorded images.

[0074] Preferably, when the region of interest is the subject's face, the images are captured by the camera with various facial expressions of the subject affecting the region of interest. In the case of an eyelid representing the region of interest, images may be captured with one or more of the following facial expressions: eye open, eye wide open, eye closed, eye convulsively closed, and eyebrow raised. Preferably, each facial expression is recorded twice, once with and once without manipulation.

[0075] In another embodiment, the determining step includes identifying one or more features in one or more reference images, identifying one or more features in one or more recorded images, determining a distance between one or more features identified in the one or more reference images and one or more features identified in the one or more recorded images, and determining a parameter according to the determined distance. Furthermore, the parameter may be determined according to one or more of a predetermined power at which the manipulation tool is operated or a predetermined force that the manipulation tool exerts on the skin, a distance between the nozzle of the blower and the skin in the area of ​​interest, and personal data including one or more of the subject's age, gender, and place of birth. All of these steps are preferably computer-processed.

[0076] In one embodiment, the control unit controlling the camera to capture one or more reference images of the region of interest while the manipulation tool is not activated; activating the manipulation tool to manipulate the skin within the region of interest; controlling the camera to record one or more images of the region of interest while the manipulation tool is activated; The parameters may be determined by comparing one or more reference images with one or more recorded images.

[0077] The determined parameters may then be displayed, particularly in combination with the recorded image and / or the reference image, for example by color coding the region of interest according to the parameters.

[0078] Preferably, the incision pattern is computer-generated for the region of interest and is based on at least the determined parameters and possibly further parameters. The incision pattern may be generated from scratch or by adapting an existing incision pattern that depends on at least the parameters. Such an incision pattern may be used only for preparation of the surgical intervention or may additionally be used for the actual surgery, i.e., when a computer-controlled incision tool, preferably a computer-controlled laser, incises the subject's skin in the region of interest along the incision pattern.

[0079] Such incision patterns may be projected onto the subject's region of interest by an illumination tool or may be displayed on the recorded image and / or the reference image.

[0080] According to a fourth aspect of the present invention there is provided a computer program element comprising computer program code means configured to carry out a method according to one of the method embodiments when executed on a control unit.

[0081] The invention will be better understood, and further objects will become apparent, when the following detailed description of the embodiments thereof is considered, such description making reference to the accompanying drawings. [Brief explanation of the drawings]

[0082] [Figure 1] 1A-1C are diagrams illustrating scenes depicting a schematic apparatus and method according to an embodiment of the present invention; [Figure 2]1 is a block diagram illustrating an apparatus according to one embodiment of the present invention. [Figure 3] 1 shows sketches of various conditions of a subject's eye, which represent images recorded during the method, according to one embodiment of the present invention. [Figure 4] 1 is a flowchart illustrating a method according to one embodiment of the present invention. [Figure 5] 4 is a flowchart illustrating a variation of a method according to an embodiment of the present invention. [Figure 6] 4 is a flowchart illustrating a variation of a method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0083] 1 shows a schematic apparatus and a scene illustrating a method according to an embodiment of the present invention. The apparatus arrangement comprises a stand 7 that can be installed in an examination center, a doctor's office, a hospital, or any other office. The stand 7 is provided for examining parameters of a subject's skin.

[0084] Specifically, the stand 7 comprises a chair 71 for the subject / patient to sit in front of a rack 72 and, optionally, in front of a table 73. The rack 72 supports a camera 1 and an operating tool 2, which will be described in more detail below. The camera 1 and the operating tool 2 are each mounted on assigned shelves 74, 75 of the rack 72, which shelves 74, 75 are here mounted to the rack 72 so as to be slidable in the z-direction.

[0085] In another embodiment, the stand comprises a table, such as an operating table, instead of a chair, for the subject / patient to lie on, in which case the camera 1 and / or the operating tool 2 may be positioned accordingly, for example in a rack above the patient.

[0086] The camera 1 and the manipulation tool 2 are electrically connected to a computer 6, such as a laptop. The electrical connection can be wired, as shown in FIG. 1, or wireless. The computer 6 typically comprises a control unit 3 in the form of a processing unit of the computer 6, and a display 4. Via a suitable interface, images captured or recorded by the camera 1 are provided to the control unit 3. The control unit 3 is further configured to operate the manipulation tool 2 via a suitable interface. Thus, in a first embodiment, the camera 1 provides digital data in the form of digital images to the control unit 3, and the manipulation tool 2 is controlled by the control unit 3 so as to be at least switched on and off.

[0087] However, in other embodiments the camera 1 may also be controlled by the control unit 3, for example it may be activated to record images and to stop recording images.

[0088] In a further embodiment, the position and / or orientation of the camera 1 and / or the handling tool 2 may be controllable by the control unit 3. For example, the shelves 74, 75 supporting the camera 1 and the handling tool 2 are electrically movable along the z-axis of the rack 72. For this purpose, each shelf 74, 75 may be assigned an electric motor (not shown) so that, via the control unit 3, the vertical position z of the shelf 74, 75, and therefore of the camera 1 and the handling tool 2, can be adjusted depending in particular on the height of the subject P to be examined.

[0089] In this scenario, a portion of the skin of subject P is desired to be inspected by the apparatus as shown. In particular, a portion of the skin on the subject's face is inspected, for example, from the perspective of elasticity. In this example, it is assumed that the region of interest is the eyelid of subject P. Therefore, both camera 1 and manipulation tool 2 are directed toward one eyelid of subject P.

[0090] The manipulation tool 2 is intended to manipulate the subject P's skin within the region of interest. Manipulation includes any deformation of the skin within the region of interest, including pulling or pushing the skin within the region of interest. The results of such manipulation are then recorded by the camera 1. Here, the manipulation tool 2 is embodied as a blower. The blower has a housing containing a fan and a nozzle 21 within or attached to the housing for blowing and / or focusing air. Thus, the blower generates an air jet AJ directed toward the region of interest on the subject's face. In FIG. 1, the air jet AJ is schematically indicated by an arrow. During manipulation, the skin within the region of interest or subregion of interest is pulled and / or folded. In the absence of a controllable manipulation tool, manipulation may be performed manually by a responsible person.

[0091] The deformation serves as an indicator of the elasticity of the subject's skin within the region of interest, which may vary from subject to subject. The deformation is recorded by camera 1 in the form of one or more images or as a video and transmitted to control unit 3 for evaluation. Control unit 3 can then analyze the one or more recorded images and automatically derive the elasticity of the subject's skin within the region of interest. The elasticity value can then be displayed to the operator, for example on display 4 of computer 6.

[0092] In a further embodiment, one or both of the camera 1 and the manipulation tool 2 are pivotally mounted on a rack 72 of the stand 7. In addition to being vertically adjustable, the devices 1 and 2 are tiltable in the vertical plane xz, for example by an angle α. This may result in even better positioning of the camera 1 and / or the manipulation tool 2 relative to the region of interest. Again, such tilting may be controllable by the control unit 3.

[0093] 2 is a block diagram illustrating an apparatus according to an embodiment of the present invention. A control unit 3 is interconnected with a camera 1 and an operating tool 2. Here, a straight arrow between the camera 1 and the control unit 3 represents the transmission of one or more images recorded by the camera 1 to the control unit 3. A dashed arrow in the opposite direction indicates optional control of the movement and / or position of the camera 1 by the control unit 3. This may include controlling the trigger 11 of the camera 1 to take one or more images. And / or this may include controlling the position of the camera relative to the rack 72, for example, by an electric motor 12 currently assigned to the camera 1, which may in fact be assigned to the shelf supporting the camera 1.

[0094] The straight arrow between the manipulation tool 2 and the control unit 3 represents the transmission of a control signal, such as on / off, to start and stop the manipulation tool 2. In this example, the actuator of the manipulation tool 2 may be, for example, a blower fan 21. The dashed arrow between the control unit 3 and the manipulation tool 2 indicates optional control of the positioning of the manipulation tool 2 by the control unit 3. This may include controlling the position of the manipulation tool 2 relative to the rack 72, for example, by the electric motor 22 currently assigned to the manipulation tool 2, which may in fact also be assigned to the shelf supporting the manipulation tool 2.

[0095] 2 also shows an example of an artificial neural network 100 on the cloud, separate from the device. The double arrow indicates that the control unit 3 may request a possible surgical outcome in response to providing an image of the area of ​​interest and a possible incision pattern. The double arrow further indicates that in a preferred embodiment, a surgeon or hospital may train the ANN 100 by supplying training data in the form of pre-operative images and incision patterns, and corresponding post-operative images.

[0096] FIG. 3 shows sketches of various conditions of a subject's eye, which may simultaneously represent images recorded during the process, according to one embodiment of the present invention.

[0097] Figure 3a) shows a sketch of a human eye with the upper eyelid F3 open. The upper eyelid F3 and the lower eyelid F4, or more precisely, the lower contour of the upper eyelid F3 and the upper contour of the lower eyelid F4, define the eyelid gap F8 and enclose the eye itself, including the vitreous body. The upper eyelid F3 supports the eyelashes, as shown in Figure 3a), as does the lower eyelid F4, which is not shown. F9 denotes the iris of the eye, which encloses the pupil. At the top, the eyebrow F1 is shown. Between the eyebrow F1 and the upper eyelid F3, or more precisely, between the eyebrow F1 and the lower contour of the upper eyelid F3, an eyelid crease is indicated by F2.

[0098] Figure 3b) shows a sketch of a human eye in a facial expression with the upper eyelid F3 closed, i.e. the iris F6 and pupil are covered by the upper eyelid F3, which in this state is in contact in a simplified manner with the upper contour of the lower eyelid F4. Between the eyebrow F1 and the lower contour of the upper eyelid F3, the eyelid crease F2 may still be visible, but by way of example, another crease F5 may become visible which may not be visible / may be covered by the upper eyelid in the open eye state according to Figure 3a).

[0099] In general, what is shown schematically as the anatomical structure of the human eye can be recognized as a feature Fx for computer vision or pattern recognition when examining an image of the human eye. For example, each of the contours shown by F1, F2, F3, F4, F5 can be a feature that is computationally detected, for example, based on color variation, 3D height or depression, and / or one or more other parameters for which the recorded image is analyzed.

[0100] The current features Fx are essentially all curves that run primarily horizontally due to the anatomy of the human eye. Furthermore, one or more of these features F1-F5 may change their horizontal position z as the subject moves their eyelids by contracting the corresponding muscles.

[0101] The additional features may not be regular features of the eye and / or facial anatomy, but individual features of the subject, such as birthmarks or liver spots. Any feature within the region of interest that is detectable by computer vision can be used to determine the parameters.

[0102] Therefore, what is shown in Figures 3a) and 3b) is not only a schematic diagram of a human eye in two different states, "eyes open" and "eyes closed", but can also represent a reference image taken by a camera or camera 1 in Figure 1.

[0103] FIG. 3c) shows the region of interest of the same subject in a different state. In this state, the subject keeps their eyes closed while an external manipulation tool in the form of a blower generates and directs an air jet AJ at the closed upper eyelid. In response to the manipulation of the upper eyelid, some of the features Fx identified in the unmanipulated closed-eye state according to FIG. 3b) may be displaced, some may remain in the same position, some may disappear, and / or some new features may appear and be newly detected in response to the manipulation. In the specific example of FIG. 3c), features F1, F3, and F4 remain in approximately the same position compared to FIG. 3b). Feature F2 may be slightly displaced along the vertical axis z to a position indicated as F2′, feature F5 may be strongly displaced to feature F5′, and new features F6′ and F7′ may appear in response to the manipulation. For example, a crease or contour F7' may appear directly where the air jet AJ meets the upper eyelid, and another crease or contour F6' may appear between F5' and F7'.

[0104] Figure 3d) shows the superposition of the two images of Figures 3b) and 3c), i.e., the superposition of an image of the closed eye recorded without manipulation and an image of the closed eye recorded under the influence of manipulation of the upper eyelid. For the two displaced features F2' and F5', the vertical distances d2 and d5 can be determined relative to their original positions on the z-axis (F2 and F5) on the z-axis (Figure 3b). Here, the vertical distances d2 and d5 are determined or measured in the vertical plane dividing the pupil. Alternatively, the vertical distance can be measured at the peak of the z-axis of each feature or at the center of the eye in the horizontal direction, etc.

[0105] Therefore, a comparison of the features identified in the images taken under various operating conditions can allow for the determination of the parameters of the skin in the region of interest, here the skin of the upper eyelid. For example, the larger the distances d2 and d5, the more elastic the skin in this region of interest. The smaller the distances, the less elastic the skin of the eyelid.

[0106] However, features other than the displaced features may also contribute to the determination. In this example, features F1, F3, and F4 remain essentially unchanged in the z-direction, features F6' and F7' may newly appear in response to the manipulation, and other features may disappear in response to the manipulation. All such classes or groups of features may contribute to the determination of the parameter.

[0107] Within a group, individual features may be weighed against each other, taking into account that their effect on elasticity may depend on their location. For example, the location of features closer to the access point of a manipulation tool may be more likely to be affected by manipulation than features further away. Concurrently or additionally, the anatomical structure, and therefore the absolute location of a feature, may affect its response to manipulation; features close to or connected to muscle or underlying tissue may require more force to displace than features in areas that are only weakly connected to the underlying tissue. Additionally and / or alternatively, manipulation force may be taken into account when determining parameters of the skin within a region of interest.

[0108] FIG. 3e) shows the subject's eyes closed without any manipulation, i.e., it is identical to FIG. 3b), but shows two incision lines C1 and C2 projected onto the subject's face, for example. These incision lines C1 and C2, particularly their arrangement and shape / curvature, as well as the horizontal position of their intersection, can be automatically determined from an elasticity measure in one embodiment. The area between the incision lines C1 and C2 may ultimately be the skin that will be removed by cutting the two incision lines C1 and C2 during surgical intervention. After removing the excess skin, the remaining skin is joined at the incision lines C1 and C2, applying tension to the skin of the upper eyelid.

[0109] In another embodiment, the image in Fig. 3e) may represent a reference image of the subject with eyes closed as in Fig. 3b), which includes superimposition of the incision lines C1, C2, e.g., by image editing software. The resulting image in Fig. 3e) may then be displayed to the subject and / or surgeon for further reference.

[0110] FIG. 4 shows a flowchart illustrating a method according to one embodiment of the present invention. In step "START," the method begins. At the start of the method, it is assumed that the subject / patient is in an examination position, e.g., at a seat or table, e.g., on stand 7, and is ready to be examined, as shown in FIG. 1 . It may be assumed that the camera and manipulation tool are ready to move and positioned relative to the subject, and in particular relative to the region of interest on the subject's body, to perform the respective tasks described above and below. In step S1, the subject's skin is manipulated within the region of interest, preferably with a manipulation tool such as a blower. Starting from the manipulation, or thereafter, one or more images of the region of interest are recorded in step S2, at least while the region of interest continues to be manipulated. In step S3, parameters of the skin within the region of interest are determined from an analysis of at least one or more images recorded from the manipulated region of interest. In optional step S4, the determined parameters are displayed on a display and shown to the subject being examined and / or the person administering the examination. The method may then "end" in this embodiment.

[0111] In a variant, in addition to or instead of displaying the parameters in step S4, an incision line and / or incision pattern may be determined in step S5 based on at least the determined parameters of the skin. Such an incision line may be displayed on a display, for example in combination with a reference image of the subject. In another variant, an incision line may be projected, for example by a light beam, onto the region of interest of the subject seated on a stand in step S6 and / or cut into the region of interest of the subject by a computer-controlled incision tool.

[0112] Step S04 represents a variant in which one or a reference image of the region of interest in its unmanipulated state is provided to the control unit and can be used in the analysis to determine the parameters in step S3. Such one or more digital images may be supplied to the control unit by file transfer, email, portable memory, etc.

[0113] FIG. 5 shows a flowchart illustrating a variant of a method according to an embodiment of the present invention. Compared to FIG. 4, an additional step is introduced after the method is started. In step S01, the manipulation tool is adjusted to a position suitable for manipulating the region of interest, taking into account the subject's height, for example, while sitting on a stand. Preferably, such position adjustment is controlled by a control unit, i.e., the manipulation tool is movably mounted on a rack by an electric motor. Thus, an operator can move the manipulation tool up or down on the rack by activating a button or a control field on a computer. In devices in which the manipulation tool is pivotally mounted on a rack and the tilt of the manipulation tool can be electrically controlled to tilt, such rotational positioning of the manipulation tool can also be included in step S01.

[0114] In more advanced systems, the positioning of the manipulation tool relative to the subject, and particularly relative to the region of interest, is fully automated. This may be one or more sensors, particularly one or more sensors including a camera, for detecting the position of the subject, particularly for detecting or detecting the region of interest on the subject after assuming the inspection position. When the position of the region of interest is determined by such one or more sensors, the manipulation tool may be automatically moved and / or tilted to a position that, when activated, allows it to manipulate the region of interest. In such embodiments, the manipulation tool is automatically positioned without human interaction.

[0115] In step S02, the same positioning is performed for the camera. The camera is placed in an appropriate position to record images from the area of ​​interest. In the example of FIG. 1, the camera is positioned in a vertical position on the rack that suits a specific person, for example, at a vertical height on the rack that is adjusted to the subject's height when sitting on the stand. The operator can move the operating tool up or down the rack by activating a button or control field on the computer. In devices in which the camera is pivotally mounted on the rack and the tilt of the camera can be controlled electrically, for example by an electric motor controllable by the control unit, such positioning of the camera can also be included in step S01.

[0116] In more advanced systems, the positioning of the camera relative to the subject, particularly relative to the region of interest on the subject's face, is fully automated. This includes one or more sensors, particularly including a camera, for detecting the position of the subject, particularly for detecting or detecting the region of interest on the subject after assuming an examination position. When the location of the region of interest is determined by such one or more sensors, the camera can be automatically moved and / or tilted to a position that allows it to record the region of interest with sufficient resolution. In such embodiments, the camera is automatically positioned without human interaction.

[0117] In step S03, the camera records one or more reference images of the person's region of interest without the manipulation tool being activated, i.e., without manipulating the region of interest. Such one or more reference images may be used later in decision step S3 to compare with images captured by the camera while the manipulation tool is activated. Thus, the images captured in step S03 serve as reference images and help identify variations in the skin layout of the region of interest in response to manipulation. After capturing the images without manipulation, the method continues to step S1, i.e., activation of the manipulation tool.

[0118] 6 shows another flowchart illustrating a variant of a method according to an embodiment of the present invention. In this embodiment, step S3 of determining skin parameters of the region of interest is further detailed. In a first step S31, one or more reference images of the region of interest of the subject without manipulation are analyzed. In this step, the region of interest is preferably searched for features by pattern recognition software. Identified features may be flagged, highlighted, or otherwise annotated in the reference images as a result of computer processing.

[0119] In a second step S32, one or more images taken of the subject's region of interest during the operation are analyzed. In this step, pattern recognition software preferably searches the recorded images for the exact features identified in the reference image in the previous step S31. Again, any recognized features may be flagged, highlighted, or otherwise annotated in the recorded image as a result of computer processing. Preferably, new features detected only in the recorded image but not in the reference image may be identified, flagged, and / or highlighted. Also, features detected in the reference image but no longer detected in the recorded image may be identified, flagged, and / or highlighted.

[0120] In step S33, a comparison is made with respect to the features recaptured in the recorded images, and these features are quantified, for example, in terms of the distance of repositioning or displacement apparently caused by the manipulation tool.

[0121] In step S34, parameters such as skin elasticity within the region of interest are determined from the results of the analysis of the previous steps S34, and possibly S33 and S32, if they result in disappearing and / or appearing features. Proceed to step S4.

[0122] While presently preferred embodiments of the present invention have been shown and described, it is to be expressly understood that the invention is not limited thereto and may be variously embodied and embodied in other ways within the scope of the appended claims.

Claims

1. 1. An apparatus for determining skin parameters within a region of interest on a subject's body, comprising: a camera (1) arranged and configured to record an image of said area of ​​interest; a control unit (3) configured to determine the parameters of the skin within the region of interest by analyzing one or more images of the region of interest recorded by the camera (1) while the region of interest is being manipulated; An apparatus comprising:

2. a manipulation tool (2) arranged and configured to manipulate the skin of the subject within the region of interest; the control unit (3) is configured to determine the parameters of the skin within the region of interest by analyzing the one or more images of the region of interest recorded by the camera (1) while the region of interest is being manipulated by the manipulation tool (2).

10. The apparatus of claim 1.

3. said parameter being elasticity; the manipulation tool (2) is configured to perform one or more of the following on the skin within the region of interest: shifting, stretching, tightening, and compressing.

3. The apparatus of claim 2.

4. the manipulation tool (2) comprises a blower for generating an air jet (AJ) directed at the area of ​​interest to manipulate the skin of the subject; 4. The apparatus of claim 3.

5. the manipulation tool (2) comprises a mechanical structure for contact manipulation of the subject's skin within the region of interest; In particular, the mechanical structure is one of a pin, a tweezer, and a cotton swab.

4. The apparatus of claim 3.

6. The control unit (3) Positioning the manipulation tool (2); activating the manipulation tool (2); adjusting a predetermined power to drive the manipulation tool (2) or adjusting the manipulation tool (2) to apply a predetermined force; Switching said manipulation tool (2) between different states represented by different activation levels; configured to control the manipulation tool (2), including one or more of: The device according to any one of claims 2 to 5.

7. the control unit (3) is configured to determine the parameters by comparing the one or more images recorded by the camera (1) while the region of interest is being manipulated, in particular by a manipulation tool (2), with one or more reference images of the region of interest in which the region of interest has not been manipulated. The device according to any one of claims 1 to 4.

8. the control unit (3) is configured to identify one or more features (Fx) in the one or more reference images; the control unit is configured to identify the one or more features (Fx) in the recorded one or more images; the control unit (3) is configured to determine a distance (dx) between the one or more features (Fx) identified in the one or more reference images and the one or more features (Fx) identified in the one or more recorded images; 8. The apparatus of claim 7.

9. the control unit (3) is configured to determine the parameter according to the determined distance (dx); 9. The apparatus of claim 8.

10. the control unit (3) is configured to determine the parameters according to a predetermined power with which the manipulation tool (2) is driven or a predetermined force with which the manipulation tool (2) is adjusted to exert, In particular, the control unit (3) is configured to determine the parameter according to the distance between the nozzle of the manipulation tool (2), which is a blower, and the area of ​​interest. The device according to any one of claims 2 to 9.

11. the control unit (3) is configured to determine the parameters according to personal data of the subject, including one or more of age, sex and place of birth; An apparatus according to any one of claims 1 to 10.

12. The control unit (3) controlling the camera (1) to take one or more reference images of the area of ​​interest while the manipulation tool (2) is not activated; activating the manipulation tool (2) to manipulate the skin within the region of interest; controlling the camera (1) to record the one or more images of the area of ​​interest while the manipulation tool (2) is activated to manipulate the skin within the area of ​​interest; determining the parameters by comparing the one or more reference images with the one or more recorded images; configured to execute a computer processing program including: An apparatus according to any one of claims 2 to 11.

13. a rack (72) on which the camera (1) and / or the operation tool (2) are mounted; a chair (71) or bed for the subject to sit or lie down in an examination position facing the camera (1) and the operating tool (2); The apparatus according to any one of claims 1 to 12, comprising:

14. the manipulation tool (2) is slidably mounted on the rack (71), allowing the manipulation tool (2) to be adjusted to different regions of interest and / or subjects of different heights; In particular, said manipulation tool (2) is electrically adjustable; In particular, the control unit (3) is configured to move the manipulation tool (2) to a predetermined position. The apparatus of claim 2 in combination with claim 13.

15. The manipulation tool (2) is pivotally mounted on the rack (71) to allow tilting of the manipulation tool (2); In particular, the tilt of the manipulation tool (2) is electrically adjustable, In particular, the control unit (3) is configured to move the manipulation tool (2) to a predetermined tilted position.

15. The device of claim 2 in combination with claim 13 or claim 14.

16. The control unit (3) Positioning the camera (1); recording said one or more images; configured to control the camera (1), An apparatus according to any one of claims 1 to 15.

17. The camera (1) is a 3D camera comprising at least two 2D cameras for recording 2D images and a processor for calculating a 3D image from the two recorded 2D images, An apparatus according to any one of claims 1 to 16.

18. the camera (1) is slidably mounted on the rack (71) to allow the camera (1) to be adjusted to different areas of interest and / or subjects of different heights; In particular, the position of said camera (1) is electrically adjustable; In particular, the control unit (3) is configured to move the camera (1) to a predetermined position, 14. The apparatus of claim 13.

19. an output unit for outputting the determined parameters; Equipped with In particular, said output unit is a display (4) for showing said determined parameters, In particular, the control unit (3) is configured to display the recorded image(s) and / or the reference image(s) on the display (4), In particular, the control unit (3) is configured to display the parameters on the display (4) in response to a determination of the control unit; In particular, the control unit (3) is configured to display the coded parameter color on the display (4), in particular to display the color-coded parameter in the region of interest in the displayed recorded image(s) and / or the reference image(s). An apparatus according to any one of claims 1 to 18.

20. the control unit (3) is configured to generate, based on the determined parameters of the skin, an incision pattern (Cx) for the region of interest, either from scratch or by adapting an existing incision pattern in preparation for a surgical intervention along the incision pattern (Cx); An apparatus according to any one of claims 1 to 19.

21. an illumination tool configured to project and / or write the incision pattern (Cx) on the body of the subject within the region of interest; 21. The apparatus of claim 20, comprising:

22. an incision tool, in particular a laser, configured to incise the incision pattern (Cx) into the skin of the subject within the region of interest; 22. The apparatus of claim 20 or claim 21, comprising:

23. the control unit is configured to computer-control the incision of the incision pattern (Cx) on the skin of the subject within the region of interest.

23. The apparatus of claim 22.

24. the control unit (3) is configured to display the incision pattern (Cx) in the region of interest of the displayed recorded image(s) and / or the one or more reference images. An apparatus according to any one of claims 20 to 23.

25. the region of interest includes an eyelid of the subject's face; An apparatus according to any one of claims 1 to 24.

26. The incision pattern (Cx) is designated to perform eyelid lifting. Apparatus according to claim 25 in combination with any one of claims 20 to 24.

27. Use of a device according to any one of claims 1 to 26 for preparing a surgical intervention in a region of interest of a subject's body.

28. 1. A method for determining skin parameters within a region of interest on a subject's body, comprising: Manipulating the subject's skin within the region of interest (S1); recording (S2) one or more images of the region of interest while the region of interest is being manipulated; determining (S3) by a computer the parameters of the skin within the region of interest by analyzing the recorded image or images; A method comprising:

29. 29. The method of claim 28, wherein the parameter is elasticity.

30. 30. The method of claim 28 or claim 29, wherein the region of interest comprises an eyelid.

31. manipulating the skin of the subject within the region of interest with a manipulation tool (2) positioned and configured to manipulate the skin of the subject within the region of interest, thereby performing one or more of shifting, stretching, tightening, and compressing the skin within the region of interest; Recording (S2) said one or more images by means of a camera (1); The method according to any one of claims 28 to 30, comprising:

32. Positioning the manipulation tool (2) by a computer (S01), and / or Activating the operating tool (2) by a computer (S1); 32. The method of claim 31 , comprising:

33. Positioning the camera (1) by a computer (S01), and / or recording said one or more images by a computer (S2); The method of any one of claims 28 to 32, comprising:

34. providing one or more reference images of the region of interest taken without manipulating the subject's skin within the region of interest (S03); determining the parameters by comparing the one or more reference images with the one or more recorded images; The method of any one of claims 28 to 33, comprising:

35. The determining step (S3) Identifying one or more features in the one or more reference images (S31); identifying the one or more features in the recorded one or more images; Determining (S33) a distance (dx) between the one or more features identified in the one or more reference images and the one or more features identified in the recorded one or more images; determining (S34) the parameters according to the determined distance (dx); 35. The method of claim 34, comprising:

36. a predetermined power at which the manipulation tool (2) is driven or a predetermined force at which the manipulation tool (2) functions; the distance between the nozzle of the manipulation tool (2), which is a blower, and the area of ​​interest; personal data including one or more of the subject's age, sex and place of birth; 32. The method of claim 31 , further comprising determining (S3) the parameters according to one or more of:

37. a computerized step (S03) of controlling the camera (1) to take one or more reference images of the area of ​​interest while the manipulation tool (2) is not activated; a computerized step of activating the manipulation tool (2) to manipulate the skin within the region of interest; a computerized step (S2) of controlling the camera (1) to record the one or more images of the area of ​​interest while the manipulation tool (2) is activated; a computerized step (S3) of determining said parameters by comparing said one or more reference images with said recorded one or more images; 32. The method of claim 31 , comprising:

38. Displaying the determined parameters (S4); In particular, displaying said one or more of said recorded images and / or said reference images and color-coding said region of interest therein; The method of any one of claims 28 to 37, comprising:

39. determining (S5) by a computer an incision pattern (Cx) for the area of ​​interest based on the determined parameters of the skin, either from scratch or by adapting an existing incision pattern in preparation for a surgical intervention according to said incision pattern (Cx), The method according to any one of claims 28 to 38, in particular, the incision pattern is specified to perform an eyelid lifting when the area of ​​interest is an eyelid.

40. 40. The method of claim 39, comprising projecting and / or writing (S6) the incision pattern (Cx) onto the area of ​​interest of the subject by means of an illumination tool.

41. 41. The method of claim 39 or claim 40, comprising incising the incision pattern (Cx) into the skin of the subject in the region of interest by a computer controlled incision tool, preferably by a computer controlled laser.

42. 41. The method according to any one of claims 39 to 40, comprising displaying the one or more of the recorded images and / or the reference image, and displaying the incision pattern (Cx) in the region of interest of the displayed one or more images.

43. submitting a pre-operative image of the region of interest and a proposed incision pattern to an artificial neural network (100); receiving a post-operative image of the region of interest from the artificial neural network (100) in response to the submission; The method of any one of claims 28 to 42, comprising:

44. A computer program element comprising computer program code means adapted to carry out the method of any one of claims 28 to 43 when executed on a control unit (3).

Images