Computer vision detection of skin irritation

Abstract

One embodiment provides a technique for applying computer vision technology to medical images to enable the identification of skin irritations caused by the wearing of medical devices. In certain embodiments, a subset of medical image data is identified using, for example, template matching technology or other trained computer vision models to facilitate timely and accurate identification of skin irritations, even in the initial stages. One embodiment provides an application program that assists a user in acquiring and evaluating medical images, which includes providing instructions to the user regarding any identified skin irritations and enabling improvements.

Description

Technical Field

[0001] The disclosed subject matter generally relates to the application of computer vision to images for object identification and classification tasks. A particular disclosed subject matter relates to the use of computer vision to address the problem of accurately detecting early-stage skin irritation associated with the wearing of medical devices.

Background Art

[0002] Skin irritation is a significant reason for people to stop wearing medical devices that are adhered to the skin using adhesive patches or stickers. For example, a Mobile Cardiac Outpatient Telemetry ("MCOT") patch system (referred to as "MCOT" or "MCOT patch") is a monitor that adheres to a user's skin in the chest area by means of a patch having an adhesive. The MCOT patch collects electrocardiogram (ECG) data from sensors and then automatically transmits that ECG data to another device via a wireless connection such as Bluetooth (registered trademark). The MCOT patch is configured to be worn for a long period of time and automatically transmits cardiac or ECG data 24 hours a day. By doing so throughout the monitoring period, the MCOT patch provides complete cardiac monitoring information to medical professionals.

[0003] To prevent the skin condition from deteriorating and minimize patient discomfort, it is important to be able to detect skin irritation associated with medical devices such as MCOT patches as early as possible. If skin irritation can be identified in a timely manner, a simple step of placing the medical device at a different location on the skin, such as a different location on the torso, can be taken.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Identifying skin irritation caused by medical device use in its early stages is difficult. Relying on manual or human review of medical images to classify skin irritation associated with medical device use is problematic because it requires skilled reviewers and standardized image quality. In many cases, manual review is only possible if there is significant or high degree of skin irritation represented in high-quality photographs; otherwise, a patient visit is required, hindering remote or automated analysis.

[0005] While conventional image processing techniques have been applied to the examination of skin in various conditions, these approaches are not adapted to address the assessment of skin irritation, particularly in its initial stages and in relation to the use of medical devices. For example, when images are captured under various conditions and of varying quality by unskilled users or patients using non-uniform hardware systems such as the various cameras used in smartphones, the need to refine computer vision applications to address the assessment of skin irritation related to the use of medical devices becomes significant. [Means for solving the problem]

[0006] Accordingly, one embodiment provides a technique for applying computer vision technology to medical images, enabling the identification of skin irritation caused by the wearing of medical devices, even in the early stages when the signal of skin irritation in the image is not strong, and / or when noisy images of varying qualities are available. In certain embodiments, for example, a subset of medical image data is identified using template matching technology or other trained computer vision models, facilitating the timely and accurate identification of skin irritation, even in the early stages. One embodiment provides an application program that assists a user in acquiring and evaluating medical images, the application program including providing instructions to the user regarding any identified skin irritation, enabling remediation.

[0007] In summary, one embodiment provides a method for acquiring a medical image containing geometric shapes related to a medical device. The method includes using one or more sets of processors to analyze the medical image to identify geometric shapes and to identify a subset of image data of the medical image related to these geometric shapes. The method includes using one or more sets of processors to determine that the subset of image data indicates skin irritation and to provide an index of skin irritation.

[0008] In one embodiment, the method includes providing a command to capture a medical image, then indicating geometric shapes within the medical image, and obtaining confirmation that the geometric shapes have been identified.

[0009] In one embodiment, analyzing a medical image involves performing template matching to identify the geometric shape, where the geometric shape has a predetermined patch shape.

[0010] In one embodiment, analyzing medical images involves using a neural network to identify geometric shapes and to identify one or more subsets of image data associated with those geometric shapes. In one embodiment, the trained model can perform the analysis and identification in a single classification step.

[0011] In one embodiment, a subset of image data related to a geometric shape includes one or more pixel data that is located at a predetermined distance from the geometric shape and pixel data that is located within the geometric shape.

[0012] In one embodiment, determining a subset of image data involves comparing one or more pixel values ​​with a set of thresholds indicating skin irritation. In one embodiment, determining that a subset of image data indicates skin irritation involves directly classifying a medical image as containing a predetermined shape and concluding that the associated subset of image data indicates skin irritation.

[0013] In one embodiment, the method includes selecting an instruction based on comparing one or more pixel values ​​of a subset of the image data with a set of thresholds. In one embodiment, the instruction is selected and provided based on a threshold from the set of thresholds, and includes an instruction using an index of skin irritation. In one embodiment, the instruction includes one or more audio and visual data indicating that a medical device should be repositioned.

[0014] In one embodiment, acquiring a medical image includes acquiring a first medical image before removing the medical device, and then acquiring a medical image after removing the medical device, wherein the first medical image facilitates the identification of geometric shapes in the medical image.

[0015] In one embodiment, acquisition, analysis, identification, determination, and provision are performed locally on the client device.

[0016] One embodiment includes a computer program product having a non-temporary computer-readable medium having code executable by one or more processors. In one embodiment, the code includes code for acquiring a medical image having a geometric shape related to a medical device; code for analyzing the medical image to identify the geometric shape; code for identifying a subset of image data of the medical image related to the geometric shape; code for determining that the subset of image data indicates skin irritation; and code for providing an index of skin irritation.

[0017] One embodiment includes, for example, a user client device or a server that provides a downloadable or executable image analysis program. In one embodiment, the device includes a set of one or more processors and a non-temporary computer-readable medium having code executable by the set of one or more processors. In one embodiment, the code includes code for acquiring a medical image having a geometric shape related to a medical device, code for analyzing the medical image to identify the geometric shape, code for identifying a subset of image data of the medical image related to the geometric shape, code for determining that the subset of image data indicates skin irritation, and code for providing an index of skin irritation.

[0018] As will become apparent upon reviewing this specification, methods, devices, systems, and products for carrying out various embodiments are provided.

[0019] Since the foregoing is a summary, it may include simplified, generalized, and omitted details, and therefore, those skilled in the art will understand that this summary is merely illustrative and not intended to be limiting in any sense.

[0020] These and other features and characteristics of the exemplary embodiments, as well as the operation and function of the relevant elements of the components and their combinations, will become more apparent when considering the following description and the appended claims with reference to the appended drawings, all of which form part of this specification. However, it should be clearly understood that the drawings are for illustrative and explanatory purposes only and are not intended to define the boundaries of the invention. [Brief explanation of the drawing]

[0021] [Figure 1] Figure 1 shows an exemplary method according to one embodiment. [Figure 2]Figure 2 illustrates an example of a geometric shape associated with a medical device and showing skin irritation in a medical image according to one embodiment. [Figure 3A] Figure 3A shows an exemplary medical image having a geometric shape associated with a medical device according to one embodiment. [Figure 3B] Figure 3B shows an exemplary medical image associated with a medical device and having a geometric shape showing skin irritation according to one embodiment. [Figure 4A] Figure 4A shows an exemplary medical image having a geometric shape associated with a medical device according to one embodiment. [Figure 4B] Figure 4B shows an exemplary medical image associated with a medical device and having a geometric shape showing skin irritation according to one embodiment. [Figure 5] Figure 5 shows a diagram of exemplary system components according to one embodiment. **DETAILED DESCRIPTION OF THE INVENTION**

[0022] In the specification, unless specifically stated clearly in context, even if not stated as plural, it includes the case where there are plural. As used in this specification, the expression that two or more parts or components are "coupled" means that these parts are directly or indirectly coupled or operate together, for example, through one or more intermediate parts or components, as long as a link occurs. As used in this specification, "operatively coupled" means that two or more elements are coupled to operate together or communicate with each other in one direction or both directions. In the specification, "number" means one or two (i.e., plural) or more integers. As used in this specification, "set" means one or more.

[0023] Furthermore, the features, structures, or properties described can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to enhance the understanding of the embodiments. However, those skilled in the art will recognize that various embodiments can be carried out without using one or more of the aforementioned specific details, or using other methods, components, materials, etc. In other examples, well-known structures, materials, or operations are not shown or described in detail to avoid ambiguity.

[0024] Medical devices (such as patches, monitors, and sensors) are attached to a patient's skin by adhesives, such as patches, stickers, or tapes. In certain situations, such as when a medical device is worn for an extended period (e.g., several hours or several days), certain patients often experience skin irritation from wearing this medical device.

[0025] When skin irritation is detected in a timely manner, it can be easily addressed, for example, by repositioning the medical device. Unfortunately, this is often not possible. For many patients, it is not easy to detect the skin irritation itself, especially in the early stages when the adverse effects of wearing the medical device can be easily avoided by repositioning it to a different area of ​​skin. The user may feel a slight itch or sensation on the skin, but nevertheless, the user will examine it in a mirror and try to determine whether any redness is due to the adhesive used to secure the medical device to the skin. If it is due to the adhesive, the user must also determine whether the irritation is of a nature that would warrant moving the medical device.

[0026] While conventional computer vision technologies have been applied to problematic spaces for examining skin in various conditions, they do not readily address the determination of whether any redness on a patient's skin can be classified into a target category, particularly related to the use of medical devices. What is needed is improved computer vision technology that enables the identification of skin irritation caused by or related to the use of medical devices. This would allow users to identify potentially problematic skin irritation earlier, at the initial onset of the condition, and without the need for frequent tracking by medical professionals for manual photographic review and instructions.

[0027] Since skin irritation is a significant reason why people stop wearing medical devices such as MCOT patches, one embodiment provides computer vision technology that enables the detection of skin irritation as early as possible. This allows for accurate and timely identification of skin irritation associated with wearing medical devices and prevents further patient frustration associated with unnecessarily asking users to reposition the medical device to a different location on their skin.

[0028] In one embodiment, for example, a camera on a smartphone or another user device is used to capture medical images (which may be still images, a series of still images, or a video), and a computer vision program is used to detect the early onset of an irritation. In another embodiment, known geometric shapes of adhesive or patch portions of the medical device are used to help computer vision processing search for irritations in a subset of medical images and identify patterns for identifying skin irritations.

[0029] This description refers to the drawings. The exemplary embodiments shown will be best understood by referring to the drawings. The following description is intended as an example only and merely illustrates specific exemplary embodiments.

[0030] Figure 1 shows an exemplary method according to one embodiment. As shown, the method includes the step of acquiring a medical image having geometric shapes related to a medical device in 102. For example, the geometric shapes include one or more predetermined or expected shapes related to an adhesive area or portion of a medical device, such as an MCOT patch. In one embodiment, the medical image is acquired as part of an application flow or a sequence of instructions. For example, optionally, the user is instructed to acquire a medical image including the medical device and the adhesive area or portion attached to the skin, as shown in 101. As part of the application or sequence of instructions, one embodiment may provide feedback to the user as part of the step of supplying an instruction to acquire a medical image in 101. For example, the user's device may display visual or other indicators, such as a bounding box, that identify the expected geometric shapes. Such feedback provided is based on a model described herein, which is used to identify the geometric shapes as part of an object detection routine. Once the user has seen and can confirm such feedback, the image may be acquired and / or acquired for use in 102.

[0031] The method, in 103, includes analyzing a medical image using a model for identifying geometric shapes, such as the adhesive portion of a medical device attached to the user's skin. In one example, the analysis in 103 includes identifying a predetermined shape, such as the geometric shape or contour of the adhesive portion of the medical device. In an embodiment that shows the assumed identified geometric shape in the medical image and confirms that the geometric shape has been identified, and then requests confirmation that the geometric shape has been correctly identified, the process may proceed in 104 to analyze a subset of the image data for identification. If user confirmation has not been received or is negative, one embodiment may instruct the user to take another medical image, provide feedback on image settings or conditions that prevent the identification of the geometric shape, etc.

[0032] The analysis of 103 involves performing template matching and identifying geometric shapes, for example, using a template matching process available from OpenCV. Thus, as will be understood by those skilled in the art, one embodiment performs object recognition in 103 by utilizing knowledge of shapes, such as a predetermined geometric shape of an MCOT patch. The template matching process focuses on detecting skin redness and / or skin irritation of a patch-related shape, such as the shape of the patch or a halo shape surrounding the patch. This allows the computer vision program to detect skin irritation earlier, where objects having a subset of medical images become the focus of the classification task.

[0033] In one embodiment, if a patient is instructed to take a photograph (or short video clip) before removing a patch, this helps a computer vision program to use the first image to easily identify an object (e.g., a medical device or a part of a medical device such as an adhesive portion), to know where the patch was located on the skin, and then to identify geometric shapes in subsequent images or video frames after the medical device has been removed. That is, in a session or sequence of an application or program, a user instruction provided in 101 may include a request to capture a medical image in which the device is worn, and then a subsequent medical image after the medical device has been removed. Such an instruction may include timing information, for example, instructing the user to wait for a period of time between image captures, for example, a few minutes. After the medical device is removed, the system can focus on a sub-area of ​​the subsequent medical image and more easily detect whether that sub-area is redder or qualitatively different from the surrounding skin area, for example by comparing image pixel values ​​such as color content with a threshold, by making a relative comparison (such as whether it is inside or outside the boundary area in the image, like a subset of pixels related to the adhesive portion of the medical device), or by a combination of the above.

[0034] In one embodiment, at 103, a trained model, such as a trained deep learning network, can be used to detect image features, for example, object detection in the form of one or more colored patches of a specific shape contained within a medical image. A neural network suitable for this type of segmentation task is a so-called U-NET, which is characterized by a wide layer at the beginning and end, and an information bottleneck in the middle. Using a noisy training set, for example, such that the geometric shape of interest is placed against a noisy background, increases the sensitivity and accuracy of the neural network for detecting a specific type of shape as an object of interest, even when the signal is very weak, as is the case during the initial stages of skin irritation. In this regard, in one embodiment, the determination at 103 that the medical image contains a geometric shape, the identification of a subset of image data associated with the geometric shape at 104, and the determination at 105 that the subset of image data associated with the geometric shape indicates skin irritation is a single classification task performed by a model trained to distinguish between normal or acceptable skin and skin irritation formed with a predetermined pattern of geometric shapes associated with a medical device.

[0035] In one embodiment, the analysis and identification of a geometric shape in 103 facilitates the identification of a subset of image data of a medical image associated with the geometric shape in 104. As further described herein, this may include identifying pixels of a medical image associated with the contour of a geometric shape. In one example, identification in 104 marks a set of pixels that lie within the contour or boundary defined by the geometric shape, as described in relation to Figure 3B. In another example, identification in 104 marks a set of pixels that lie around the contour of a geometric shape, for example, within the outer edge of the threshold distance of this geometric shape, as described in relation to Figure 4B. In one embodiment, the identification or marking of a subset of data associated with a geometric shape can be used to provide feedback to the user, for example, by highlighting an area in a medical image that has been identified as appropriate for analysis regarding skin irritation associated with wearing a medical device.

[0036] The method described above may include the step in 105 of determining that a subset of image data indicates a skin irritation. As described herein, it is possible to utilize a model trained to identify geometric shapes themselves and use the above detection as a surrogate for identifying skin irritations. In such an example, one or more models are trained to identify and classify geometric shapes, for example, distinguishing classes of geometric shapes based on redness or color data.

[0037] The embodiments may also, or alternatively, further process a subset of image data related to geometric shapes to identify and / or refine the type or characteristics of skin irritation. For example, pixels identified as related to geometric shapes as a result of processing in steps 103 and / or 104 may be further analyzed in 105 to identify the state or classification of skin irritation based on a subset of image data, for example, by additional comparison with one or more thresholds. In one example, the determination in 105 may include comparing a subset of image data with one or more thresholds to facilitate the identification of a skin irritation of a particular type, level, or nature. In one embodiment, the one or more thresholds may be based on image data, e.g., the color value of a pixel, its relative difference to other parts of a medical image, or similar.

[0038] As shown in Figure 1, the method includes, in 106, the step of providing an index of the skin irritation after the skin irritation has been identified. For example, one embodiment may provide output via a smartphone application or similar program to indicate that a skin irritation has been identified. In one example, one embodiment may use the one or more thresholds to associate the skin irritation with a level, score or similar which is suitable for selecting feedback or advice for the user, including, for example, an instruction or similar to reposition a medical device to a designated area.

[0039] As shown in Figure 2, the medical image 200 includes one or more geometric shapes 201a, 201b related to the adhesive portion (e.g., tape, patch, etc.) of a medical device worn on the user's skin. In one embodiment, computer vision processing can be improved if the locations of the geometric shapes 201a, 201b are known, for example by performing one or more preprocessing steps to isolate a subset of the image data. As described herein, this is achieved by acquiring a first medical image before removing the medical device and comparing this first medical image with a medical image 200 when the medical device is removed. As described herein, the processing also includes providing feedback on the assumed locations of the geometric shapes 201a, 201b in the medical image 200 and receiving confirmation that the geometric shapes 201a, 201b have been properly identified. Visual markers in the medical image 200 can be used to confirm whether there are any detected skin irritations at the location where the medical device was placed. Furthermore, preprocessing for identifying subsets of medical images is achieved by applying one model to identify the subset, and then applying a second trained model or rule-based processing to the subset for stimulus detection or classification.

[0040] In one embodiment, referring to Figures 3A-B, skin irritation is detected by computer vision while the medical devices 303a, 303b are still attached to the body. For example, if the adhesive portion (e.g., tape, patch, etc.) 301a, 301b of the medical device is transparent, the visual change to the skin is often detected through the adhesive portion 301a, 301b. Over time, by comparing medical images 300a, 300b of the patch on the body taken over, for example, 10 days, the system detects differences in skin irritation in areas of skin 302a, 302b related to the geometric shape. In the example of Figure 3A, medical image 300a shows the medical device 303a (e.g., MCOT patch) along with dashed lines indicating the edges or periphery of the geometric shape defined by the transparent adhesive portion 302a used to attach the medical device 303a to the skin. In Figure 3B, medical image 300b schematically shows skin irritation in area 302b beneath the transparent adhesive portion 301b (shown with different hatching compared to 302a in the example in Figure 3B), which is detected by computer vision while the medical device 303b is still being worn by the user.

[0041] As shown in Figures 4A-B, in the case of opaque adhesives or tapes 401a, 401b, computer vision can be used to detect areas 404b of skin irritation in medical images 400a, 400b over time while the medical devices 403a, 403b are still attached to the body. In other words, skin irritation occurs not only in the area covered by the geometric shape of the adhesives 401a, 401b, i.e., in the area beneath the adhesives 401a, 401b, but also when it spreads to the skin adjacent to the adhesives 401a, 401b, as shown in area 404b, for example, this pattern, such as a halo effect, is detected by acquiring medical images 400a, 400b including the medical devices 403a, 403b attached to the body, and having computer vision processing compare, for example, the medical image 400a of medical device 403a acquired on different days, e.g., the medical image 400b of medical device 403b acquired on day 10, identifying the geometric shape that is the area of ​​the halo effect in the medical image 400b, and making a determination regarding the skin irritation, such as its presence, type, level, and recommended improvement. Furthermore, when time-series data is available, it can be used to enhance one or more indicators or instructions, such as those provided in Figure 1, 106, which indicate, for example, the rate of increase in stimulation or the trend of predicted skin irritation.

[0042] In one embodiment, one or more indicators or commands, provided, for example in Figure 106, may include predictions such as estimating the remaining time until skin irritation reaches a level requiring a replacement action by the patient, such as moving the patch to a new location. For example, medical image 400a (taken, e.g., on day 1) may show irritation level 0, while medical image 400b (taken, e.g., on day 10) may show irritation level 1. Thus, one embodiment can, for example, estimate a future moment, day, or time period in which skin irritation will reach a level requiring a replacement action, based on trend data of this patient and / or patient population, and provide timely warnings and / or further commands.

[0043] In one embodiment, one or more indicators or instructions, provided, for example in 106 of Figure 1, include a proposed placement area for a patch or medical device based on the detected area or sub-area of ​​skin irritation. For example, in the case of skin irritation, the action to be taken is to replace or move the patch. However, in the case of an ECG patch, the patch cannot be placed anywhere on unirritated skin, as it must still be in a position where an ECG signal can be detected. Therefore, ECG patches should generally not be moved over long distances. In situations where skin irritation is not uniform across the entire surface of the patch, such as detected using computer vision provided in one embodiment, the user is instructed to avoid areas of more pronounced irritation compared to other areas covered by the patch's adhesive. Therefore, the instructions provided in 106 may include advice to replace the patch, particularly to avoid sub-areas showing increased irritation in medical images. As described herein, one embodiment may provide such instructions in various forms, including real-time augmented reality feedback, such as superimposing an icon or representation of a patch onto a new location on the user's skin, on a live video of the user captured using a patient's local device, such as a smartphone or similar.

[0044] In the example shown in Figures 4A-B, medical image 400a is acquired on the first day, for example, by an application running on the user's smartphone. This image is analyzed to identify a predetermined geometric shape associated with the medical device 403a, in this case the adhesive 401a in the image. Medical image 400a can be classified as not showing skin irritation. Subsequently, another medical image 400b is acquired by the user, for example, according to the application's flow or program schedule. Medical image 400b is also analyzed to identify one or more of the predetermined shapes, in this example the adhesive 401b associated with the medical device 403b, or the region of the halo effect shown in region 404b. Region 404b in medical image 400b is identified as a subset of the image data, i.e., a skin region within a predetermined number, such as the number of pixels, surrounding the first geometric shape, or as a skin region within the second geometric shape, i.e., the region 404b of the halo effect itself. Next, using region 404b, it is determined, for example, whether skin irritation is detected around adhesive 401b, through direct indicators based on the model's ability to detect region 404b as an object within medical image 400b (compared to medical image 400a), and / or through submission of pixel data of region 404b to further analysis, such as comparison with one or more thresholds, further model classification or categorization, and / or association with instructions for feedback to the user.

[0045] Therefore, one embodiment can better detect skin irritations associated with a specific geometric shape, for example, a known shape of an MCOT patch, or an area of ​​halo effect associated with that patch. One embodiment provides a mechanism for isolating a subset of image data for skin irritation analysis, for example, this subset of image data includes one or more pixel data within a predetermined distance from the geometric shape and one or more pixel data within the geometric shape.

[0046] In one embodiment, determining that a subset of image data indicates skin irritation involves comparing one or more pixel values ​​to a set of thresholds indicating skin irritation and providing feedback to the user. For example, a data structure such as a table can store information on pixel values, such as the average or total color value, or a range thereof, related to instructions or program routines, such as instructions to display an index of skin irritation or requests to move a medical device. In one embodiment, instructions are selected based on comparing one or more pixel values ​​to the set of thresholds. For example, the total pixel value of a region, such as region 404b, may fall within a numerical range indicating initial skin irritation caused by adhesion to the skin. This numerical range may be stored in a table or stored in logical association with instructions such as instructions indicating that the skin irritation should be repositioned or removed for further imaging and analysis. In one example, the instructions may include one or more audio and visual data, output, for example, via a mobile application, indicating that the medical device should be repositioned. The indicator may further provide, for example, an output of a visual or graphical representation indicating the new position of a medical device, as part of an augmented reality program that places the proposed position of the medical device onto the user's image.

[0047] One embodiment can be implemented on a variety of devices, including, for example, a user device such as a smartphone or tablet running a mobile application. In one embodiment, referring back to Figure 1, steps 102 (acquisition), 103 (analysis), 104 (identification), 105 (determination), and 106 (providing) are performed locally on a client device, such as a smartphone running a mobile application. In such embodiments, a trained model, such as a template matching model, a deep neural network, or the like, may be downloaded and run locally on the user device. For example, an application program as described herein may include an image analysis model and one or more APIs for calling the machine learning (ML) subsystem of the user device, such as calling the Core ML image classification and / or object detection API (Application Programming Interface) on an Apple iPhone. In some embodiments, the model version is selected based on identification requirements, for example, based on the model or type of medical device, adhesive and / or skin type (e.g., downloaded, activated to run on the image, etc.). For example, a model trained to identify one or more predetermined shapes is selected, for example, by the user or as part of an application routine, based on knowledge of what type of device the user is wearing. Thus, one or more models can be used in different situations; for example, two or more models can be used, and each model can be used to identify each of two shapes of interest, for example, the medical device and / or each of the adhesive parts of a medical device having a different geometric shape, depending on which model of medical device the user is wearing.

[0048] Therefore, one embodiment may include an application program configured to execute computer program instructions, such as outlined in Figure 1, which, in combination with the hardware of the local user device, enables the identification of geometric shapes of interest and the classification of medical image data, such as a subset of medical image data related to the geometric shapes, to detect skin irritations and provide the user with appropriate instructions and feedback. Such an embodiment allows the user's confidential medical images to remain solely on the user device, increasing the user's confidence when applying any image analysis or computer vision techniques to the user's medical images.

[0049] Referring to Figure 5, it will be readily apparent that a particular embodiment may be implemented using any of a wide variety of devices or combinations of devices and components. Figure 5 shows an example of a computer 500 and its components, which is used in a device to perform functions or operations described herein, for example, to perform medical image analysis for identifying skin irritation. In addition, circuits other than those shown in Figure 5 may be used in one or more embodiments. The example in Figure 5 includes a particular functional block, as shown, which can be integrated on a single semiconductor chip to meet specific application requirements.

[0050] One or more processing units are provided, which include a central processing unit (CPU) 510, one or more graphics processing units (GPUs), and / or microprocessing units (MPUs), which include arithmetic logic units (ALUs) for performing arithmetic and logical operations, instruction decoders for decoding instructions and providing information to timing and control units, and registers for temporary data storage. The CPU 510 may have a single integrated circuit with several units, and its design and device will vary according to the selected architecture. As described herein, certain functional modules, such as machine learning (ML) hardware or chips, may be included in a device such as computer 500 to facilitate certain functions, such as image analysis, object detection, and object identification.

[0051] Computer 500 also includes a memory controller 540 having a direct memory access (DMA) controller for transferring data between memory 550 and hardware peripherals. The memory controller 540 includes a memory management unit (MMU) that functions to handle cache control, memory protection, and virtual memory. Computer 500 supports various communication protocols (e.g., I 2 It may include a controller for communication using USB (C, USB, etc.).

[0052] The memory 550 may include various types of memory, volatile and non-volatile, such as read-only memory (ROM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), flash memory, and cache memory. The memory 550 may include embedded programs, code, and downloaded software, such as a medical image acquisition and analysis program 550a that provides coded methods as illustrated and described in relation to Figure 1. As described herein, the program 550a may include labeled training images (e.g., positive or negative examples of geometric shapes, training sets providing geometric shapes of interest against noisy background image data), or a trained neural network trained with descriptive metadata useful for identifying skin irritations based on classification of skin irritations, as described herein. As an example, but not limited to, the memory 550 may also include operating systems, application programs, other program modules, code, and program data that are downloaded, updated, or modified via a remote device.

[0053] The system bus enables communication between various components of the computer 500. I / O interfaces 530 and radio frequency (RF) devices 520, such as Wi-Fi and communication radios, are included to enable the computer 500 to transmit data to and receive data from remote devices using wireless mechanisms; it should be noted that data exchange interfaces are also used for wired data exchange. The computer 500 can operate in a networked or distributed environment using logical connections to one or more other remote computers or databases 570, such as a database storing trained models for analyzing medical images, a database storing downloadable applications or programs for said analysis, etc. Such logical connections may include networks such as a local area network (LAN) or wide area network (WAN), but may also include other networks / buses. For example, the computer 500 can exchange data with and between peripheral devices 560, such as a camera for capturing medical image data, and the peripheral devices may be integrated into the same housing or unit as the computer 500.

[0054] Therefore, the computer 500 can execute program instructions or code configured to acquire, store and analyze medical image data, and to perform other functions of the embodiment described in relation to Figure 1. A user can connect to the computer 500 (e.g., by entering commands and information) via an input device connected to the I / O interface 530. A display 580 or other type of output device is connected to or incorporated into the computer 500, for example, via an interface selected from the I / O interface 530.

[0055] It should be noted that the various functions described herein are implemented using instructions or code stored in memory, for example, memory 550, which are transmitted to and executed by a processor, for example, CPU 510. The computer 500 includes one or more storage devices for permanently storing programs and other data. When used herein, a storage device is a non-temporary computer-readable storage medium. Some examples of non-temporary storage devices or computer-readable storage media, but not limited to these, include, for example, memory 550, storage devices integrated into the computer 500 such as a hard disk or solid-state drive, and removable storage devices such as an optical disc or memory stick.

[0056] Program code stored in memory or storage devices is transmitted using any suitable transmission medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination of the above. Program code for performing operations according to various embodiments is written in any combination of one or more programming languages. Program code can be executed as a standalone software package entirely on a single device, partially on a single device, partially on a single device and partially on other devices, or entirely on other devices. In one embodiment, program code is stored in a non-temporary medium and executed by a processor to perform the functions or operations specified herein. In some cases, the devices referenced herein are connected via any type of connection or network, including a local area network (LAN) or a wide area network (WAN), i.e., the connection is made via other devices (e.g., via the Internet using an Internet service provider), via a wireless connection, or via a hardwired connection such as a USB connection.

[0057] In a claim, no reference symbol placed between parentheses should be construed as limiting the claim. The words “having” or “including” do not preclude the existence of elements or steps other than those listed in the claim. In a device claim listing several means, some of these means may be embodied by the same item of hardware. The absence of a statement that an element is multiple does not preclude that an element is multiple. In any device claim listing several means, some of these means may be embodied by one of the same item of hardware. The mere fact that several elements are listed in different dependent claims does not indicate that these elements cannot be used in combination. The “about” or similar relative meaning applied to numbers includes, for example, the usual (conventional) rounding of numbers with a fixed base such as 5 or 10.

[0058] Even if the present invention is described in detail for illustrative purposes based on what is currently considered to be the most practical and preferred embodiment, it should be understood that such details are for illustrative purposes only, and that the present invention is not limited to the disclosed embodiments, but rather intended to include modifications and equivalent configurations within the spirit and scope of the appended claims. For example, it should be understood that the present invention also takes into consideration, wherever possible, that one or more features of any embodiment may be combined with one or more features of any other embodiment.

Claims

1. To acquire medical images with geometric shapes related to medical devices, Using one or more sets of processors, the medical image is analyzed to identify the geometric shape, Using the set of one or more processors, identify a subset of image data of the medical image related to the geometric shape, Using the set of one or more processors, it is determined that a subset of the image data indicates skin irritation. Using the aforementioned set of one or more processors, an indicator of skin irritation is provided. A method of having.

2. To provide commands for capturing the aforementioned medical images, Subsequently, the geometric shape within the medical image is shown, To obtain confirmation that the aforementioned geometric shape has been identified. The method according to claim 1, comprising:

3. The method according to claim 1, wherein the analysis comprises performing template matching to identify the geometric shape.

4. The method according to claim 3, wherein the geometric shape has a predetermined patch shape.

5. The method according to claim 1, wherein one or more of the analysis and identification involves using a neural network to identify one or more of the geometric shapes and subsets of the image data associated with the geometric shapes.

6. The method according to claim 1, wherein the subset of the image data relating to the geometric shape comprises one or more of the pixel data located within a predetermined distance from the geometric shape and the pixel data located within the geometric shape.

7. The method according to claim 1, wherein the determination comprises comparing one or more pixel values ​​with a set of threshold values ​​indicating skin irritation.

8. The method according to claim 7, comprising selecting an instruction based on comparing one or more pixel values ​​with the set of thresholds.

9. The method according to claim 8, wherein the instruction is selected based on a threshold from the set of thresholds.

10. The method according to claim 9, wherein the provision includes the instruction using an indicator of skin irritation.

11. The method according to claim 10, wherein the instruction includes one or more audio data and visual data indicating that the medical device should be repositioned.

12. To obtain the above means, Before removing the medical device, a first medical image is obtained, Subsequently, the medical image is acquired after the medical device is removed. The method according to claim 1, wherein the first medical image facilitates the identification of the geometric shape in the medical image.

13. The method according to claim 1, wherein the acquisition, analysis, identification, determination, and provision are performed locally on the client device.

14. A computer program product having a non-temporary computer-readable medium having code executable by one or more sets of processors, wherein the code is Code to acquire medical images with geometric shapes related to medical devices, The aforementioned medical image is analyzed to obtain a code that identifies the geometric shape, A code that identifies a subset of the image data of the medical image related to the geometric shape, The subset of the aforementioned image data includes a code that indicates skin irritation, A code that provides an indicator of skin irritation and A computer program product that has [certain characteristics].

15. A set of one or more processors, A non-temporary computer-readable medium having code executable by the aforementioned set of one or more processors A device having the code, Code to acquire medical images with geometric shapes related to medical devices, The aforementioned medical image is analyzed to obtain a code that identifies the geometric shape, A code that identifies a subset of the image data of the medical image related to the geometric shape, The subset of the image data includes a code that indicates skin irritation, A code that provides an indicator of skin irritation and A device having

Images