Systems, methods and computer programs for analyzing images of portion of person to detect severity of medical condition

Abstract

To analyze an image of a body to determine whether the image involves a level of change of a severity of a medical condition.SOLUTION: Methods, systems and computer programs for monitoring skin condition of a person. In one aspect, a method can include: obtaining data representing a first image, the first image depicting skin from at least a portion of a body of a person; generating a severity score that indicates a likelihood that the person is trending towards an increased severity of an auto-immune condition or trending towards a decreased severity of an auto-immune condition; comparing the severity score to a historical severity score, where the historical severity score is indicative of a likelihood that a historical image of the user depicts skin of a person having the auto-immune condition; and determining based on the comparison, whether the person is trending towards an increased severity of the auto-immune condition or trending towards a decreased severity of the auto-immune condition.SELECTED DRAWING: Figure 1

Description

[Background technology]

[0001] Vitiligo is a condition that causes spots on the skin due to loss of skin color, which can occur when pigment-producing cells die or stop functioning. Summary of the Invention

[0002] According to one innovative aspect of the present disclosure, a system is disclosed for analyzing an image of a portion of a person's body to determine whether the image depicts a person with a particular medical condition or a level of change in medical condition severity.

[0003] In one aspect, a data processing system for detecting the onset of an autoimmune condition is disclosed. The system may include one or more computers and one or more storage devices having instructions stored thereon, which, when executed by the one or more computers, cause the one or more computers to perform operations. In one aspect, the operations may include acquiring, by the one or more computers, data representing a first image depicting skin on at least a portion of a human body; providing, by the one or more computers, the data representing the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model depicts the skin of a human having an autoimmune condition; acquiring, by the one or more computers, output data generated by the machine learning model based on processing of the data representing the first image by the machine learning model, the output data representing the likelihood that the first image depicts the skin of a human having an autoimmune condition; and determining, by the one or more computers, whether the human has the autoimmune condition based on the acquired output data.

[0004] Other versions include corresponding devices, methods, and computer programs, where the computer program performs the actions of the method defined by instructions encoded on a computer-readable storage device.

[0005] These and other versions may optionally include one or more of the following features: For example, in some implementations, the person's body part is a face.

[0006] In some implementations, acquiring data representing the first image may include acquiring, by one or more computers, image data that is a selfie image generated by a user device.

[0007] In some embodiments, acquiring data representing the first image may include, based on a determination that access to the camera of the user device is permitted, occasionally acquiring image data representing at least a portion of the person's body using the camera of the user device, where the occasionally acquired image data is generated and acquired without receiving explicit instructions from the person to generate and acquire the image data.

[0008] According to another innovative aspect of the present disclosure, a data processing system for monitoring a person's skin condition is disclosed. The system may include one or more computers and one or more storage devices having instructions stored thereon, which, when executed by the one or more computers, cause the one or more computers to perform operations. In one aspect, the operations include obtaining, by the one or more computers, data representing a first image depicting skin on at least a portion of the person's body; and generating, by the one or more computers, a severity score indicative of a likelihood that the person's autoimmune condition is trending upward in severity or a likelihood that the person's autoimmune condition is trending downward in severity, wherein generating the severity score includes providing, by the one or more computers, the data representing the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model depicts skin of a person having an autoimmune condition; and generating, by the one or more computers, a severity score indicative of a likelihood that the image data processed by the machine learning model is a likelihood that the image data is a representation of skin of a person having an autoimmune condition. obtaining, by one or more computers, output data generated by the machine learning model based on the first image, the output data representing a likelihood that the first image depicts the skin of a person having an autoimmune condition, and the output data generated by the machine learning model is a severity score; comparing, by the one or more computers, the severity score to a past severity score indicating a likelihood that a past image of the user depicts the skin of a person having an autoimmune condition; and determining, by the one or more computers, whether the severity of the person's autoimmune condition is trending upward or whether the severity of the person's autoimmune condition is trending downward based on the comparison.

[0009] Other versions include corresponding devices, methods, and computer programs, where the computer program performs the actions of the method defined by instructions encoded on a computer-readable storage device.

[0010] These and other versions may optionally include one or more of the following features: For example, in some embodiments, determining whether the severity of the person's autoimmune condition is trending upward or whether the severity of the person's autoimmune condition is trending downward may include determining, by one or more computers, that the severity score is greater than a previous severity score by more than a threshold, and determining that the severity of the person's autoimmune condition is trending upward based on determining that the severity score is greater than the previous score by more than a threshold.

[0011] In some embodiments, determining whether the severity of the person's autoimmune condition is trending upward or whether the severity of the person's autoimmune condition is trending downward may include determining, by one or more computers, that the severity score is less than a previous severity score by more than a threshold, and determining that the severity of the person's autoimmune condition is trending downward based on determining that the severity score is less than a previous score by more than a threshold.

[0012] According to another innovative aspect of the present disclosure, a data processing system for detecting the onset of a medical condition is disclosed, which may include one or more computers and one or more storage devices having instructions stored thereon that, when executed by the one or more computers, cause the one or more computers to perform operations. In one aspect, the operations may include obtaining, by one or more computers, data representing a first image depicting skin on at least a portion of a person's body; identifying, by one or more computers, previous images similar to the first image; identifying, by one or more computers, one or more attributes of the previous images to be associated with the first image; generating, by one or more computers, a vector representation of the first image including data describing the one or more attributes; providing, by one or more computers, the generated vector representation of the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model depicts skin of a person with a medical condition; obtaining, by one or more computers, output data generated by the machine learning model based on processing of the generated vector representation of the first image by the machine learning model; and determining, by one or more computers, whether the person is suffering from the medical condition based on the obtained output data.

[0013] Other versions include corresponding devices, methods, and computer programs, where the computer program performs the actions of the method defined by instructions encoded on a computer-readable storage device.

[0014] These and other versions may optionally include one or more of the following features: For example, in some embodiments, the condition comprises an autoimmune condition.

[0015] In some implementations, the one or more attributes include historical image attributes such as lighting conditions, time of day, date, GPS coordinates, facial hair, lesion area, sunscreen use, makeup use, or temporary cuts or bruises.

[0016] In some embodiments, identifying by one or more computers a previous image similar to the first image may include determining by one or more computers that the previous image is a most recently saved image having one or more attributes including data identifying the location of the diseased area within the previous image.

[0017] These and other innovative aspects of the present disclosure are described in more detail in the specification, drawings, and claims. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram of a system for analyzing an image of a portion of a person to determine whether the image depicts a person with a particular medical condition. [Figure 2] 1 is a flowchart of a process for analyzing an image of a portion of a person to determine whether the image depicts a person with a particular medical condition. [Figure 3] 1 is a flowchart of a process for analyzing an image of a portion of a person to determine whether the image depicts a person trending toward increasing severity of a medical condition or decreasing severity of a particular medical condition. [Figure 4] 1 is a flowchart of a process for generating optimized images for input into a machine learning model trained to analyze images of portions of a person to determine whether the images depict a person with a particular medical condition. [Figure 5] FIG. 1 is a diagram of system components that may be used to implement a system for analyzing an image of a portion of a person to determine whether the image depicts a person with a particular medical condition. DETAILED DESCRIPTION OF THE INVENTION

[0019] The present disclosure is directed to systems, methods, and computer programs for analyzing images of a person to detect whether the image depicts the person with a specific medical condition. In some embodiments, the specific medical condition may be an autoimmune condition, such as vitiligo. Detecting whether the person has a specific medical condition may include detecting that the person has the specific medical condition, detecting that the person is trending toward an increase in severity of the specific medical condition, detecting that the person is trending toward a decrease in severity of the specific medical condition, or detecting that the person does not have the specific medical condition.

[0020] Detection of some medical conditions, such as vitiligo, may require analysis of changes in color or other aspects of a person's skin pigment as depicted by an image of at least a portion of the person's body. Therefore, such analysis essentially relies on generating an input image for an image analysis module that presents an accurate depiction of the patient's skin. Numerous environmental and non-environmental factors can cause distortions in an image of a person. For example, environmental factors such as lighting, rain, or fog can distort the accurate representation of a person's skin pigment in an image. Similarly, non-environmental factors such as camera filters, such as "selfie mode," "beauty mode," or programmed image stabilization or image enhancement, can distort the accurate representation of a person's skin pigment. The present disclosure provides a significant technical improvement in that it can preprocess images and modify vector representations of these images to account for these distortions caused by environmental factors, non-environmental factors, or both. As a result, an optimized input image vector representation can be generated for input to the image analysis module of the present disclosure, which more accurately depicts a person's skin pigment compared to input images generated using conventional systems. Therefore, based on the output generated by the image analysis module of the present disclosure, the determination made by the present disclosure as to whether a person depicted in an image has a particular medical condition is more accurate than conventional systems.

[0021] 1 is a diagram of a system 100 for analyzing an image of a portion of a person to determine whether the image depicts a person with a particular medical condition. System 100 may include a user device 110, a network 120, and an application server 130. Application server 130 may include an application programming interface (API) module 131, an input generation module 132, an image analysis module 133, an output analysis module 135, and a notification module 137. Application server 130 also has access to images stored in a past image database 134 and past scores stored in a past score database 136. In some embodiments, one or both of these databases may be stored on application server 130. In other embodiments, all or a portion of one or both of these databases may be stored on another computer accessible by application server 130.

[0022] For purposes of this specification, the term module may include one or more software components, one or more hardware components, or any combination thereof, which may be used to realize the functionality attributed to the respective module according to this specification.

[0023] A software component may include, for example, one or more software instructions that, when executed, cause a computer to perform the functions attributed to the respective modules herein. A hardware component may include, for example, one or more processors, such as a central processing unit (CPU) or a graphics processing unit (GPU), a memory device, or a combination thereof, where the one or more processors are configured to execute the software instructions to cause the one or more processors to perform the functions attributed to the modules herein, and the memory device is configured to store the software instructions. Alternatively or additionally, a hardware component may include one or more circuits, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), configured to perform operations using hardwired logic to perform the functions attributed to the modules herein.

[0024] In some implementations, the system 100 may begin executing a process to generate first image data 112a representing a first image of a body part of the person 105 using the camera 110a of the user device 110. In some implementations, the first image data 112a may include still image data, such as a GIF image or a JPEG image. In some implementations, the first image data 112a may include video data, such as an MPEG-4 video. In some implementations, the user device 110 may include a smartphone. However, in other implementations, the user device 110 may be any device that includes a camera. For example, in some implementations, the user device may be a smartphone, tablet computer, laptop computer, desktop computer, smartwatch, smartglasses, or the like, that includes an integrated camera or is connected to a camera. In the example of FIG. 1 , the user device 110 uses the camera 110a to capture an image of the face of the person 105. However, the present disclosure is not so limited and instead, the camera 110a of the user device 110 may be used to capture an image of any body part of the person 105.

[0025] In some implementations, user device 110 may generate first image data 112a representing a first image of a body part of person 105 in response to a command from person 105. For example, first image data 112a may be generated in response to a user selection of a physical button on user device 110 or a visual representation of a button displayed on a graphical user interface of user device 110. However, the present disclosure need not be so limited. Instead, in some implementations, user device 110 may have programmed logic installed that causes user device 110 to periodically or asynchronously generate image data of a body part of person 105.

[0026] In the latter scenario, the programmed logic of the user device 110 may configure the user device 110 to detect that a body part of the person 105, such as the face of the person 105, is within the line of sight of the camera 110a. Then, based on a determination that a body part of the person is within the line of sight of the camera 110a, the user device 110 may automatically trigger the user device 110 to generate image data representing an image of the face of the person 105. This may ensure that images of the person are continuously captured and analyzed, regardless of whether the person 105 is explicitly engaging with the system 100. This may be important in situations where the person 105 may be suffering from a particular medical condition, such as vitiligo, because the person 105 may be psychologically affected by changes in their skin pigmentation and may be disinclined to open an application, take an image of themselves, and submit it to the application server 130 to determine whether their regimen is increasing or decreasing the severity of their vitiligo.

[0027] The user device 110 may generate a first data structure 112 containing first image data 112a and transmit the generated first data structure 112 to the application server 130 using the network 120. The generated first data structure 112 may include fields that structure the first image data 112a and any metadata necessary to transmit the first image data 112a to the application server 130, such as a destination address of the application server 130. In some implementations, the first data structure 112 may be implemented as multiple different messages used to transmit the first image data 112a from the user device 110 to the application server 130. For example, the concept of the first data structure 112 may be implemented by packetizing the image data 112a into multiple different packets and transmitting the packets over the network 120 toward their intended destinations in the application server 130. In other implementations, first data structure 112 may be conceptually viewed as an electronic message, such as an email sent via SMTP with first image data 112a attached to the email. In the example of Figure 1, network 120 may include a wired Ethernet network, a wired optical network, a WiFi network, a LAN, a WAN, a cellular network, the Internet, or any combination thereof.

[0028] The application server 130 may receive the first data structure 112 via an application programming interface (API) 131. The API 131 may be a software module, a hardware module, or a combination thereof that may act as an interface between one or more user devices, such as the user device 110, and the application server 130. The API 131 may process the first data structure 112 to extract the first image data 112a. The API 131 may provide the first image data 112a as input to the input generation module 132.

[0029] The input generation module 132 may process the first image data 112a to prepare the first image data 112a for input to the image analysis module 133. In some implementations, this may include formal processing, such as vectorizing the first image data 112a for input to the image analysis module 133. Vectorizing the first image data 112a may include, for example, generating a vector including multiple fields, where each field of the vector corresponds to a pixel of the first image data 112a. The generated vector may include, in each of the vector fields, a numerical value that represents one or more characteristics of the pixel of the image to which the field corresponds. The resulting vector may be a numerical representation of the first image data 112a suitable for input and processing by the image analysis module 133. In such implementations, the generated vector may be provided as an input to the image analysis module 133 for further processing by the system 100.

[0030] However, in some implementations, such as the example of FIG. 1 , input generation module 132 may perform additional operations to prepare first image data 112a for input to image analysis module 133 before providing first image data 112a as input to image analysis module 133. For example, input generation module 132 may optimize image 112a for input to image analysis module 133 based on past images stored in past image database 134 showing body parts of person 105. These past images stored in past image database 134 may include images of person 105 previously submitted to application server 130 for analysis. In other implementations, past images stored in past image database 134 may be images obtained from one or more other sources, such as images captured during a doctor's visit or images obtained from a social media account associated with person 105. These examples of past images should not be considered limiting, and past images of person 105 stored in past image database 134 may be obtained by any means.

[0031] In some implementations, one or more of the past images may be associated with metadata that describes attributes of the past image. For example, the metadata may be used to annotate each of a plurality of past images to indicate attributes of the past image, such as lighting conditions, time of day, date, GPS coordinates, facial hair, areas of lesions, sunscreen use, makeup use, or temporary cuts or bruises, and to tag areas as to whether the past image accurately represents the skin pigmentation of person 105, taking into account environmental or non-environmental factors associated with the past image. In some implementations, these tags may be assigned by a human user based on a review of the past images.

[0032] Input generation module 132 may optimize image 112a in a number of different ways using past images stored in past image database 134. For purposes of this disclosure, “optimizing” an image such as image 112a may include (i) generating data representing the image or (ii) generating data associated with the image that can be provided as input to image analysis module 133 to make image 112a more suitable for processing by image analysis module 133. An optimized image may be more suitable for processing by the image analysis module if it causes image analysis module 133 to generate better output data 133a compared to output data generated by image analysis module 133 processing the pre-optimized image. A better output may include, for example, an output that enables output analysis module 135 to make a more accurate determination, based on output data 133a generated by image analysis module 133, regarding whether a person has a particular medical condition, whether the person's particular medical condition is trending upward in severity, whether the person's particular medical condition is trending downward in severity, or whether the person is free of a particular medical condition.

[0033] In some implementations, image 112a may be processed in a number of different ways by input generation module 132 to generate optimized image 112b. In one implementation, input generation module 132 may perform a comparison of newly received image 112a with previous images 134. Upon identifying a previous image that is sufficiently similar to optimized image 112b, input generation module 132 may set the values ​​of one or more fields of the image vector to values ​​corresponding to the metadata attributes of the identified previous image that was identified as similar to input image 112a.

[0034] For example, the input generation module 132 may identify the newly acquired image 112a as similar to one of the previous images. In some implementations, similarity may be determined based on image similarity, for example, based on a vector-based comparison of a vector representing the image 112a with one or more vectors representing respective previous images. If the newly acquired image 112a is determined to be similar to a previous image captured in particular lighting conditions, the input generation module 132 may set a field in the image vector representation of the optimized image 112b to indicate that the image 112a was taken in the particular lighting conditions. This additional information may provide a signal to the image analysis module 133 that may inform the inferences made by the image analysis module 133.

[0035] As another example, if a newly acquired image 112a is identified as similar to a previous image capturing a person 105 wearing sunscreen, the input generation module 132 may set a field in the image vector representation of the optimized image 112b to indicate that the image 112a was taken with the depicted person 105 wearing sunscreen. This additional information may provide a signal to the image analysis module 133 that may inform the inferences made by the image analysis module 133.

[0036] As another example, the input generation module may identify a relationship between the newly acquired image 112a and similar previous images. In some implementations, the similarity between the image 112a and previous images may be identified based on a temporal relationship between the images. For example, a particular previous image may be identified as similar to the image 112a if it is the most recently captured or most recently saved image depicting a portion of the skin of the person 105. In such a case, the input generation module 132 may generate data to include in the vector 112b representing the optimized image based on metadata associated with the similar previous image that indicates the location of known vitiligo lesions depicted on the skin of the person 105 depicted by the previous image. This additional information may provide signals to the image analysis module 133 that may inform the inferences made by the image analysis module 133.

[0037] None of these examples should be construed as limiting the scope of the present disclosure. Instead, any metadata describing any attribute of any past photograph can be used to optimize the image for input to image analysis module 133.

[0038] The input generation module 132 may generate a vector representation of the optimized image 112b for input to the image analysis module. The vector representation may include a vector including multiple fields, each field of the vector corresponding to a pixel of the first image data 112a, with one or more fields representing additional information attributed to the first image data 112a from one or more similar previous images. The generated vector 112b may include a numerical value for each of the vector fields representing one or more characteristics of the image pixel to which the field corresponds, with the one or more numerical values ​​indicating the presence, absence, degree, location, or other characteristics of the additional information attributed to the input image.

[0039] The image analysis module 133 may be configured to analyze the vector representation of the optimized image 112b and generate output data 133a, where the output data 133a indicates the likelihood that the image 112a represented by the vector representation of the optimized image 112b depicts a person with a medical condition, such as vitiligo. Based on the vectors representing the optimized image data 112b processed by the image analysis module 133, the output data 133a generated by the image analysis model 133 may be analyzed by the output analysis module 135 to determine whether the person 105 has a medical condition.

[0040] In some implementations, the image analysis module 133 may include one or more machine learning models that are trained to determine the likelihood that image data, such as the vector representation of the optimized image data 112b processed by the machine learning models, represents an image depicting the skin of an individual 105 with one or more medical conditions, such as an autoimmune condition. In some implementations, the autoimmune condition may be vitiligo. That is, the machine learning model may be trained to generate output data 133a, which may represent a value such as the probability that the individual depicted in the image data represented by the vector representation 112b processed by the machine learning model is likely to have vitiligo or is likely not to have vitiligo. However, the machine learning model does not actually classify the output data 133a generated by the machine learning model. Instead, the machine learning model generates the output data 133a and provides the output data 133a to the output analysis module 135, which may be configured to threshold the output data 133a into one or more classes of individuals 105.

[0041] The machine learning model can be trained in a number of different ways. In one embodiment, training can be accomplished using a simulator that generates training labels for training vectors representing optimized images. The training labels can provide an indication as to whether the training vector representation corresponds to an image of an individual with a pathology or an image of an individual without the pathology. In such an embodiment, each training vector representing the optimized image can be provided as an input to the machine learning model and processed by the machine learning model. The training output generated by the machine learning model can then be used to identify a predicted label for the training vector representation. The predicted label for the training vector representation can be compared to the training label corresponding to the processed training vector representation. Parameters of the first machine learning model can then be adjusted based on the difference between the predicted label and the training label. This process can continue iteratively for each of multiple training vector representations until the predicted label for the newly processed training vector representation begins to match the training label generated by the simulator for the training vector representation within a predetermined error level.

[0042] Output data 133a generated by image analysis module 133, such as a machine learning model trained to process the vector representation of the optimized image and generate output data 133a indicative of the likelihood that the image corresponding to the vector representation depicts a person with a particular medical condition, may be provided as input to output analysis module 135. Output analysis module 135 may receive the output data 133a and apply one or more business logic rules, such as probabilities, to the output data 133a to determine whether the person depicted in image 112a on which the vector representation of the optimized image was based is with or without the medical condition.

[0043] In such an implementation, a single threshold may be used by the output analysis module 135 to evaluate the output data 133a. For example, in some implementations, the output analysis module 135 may obtain the output data 133a, such as a probability, and compare the obtained output data 133a to a predetermined threshold. If the output analysis module 135 determines that the obtained output data 133a does not meet the predetermined threshold, the output analysis module 135 may determine that the person 105 does not have a particular medical condition. Alternatively, if the output analysis module 135 determines that the obtained output data 133a meets the predetermined threshold, the output analysis module 135 may determine that the person 105 has a particular medical condition.

[0044] In some implementations, output analysis module 135 may generate output data 135a, where output data 135a includes data indicative of a determination made by output analysis module 135 based on generated output data 133a regarding whether person 105 is suffering from a medical condition. Notification module 137 may generate notification 137a that includes rendering, when rendered by user device 110, an alert or other visual message on the display of user device 110 that communicates to person 105 the determination made by output analysis module 135. However, the present disclosure need not be limited in this manner. For example, notification 137a, when processed by user device 110, may be configured to communicate the determination of output analysis module 135 in other manners. For example, notification 137a, when processed by user device 110, may be configured to communicate the result of the determination of output analysis module 135 based on output data 133a via haptic feedback or an audio message, separate from or in combination with the visual message. The notification 137 a may be sent by the application server 130 to the user device 110 over the network 120 .

[0045] However, the subject matter herein is not limited to application server 130 sending notification 137a to user device 110. For example, application server 130 may also send notification 137a to another computer, such as a different user device. In some implementations, notification 137a may be sent to a user device of person 105's doctor, family member, or other person, for example.

[0046] Output analysis module 135 may also make other types of determinations. In some implementations, for example, output analysis module 135 may make a determination regarding whether a vector representation of an optimized image corresponds to an image depicting a person whose medical condition is trending upward or downward in severity.

[0047] 1 , after image analysis module 133 generates output data based on processing the vector representation of optimized image 112b, output analysis module 135 may store output data 133a, such as a probability or severity score, in historical scores 136 database. This output data may be used as a severity score representing the level of severity of a medical condition suffered by patient 105 depicted by image 112a. In some implementations, this severity score may indicate the likelihood that the severity of the person's 105 condition is trending upward or the likelihood that the severity of the person's 105 condition is trending downward. Then, at a later point in time, user device 110 may capture a second image 114a of user 105 using camera 110a. User device 110 may transmit second image 114a to an application server over network 120 using second data structure 114. The API module 131 may receive the second data structure, extract the image 114 a, and then provide the image 114 a as an input to the input generation module 132 .

[0048] Continuing with this example, the input generation module 132 may perform the operations described above to optimize the image 114a. In some implementations, this may include performing a search of the past image database 134 and populating the current image 114a with attributes of one or more of the past images. The input generation module 132 may generate a second vector representation of the optimized image 114b based on the populated attributes. The input generation module 132 may provide the second vector representation of the optimized image 114b as an input to the image analysis module 133. The image analysis module 133 may process the second vector representation of the optimized image 114b to generate second output data 133b, where the second output data 133b indicates the likelihood that the second image 114a depicts a person 105 with a particular medical condition.

[0049] At this point, the output analysis module 135 may analyze the second output data 133b generated based on the second vector representation of the optimized image 114b, taking into account the first output data 133a generated based on the first vector representation of the optimized image 112b. Specifically, the output analysis module 135 may determine, based on changes in the second output data 133b relative to the first output data 133a, whether the severity of a particular medical condition of the person 105 depicted by the image 114a is trending upward or downward. For example, assume that a scale has been established in which an output value of "1" means that the person has a medical condition and an output value of "0" means that the person does not have the medical condition. Under such a scale, if the first output data 133a is 0.65 and the second output data 133b is 0.78, the difference between the first output data 133a and the second output data 133b indicates that the severity of the medical condition of the person 105 is trending upward. Similarly, in a scenario where the first output data 133a is 0.65 and the second output data 133b is 0.49 on the same scale, the difference between the first output data 133a and the second output data 133b indicates that the severity of the person's 105 condition is trending downward.

[0050] None of these examples are intended to limit the present disclosure. Other scales may be used, such as, for example, a "1" meaning that a person does not have the condition and a "0" meaning that a person has the condition. As another example, the scale may be determined so that a "-1" means that a person does not have the condition and a "1" means that a person has the condition. In fact, any scale may be used and may be adjusted based on the range of the output data 133 a, 133 b generated by the input generation module 132.

[0051] However, the present disclosure need not be so limited. For example, in some implementations, output analysis module 135 may use other processes, systems, or combinations thereof to determine whether the severity of a particular medical condition of a person depicted by image 114a is trending upward or whether the severity of a particular medical condition of such person is trending downward. For example, in some implementations, output analysis module 135 may include one or more machine learning models that have been trained to predict whether output data 133b generated by ML model 133 indicates an upward trend in the severity of a particular medical condition of a person depicted by image 114a or an downward trend in the severity of a particular medical condition of such person.

[0052] More specifically, the output analysis module 135 of such an embodiment may include one or more machine learning models trained to identify the likelihood that the severity of a medical condition (e.g., an autoimmune condition) of a person associated with a current severity score generated based on image 114a and one or more previous severity scores, such as the severity score generated based on image 112a, is trending upward or the likelihood that the severity of such person's medical condition (e.g., an autoimmune condition) is trending downward. That is, the machine learning models may be trained to generate output data 135a that may represent values ​​such as the probability that the severity of a medical condition (e.g., an autoimmune condition) of a person associated with a current severity score generated based on image 114a and one or more previous severity scores, such as the severity score generated based on image 112a, is trending upward or the probability that the severity of such person's medical condition (e.g., an autoimmune condition) is trending downward. The output data generated by the one or more machine learning models of the output analysis module 135 may then be analyzed to determine whether the severity of the medical condition (e.g., an autoimmune condition) of the person associated with the current severity score and the one or more previous severity scores is trending upward or whether the severity of the medical condition (e.g., an autoimmune condition) of such person is trending downward. In some embodiments, the one or more machine learning models may be trained to receive multiple previous severity scores as inputs in addition to the current severity score to provide the machine learning model with more data signals to consider when determining whether the medical condition of the person associated with the severity score is trending upward or downward.

[0053] The determination made by output analysis module 135 may be transmitted to user device 110 or other user devices using notification module 137a. For example, output analysis module 135 may generate output data 135a indicating whether the severity of the medical condition of person 105 is trending upward, whether the severity of the medical condition of person 105 is trending downward, whether there is no change in the severity of the medical condition of person 105, etc. The output data 135a may be provided to notification module 137, which may generate notification 137a based on the output data 135a. Application server 130 may notify user device 110 or other user devices by sending notification 137a to one or more of the respective user devices.

[0054] Additional applications may be used to analyze output data 135a to indicate whether the severity of the person's 105 condition is trending upward, whether the severity of the person's 105 condition is trending downward, or whether there is no change in the severity of the person's 105 condition. In some implementations, for example, output data 135a or notification 137a may include data representing a degree of change between first output data 133a and second output data 133b based on vectors corresponding to first image data 112a and second image data 114a, respectively. Software on user device 110 or another user device may analyze the degree of change between first output data 133a and second output data 133b to generate one or more alerts to person 105 or the person's physician. Such alerts may remind person 105 to apply their medication, suggest to the physician to adjust the person's prescription, or the like. For example, in some implementations where the medical condition is vitiligo, the software may be configured to determine that a difference between the first output data 133a and the second output data 133b indicates that the user's vitiligo lesions are becoming more severe. In such cases, the software may generate an alert based on the degree of change between the first output data 133a and the second output data 133b to remind the person 105 to apply their medication, suggest that the person 105 apply their medication more frequently, or suggest that the doctor increase the person's 105 medication dosage. Other applications of a similar scope are also intended to be within the scope of this disclosure. While the analysis for these reminder / suggestion alerts is described as being performed by an application on the user device, the disclosure is not so limited. Alternatively, the analysis of the degree of difference between the output data 133a and the output data 133b may be performed by the output analysis module 135 on the application server 130, and the reminder / suggestion alert may be generated by the notification module 137.

[0055] Although it is not explicitly shown that notification module 137 passes notification 137a through API module 131, in some embodiments, data communication between the user device and the application server is considered to occur through API 131, which is a form of middleware between application server 130 and the user device(s).

[0056] 2 is a flowchart of a process 200 for analyzing an image of a portion of a person to determine whether the image depicts a person with a particular medical condition. Generally, the process 200 may include: acquiring, by one or more computers, data representing a first image depicting skin on at least a portion of the person's body (210); providing, by the one or more computers, the data representing the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model depicts the skin of a person with an autoimmune condition (220); acquiring, by the one or more computers, output data generated by the machine learning model based on processing of the data representing the first image by the machine learning model, the output data representing a likelihood that the first image depicts the skin of a person with the autoimmune condition (230); and determining, by the one or more computers, whether the person has the autoimmune condition based on the acquired output data (240).

[0057] 3 is a flowchart of a process 300 for analyzing an image of a portion of a person to determine whether the image depicts a person trending toward an increasing severity of a medical condition or a decreasing severity of a particular medical condition. For example, in some embodiments, process 300 may include obtaining, by one or more computers, data representing a first image depicting skin on at least a portion of the person's body (310), generating, by the one or more computers, a severity score indicative of a likelihood that the severity of the person's autoimmune condition is trending toward an increasing severity or a likelihood that the severity of the person's autoimmune condition is trending toward a decreasing severity (320), comparing, by the one or more computers, the severity score to a previous severity score indicative of a likelihood that previous images of the user depict the skin of a person with an autoimmune condition (330), and determining, by the one or more computers, whether the severity of the person's autoimmune condition is trending toward an increasing severity or a decreasing severity of the person's autoimmune condition based on the comparison (340).

[0058] FIG. 4 is a flowchart of a process 400 for generating an optimized image for input into a machine learning model trained to analyze an image of a portion of a person to determine whether the image depicts a person with a particular medical condition. Generally, process 400 may include obtaining, by one or more computers, data representing a first image depicting skin on at least a portion of a person's body (410); identifying, by one or more computers, previous images similar to the first image (420); identifying, by one or more computers, one or more attributes of the previous images to be associated with the first image (430); generating, by one or more computers, a vector representation of the first image including data describing the one or more attributes (440); providing, by one or more computers, the generated vector representation of the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model depicts the skin of a person with a particular medical condition (450); obtaining, by one or more computers, output data generated by the machine learning model based on processing of the generated vector representation of the first image by the machine learning model (460); and determining, by one or more computers, whether the person is affected by the medical condition based on the obtained output data (470).

[0059] FIG. 5 is a diagram of system components that may be used to implement a system for analyzing an image of a portion of a person to determine whether the image depicts a person with a particular medical condition.

[0060] Computing device 500 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Computing device 550 is intended to represent various forms of mobile devices, such as personal digital assistants, mobile phones, smartphones, and other similar computing devices. Additionally, computing device 500 or 550 may include a Universal Serial Bus (USB) flash drive. The USB flash drive may store an operating system and other applications. The USB flash drive may include input / output components, such as a wireless transmitter or a USB connector that can be inserted into a USB port on another computing device. The components, their connections and relationships, and their functions shown herein are for illustrative purposes only and are not intended to limit the scope of the invention(s) described and / or claimed herein.

[0061] Computing device 500 includes a processor 502, memory 504, a storage device 506, a high-speed interface 508 connecting memory 504 to a high-speed expansion port 510, and a low-speed interface 512 connecting a low-speed bus 514 to storage device 506. Each of the components 502, 504, 506, 508, 510, and 512 are interconnected using various buses and may be mounted on a common motherboard or otherwise as desired. Processor 502 processes instructions executed within computing device 500, including instructions stored in memory 504 or storage device 506, to display graphical information for a GUI on an external input / output device, such as a display 516 connected to high-speed interface 508. In other embodiments, multiple processors and / or multiple buses may be used, along with multiple memories and memory types as desired. Multiple computing devices 500 may also be connected, for example, as a server bank, a cluster of blade servers, or a multiprocessor system, with each device providing a portion of the required operations.

[0062] The memory 504 stores information within the computing device 500. In one implementation, the memory 504 is a volatile memory unit(s). In another implementation, the memory 504 is a non-volatile memory unit(s). The memory 504 may also be another form of computer-readable medium, such as a magnetic disk or an optical disk.

[0063] The storage device 506 can provide mass storage for the computing device 500. In one embodiment, the storage device 506 can be or include a computer-readable medium, such as a floppy disk drive, a hard disk drive, an optical disk drive, or a tape drive, or a device array including flash memory or other similar solid-state memory devices, or devices in a storage area network or other structure. A computer program product can be tangibly embodied on an information carrier. The computer program product can also include instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer-readable or machine-readable medium, such as memory 504, the storage device 506, or memory on the processor 502.

[0064] The high-speed controller 508 manages bandwidth-intensive operations of the computing device 500, while the low-speed controller 512 manages less bandwidth-intensive operations. This allocation of functionality is merely exemplary. In one embodiment, the high-speed controller 508 is connected to the memory 504, a display 516, e.g., via a graphics processor or accelerator, and a high-speed expansion port 510, which can accept various expansion cards (not shown). In an embodiment, the low-speed controller 512 is connected to the storage device 506 and the low-speed expansion port 514. The low-speed expansion port, which may include various communication ports, e.g., USB, Bluetooth, Ethernet, Wireless Ethernet, etc., may be connected to one or more input / output devices, e.g., via a network adapter, such as a keyboard, a pointing device, a microphone / speaker pair, a scanner, or a network device, such as a switch or router. The computing device 500, as shown in the figure, can be implemented in many different forms. For example, the computing device 500 may be implemented as a standard server 520, or multiple times in a cluster of such servers. Computing device 500 may also be implemented as part of a rack server system 524. Furthermore, computing device 500 may be implemented in a personal computer, such as a laptop computer 522. Alternatively, the components of computing device 500 may be combined with other components of a mobile device (not shown), such as device 550. Each such device may include one or more of computing devices 500, 550, and the entire system may be made up of multiple computing devices 500, 550 in communication with each other.

[0065] Computing device 500, as shown in the figure, can be implemented in many different forms. For example, computing device 500 may be implemented as a standard server 520, or multiple times in a cluster of such servers. Computing device 500 may also be implemented as part of a rack server system 524. Furthermore, computing device 500 may be implemented in a personal computer, such as a laptop computer 522. Alternatively, the components of computing device 500 may be combined with other components in a mobile device (not shown), such as device 550. Each such device may include one or more of computing devices 500, 550, and the entire system may be made up of multiple computing devices 500, 550 communicating with each other.

[0066] Computing device 550 includes a processor 552, memory 564, and input / output devices such as a display 554, a communication interface 566, a transceiver 568, among other components. To provide additional storage, device 550 may also be provided with a storage device such as a microdrive or other device. Each of components 552, 564, 554, 566, and 568 are interconnected using various buses, and some of the components may be mounted on a common motherboard or otherwise as desired.

[0067] The processor 552 may execute instructions within the computing device 550, including instructions stored in the memory 564. The processor may be implemented as a chipset of chips including separate analog and digital processors. Furthermore, the processor may be implemented using any of a number of architectures. For example, the processor 552 may be a CISC (Complex Instruction Set Computer) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimum Instruction Set Computer) processor. The processor may provide coordination of other components of the device 550, such as, for example, a user interface, applications executed by the device 550, and control of wireless communications by the device 550.

[0068] Processor 552 may communicate with a user via control interface 558 and display interface 556 connected to display 554. Display 554 may be, for example, a TFT (thin film transistor liquid crystal) display, an OLED (organic light emitting diode) display, or other suitable display technology. Display interface 556 may comprise suitable circuitry for driving display 554 to present graphics and other information to a user. Control interface 558 may receive commands from a user and translate the commands for submission to processor 552. Additionally, an external interface 562 may be provided in communication with processor 552 to enable short-range communication between device 550 and other devices. External interface 562 may, for example, provide for wired communication in some implementations or for wireless communication in other implementations; multiple interfaces may also be used.

[0069] Memory 564 stores information within computing device 550. Memory 564 may be implemented as one or more of computer-readable medium(s), volatile memory unit(s), or non-volatile memory unit(s). Expansion memory 574 may also be provided and connected to device 550 via expansion interface 572, which may include, for example, a SIMM (single in-line memory module) card interface. Such expansion memory 574 may provide additional storage space for device 550 or may store applications or other information for device 550. Specifically, expansion memory 574 may include instructions for performing or supplementing the processes described above, and may also include secure information. Thus, for example, expansion memory 574 may be provided as a security module for device 550 and may be programmed with instructions that enable secure use of device 550. Furthermore, secure applications may be provided via SIMM cards along with additional information, such as placing identifying information on the SIMM card in an unhackable manner.

[0070] The memory may include, for example, flash memory and / or NVRAM memory, as discussed below. In one embodiment, a computer program product is tangibly embodied on an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier may be a computer-readable or machine-readable medium, such as memory 564, expansion memory 574, or memory on processor 552, and may be received, for example, via transceiver 568 or external interface 562.

[0071] Device 550 may communicate wirelessly via communication interface 566, which may include digital signal processing circuitry as needed. Communication interface 566 may provide for communication under various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication may be performed, for example, via radio frequency transceiver 568. Additionally, short-range communication may be performed using, for example, Bluetooth, Wi-Fi, or other such transceivers (not shown). Furthermore, GPS (Global Positioning System) receiver module 570 may provide additional navigation-related and location-related wireless data to device 550, which may be used as needed by applications executing on device 550.

[0072] Device 550 may also perform audible communications using audio codec 560, which may receive speech information from a user and convert it into usable digital information. Similarly, audio codec 560 may generate audible sounds for the user, such as through a speaker in the handset of device 550. Such sounds may include sounds from voice calls, recorded sounds such as voice messages, music files, and the like, and may also include sounds generated by applications running on device 550.

[0073] The computing device 550, as shown, can be implemented in many different forms. For example, the computing device 550 may be implemented as a mobile phone 580. The computing device 550 may also be implemented as part of a smartphone 582, a personal digital assistant, or other similar mobile device.

[0074] Various implementations of the systems and methods described herein may be implemented in digital electronic circuitry, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations of such implementations. These various implementations may include implementation of one or more computer programs executable and / or interpretable by a programmable system including at least one programmable processor, which may be special purpose or general purpose, coupled to, and capable of receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

[0075] These computer programs (also known as programs, software, software applications, or code) contain machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device, such as a magnetic disk, optical disk, memory, programmable logic device (PLD), used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives the machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0076] To provide for interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device for displaying information to a user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and a keyboard and pointing device for allowing a user to provide computer input, such as a mouse or trackball. Other types of devices may be used to provide for interaction with a user as well; for example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including voice input, speech input, or tactile input.

[0077] The systems and techniques described herein may be implemented in a computing system that includes back-end components, such as a data server, or middleware components, such as an application server, or front-end components, such as a client computer having a graphical user interface or web browser that allows a user to interact with an implementation of the systems and techniques described herein, or any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, e.g., a communications network. Examples of communications networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0078] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0079] Other embodiments Numerous embodiments have been described. However, it will be understood that various modifications may be made without departing from the spirit and scope of the present invention. Furthermore, the logic flows depicted in the figures do not require the particular order or sequential order shown to achieve desirable results. Furthermore, other steps may be provided to, or steps may be deleted from, the described flows, and other components may be added to, or deleted from, the described systems. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. 1. A method for detecting the onset of an autoimmune condition, comprising: acquiring, by one or more computers, data representing a first image depicting skin on at least a portion of a human body; providing, by the one or more computers, the data representing the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model describes the skin of a person with the autoimmune condition; obtaining, by the one or more computers, output data generated by the machine learning model based on the data representing the first image being processed by the machine learning model, the output data representing a likelihood that the first image depicts skin of a person with the autoimmune condition; determining, by the one or more computers, whether the person has the autoimmune condition based on the obtained output data; and A method comprising:

2. The method of claim 1 , wherein the part of the body of the person is a face.

3. Obtaining the data representing the first image includes: acquiring, by the one or more computers, image data, the image data being a selfie image generated by a user device; The method of claim 1 , comprising:

4. Obtaining the data representing the first image includes: based on a determination that access to a camera of a user device is permitted, acquiring image data representing at least a portion of a body of a person using the camera of the user device from time to time, the image data acquired from time to time being image data generated and acquired without receiving explicit instructions from the person to generate and acquire the image data; The method of claim 1 , comprising:

5. 1. A data processing system for detecting the onset of an autoimmune condition, said system comprising: one or more computers; one or more storage devices containing instructions; the instructions, when executed by the one or more computers, cause the one or more computers to perform operations, the operations including: acquiring, by the one or more computers, data representing a first image depicting skin on at least a portion of a human body; providing, by the one or more computers, the data representing the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model describes the skin of a person with the autoimmune condition; obtaining, by the one or more computers, output data generated by the machine learning model based on the data representing the first image being processed by the machine learning model, the output data representing a likelihood that the first image depicts skin of a person with the autoimmune condition; determining, by the one or more computers, whether the person has the autoimmune condition based on the obtained output data; and Including, the system.

6. The system of claim 5 , wherein the part of the body of the person is a face.

7. Obtaining the data representing the first image includes: acquiring, by the one or more computers, image data, the image data being a selfie image generated by a user device; The system of claim 5 , comprising:

8. Obtaining the data representing the first image includes: based on a determination that access to a camera of a user device is permitted, acquiring image data representing at least a portion of a body of a person using the camera of the user device from time to time, the image data acquired from time to time being image data generated and acquired without receiving explicit instructions from the person to generate and acquire the image data; The system of claim 5 , comprising:

9. A non-transitory computer-readable medium storing software including instructions executable by one or more computers, the instructions, when executed by the one or more computers, causing the one or more computers to perform operations, the operations including: acquiring, by the one or more computers, data representing a first image depicting skin on at least a portion of a human body; providing, by the one or more computers, the data representing the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model depicts the skin of a person with an autoimmune condition; obtaining, by the one or more computers, output data generated by the machine learning model based on the data representing the first image being processed by the machine learning model, the output data representing a likelihood that the first image depicts skin of a person with the autoimmune condition; determining, by the one or more computers, whether the person has the autoimmune condition based on the obtained output data; and 1. A computer-readable medium comprising:

10. The computer-readable medium of claim 9 , wherein the part of the body of the person is a face.

11. Obtaining the data representing the first image includes: acquiring, by the one or more computers, image data, the image data being a selfie image generated by a user device; 10. The computer-readable medium of claim 9, comprising:

12. Obtaining the data representing the first image includes: based on a determination that access to a camera of a user device is permitted, acquiring image data representing at least a portion of a body of a person using the camera of the user device from time to time, the image data acquired from time to time being image data generated and acquired without receiving explicit instructions from the person to generate and acquire the image data; 10. The computer-readable medium of claim 9, comprising:

13. 1. A method for monitoring a skin condition of a person, comprising: acquiring, by one or more computers, data representing a first image depicting skin on at least a portion of a human body; generating, by the one or more computers, a severity score indicative of a likelihood that the severity of the autoimmune condition of the person is trending upward or a likelihood that the severity of the autoimmune condition of the person is trending downward, wherein generating the severity score comprises: providing, by the one or more computers, the data representing the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model describes the skin of a person with the autoimmune condition; obtaining, by the one or more computers, output data generated by the machine learning model based on the data representing the first image being processed by the machine learning model, the output data representing a likelihood that the first image depicts skin of a person with the autoimmune condition, and the output data generated by the machine learning model being the severity score; generating said severity score, comparing, by the one or more computers, the severity score to a historical severity score indicating the likelihood that a historical image of the user depicts the skin of a person with the autoimmune condition; determining, by the one or more computers, whether the severity of the autoimmune condition of the person is trending upward or downward based on the comparison; and A method comprising:

14. Determining whether the severity of the autoimmune condition of the person is trending upward or downward may include: determining, by the one or more computers, that the severity score is greater than the past severity score by more than a threshold; determining that the severity of the autoimmune condition of the person is on an increasing trend based on determining that the severity score is greater than the previous score by more than the threshold; 14. The method of claim 13, comprising:

15. Determining whether the severity of the autoimmune condition of the person is trending upward or downward may include: determining, by the one or more computers, that the severity score is less than the past severity score by more than a threshold; determining that the severity of the autoimmune condition of the person is on a decreasing trend based on determining that the severity score is lower than the past score by more than the threshold; 14. The method of claim 13, comprising:

16. 1. A data processing system for monitoring a human skin condition, said system comprising: one or more computers; one or more storage devices containing instructions; the instructions, when executed by the one or more computers, cause the one or more computers to perform operations, the operations including: acquiring, by the one or more computers, data representing a first image depicting skin on at least a portion of a human body; generating, by the one or more computers, a severity score indicative of a likelihood that the severity of the autoimmune condition of the person is trending upward or a likelihood that the severity of the autoimmune condition of the person is trending downward, wherein generating the severity score comprises: providing, by the one or more computers, the data representing the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model describes the skin of a person with the autoimmune condition; obtaining, by the one or more computers, output data generated by the machine learning model based on the data representing the first image being processed by the machine learning model, the output data representing a likelihood that the first image depicts skin of a person with the autoimmune condition, and the output data generated by the machine learning model being the severity score; generating said severity score, comparing, by the one or more computers, the severity score to a historical severity score indicating the likelihood that a historical image of the user depicts the skin of a person with the autoimmune condition; determining, by the one or more computers, whether the severity of the autoimmune condition of the person is trending upward or downward based on the comparison; and Including, the system.

17. Determining whether the severity of the autoimmune condition of the person is trending upward or downward may include: determining, by the one or more computers, that the severity score is greater than the past severity score by more than a threshold; determining that the severity of the autoimmune condition of the person is on an increasing trend based on determining that the severity score is greater than the previous score by more than the threshold; 17. The system of claim 16, comprising:

18. Determining whether the severity of the autoimmune condition of the person is trending upward or downward may include: determining, by the one or more computers, that the severity score is less than the past severity score by more than a threshold; determining that the severity of the autoimmune condition of the person is on a decreasing trend based on determining that the severity score is lower than the past score by more than the threshold; 17. The system of claim 16, comprising:

19. A non-transitory computer-readable medium storing software including instructions executable by one or more computers, the instructions, when executed by the one or more computers, causing the one or more computers to perform operations, the operations including: acquiring, by the one or more computers, data representing a first image depicting skin on at least a portion of a human body; generating, by the one or more computers, a severity score indicative of a likelihood that the severity of the autoimmune condition of the person is trending upward or a likelihood that the severity of the autoimmune condition of the person is trending downward, wherein generating the severity score comprises: providing, by the one or more computers, the data representing the first image as input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model describes the skin of a person with the autoimmune condition; obtaining, by the one or more computers, output data generated by the machine learning model based on the data representing the first image being processed by the machine learning model, the output data representing a likelihood that the first image depicts skin of a person with the autoimmune condition, and the output data generated by the machine learning model being the severity score; generating said severity score, comparing, by the one or more computers, the severity score to a historical severity score indicating the likelihood that a historical image of the user depicts the skin of a person with the autoimmune condition; determining, by the one or more computers, whether the severity of the autoimmune condition of the person is trending upward or downward based on the comparison; and 1. A computer-readable medium comprising:

20. Determining whether the severity of the autoimmune condition of the person is trending upward or downward may include: determining, by the one or more computers, that the severity score is greater than the past severity score by more than a threshold; determining that the severity of the autoimmune condition of the person is on an increasing trend based on determining that the severity score is greater than the previous score by more than the threshold; 20. The computer-readable medium of claim 19, comprising:

21. Determining whether the severity of the autoimmune condition of the person is trending upward or downward may include: determining, by the one or more computers, that the severity score is less than the past severity score by more than a threshold; determining that the severity of the autoimmune condition of the person is on a decreasing trend based on determining that the severity score is lower than the past score by more than the threshold; 20. The computer-readable medium of claim 19, comprising:

22. 1. A method for detecting the onset of a medical condition, comprising: acquiring, by one or more computers, data representing a first image depicting skin on at least a portion of a human body; identifying, by the one or more computers, past images similar to the first image; identifying, by the one or more computers, one or more attributes of the previous image to be associated with the first image; generating, by the one or more computers, a vector representation of the first image including data describing the one or more attributes; providing, by the one or more computers, the generated vector representation of the first image as an input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model depicts the skin of a person having the medical condition; obtaining, by the one or more computers, output data generated by the machine learning model based on the generated vector representation of the first image being processed by the machine learning model; determining, by the one or more computers, whether the person has the medical condition based on the obtained output data; and A method comprising:

23. 23. The method of claim 22, wherein the condition comprises an autoimmune condition.

24. 23. The method of claim 22, wherein the one or more attributes include past image attributes such as lighting conditions, time of day, date, GPS coordinates, facial hair, lesion area, sunscreen use, makeup use, or temporary cuts or bruises.

25. Identifying, by the one or more computers, past images similar to the first image, identifying, by the one or more computers, the past image as the most recently saved image of the person; 23. The method of claim 22, comprising:

26. 26. The method of claim 25, wherein the one or more attributes include data identifying the location of a diseased area in the previous image.

27. 1. A data processing system for detecting the onset of a medical condition, said system comprising: one or more computers; one or more storage devices containing instructions; the instructions, when executed by the one or more computers, cause the one or more computers to perform operations, the operations including: acquiring, by the one or more computers, data representing a first image depicting skin on at least a portion of a human body; identifying, by the one or more computers, past images similar to the first image; identifying, by the one or more computers, one or more attributes of the previous image to be associated with the first image; generating, by the one or more computers, a vector representation of the first image including data describing the one or more attributes; providing, by the one or more computers, the generated vector representation of the first image as an input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model depicts the skin of a person having the medical condition; obtaining, by the one or more computers, output data generated by the machine learning model based on the generated vector representation of the first image being processed by the machine learning model; determining, by the one or more computers, whether the person has the medical condition based on the obtained output data; and Including, the system.

28. 28. The system of claim 27, wherein the medical condition comprises an autoimmune condition.

29. 28. The system of claim 27, wherein the one or more attributes include past image attributes such as lighting conditions, time of day, date, GPS coordinates, facial hair, lesion area, sunscreen use, makeup use, or temporary cuts or bruises.

30. Identifying, by the one or more computers, past images similar to the first image, identifying, by the one or more computers, the past image as the most recently saved image of the person; 28. The system of claim 27, comprising:

31. 31. The system of claim 30, wherein the one or more attributes include data identifying a location of a diseased area within the previous image.

32. A non-transitory computer-readable medium storing software including instructions executable by one or more computers, the instructions, when executed by the one or more computers, causing the one or more computers to perform operations, the operations including: acquiring, by the one or more computers, data representing a first image depicting skin on at least a portion of a human body; identifying, by the one or more computers, past images similar to the first image; identifying, by the one or more computers, one or more attributes of the previous image to be associated with the first image; generating, by the one or more computers, a vector representation of the first image including data describing the one or more attributes; providing, by the one or more computers, the generated vector representation of the first image as an input to a machine learning model trained to determine a likelihood that image data processed by the machine learning model depicts the skin of a person with a medical condition; obtaining, by the one or more computers, output data generated by the machine learning model based on the generated vector representation of the first image being processed by the machine learning model; determining, by the one or more computers, whether the person has the medical condition based on the obtained output data; and 1. A computer-readable medium comprising:

33. 33. The computer-readable medium of claim 32, wherein the medical condition comprises an autoimmune condition.

34. 33. The computer-readable medium of claim 32, wherein the one or more attributes include past image attributes such as lighting conditions, time of day, date, GPS coordinates, facial hair, lesion area, sunscreen use, makeup use, or temporary cuts or bruises.

35. Identifying, by the one or more computers, past images similar to the first image, identifying, by the one or more computers, the past image as the most recently saved image of the person; 33. The computer-readable medium of claim 32, comprising:

36. 36. The computer-readable medium of claim 35, wherein the one or more attributes include data identifying a location of a diseased area within the previous image.

Images