A system and method for analysing excreta

Abstract

The invention provides a computer implemented method of analysing at least one excreta, comprising the steps of; - receiving a first visual information of a first excreta of a first subject; identifying at least one region of interest within the first visual information; - applying a trained machine learning model to the identified at least one region of interest in order to identify characteristics of the first excreta; and determining a microbiome profile of the first excreta based on the identified characteristics of the first excreta, wherein said characteristics comprise at least one of colour, consistency, and texture. The invention further provides a non-transitory computer readable medium and a system for analysing at least one excreta.

Description

[0001] A system and method for analysing excreta

[0002] FIELD OF THE INVENTION

[0003]

[0001] The invention relates to a method and a system for analysing the consistency of an excreta and determining microbiome profile of the excreta.

[0004] BACKGROUND OF THE INVENTION

[0005]

[0002] Excreta samples of a living subject can be indicating of health conditions for said subject. Chemical analysis of the excreta may provide intrinsic information relating to gut health, for example. By contrast, the visual appearance of the excreta (e.g. stool) may be indicating of a condition relating to the movement of bowels, such as identifying a subject being constipated or having diarrhea. Other visual indicators of excreta may provide further indicating measures of a bowel movement condition. Such self-assessed visual inspection of excreta, however, is often subjective and open to inconsistencies for periodic evaluation. Furthermore, it is not possible to predict the microbiome profile of the excreta, with merely visual inspection.

[0006]

[0003] Programs or apps are known which allow to introduce or to capture images of stool and manually select a score of a stool analysis scale that better suits the stool on the image, in order to keep a log of the digestive system performance throughout time. Also programs or apps are known which allow the use of colour recognition techniques to automatically detect the colour of stool.

[0007]

[0004] Although these apps may help subjects and / or health care professionals analyse some characteristics of the stool, it is sometimes difficult to manually make a classification, and it is tedious to manually keep a log. In fact, it is not possible to predict the microbiome profile of the excreta, manually. Additionally, in most cases, there is still a need to get an advice from a health care professional, after such an analysis of the stool.

[0008]

[0005] Therefore, there is a need for a more robust visual evaluation of excreta by providing an accurate and fast analysis of the excreta by additionally predicting an estimated microbiome profile of the excreta.

[0009] SUMMARY OF THE INVENTION

[0010]

[0006] According to an aspect, the invention provides a computer implemented method of analysing at least one excreta, comprising the steps of; - receiving a first visual information of a first excreta of a first subject;

[0011] - identifying at least one region of interest within the first visual information; applying a trained machine learning model to the identified at least one region of interest in order to identify characteristics of the first excreta; and determining a microbiome profile of the first excreta based on the identified characteristics of the first excreta, wherein said characteristics comprise at least one of colour, consistency, and texture.

[0012]

[0007] In an embodiment, the method further comprises the step of determining a health condition of the first subject based on the determined microbiome status of the first excreta.

[0013]

[0008] The method may further comprise the step of outputting data indicating at least one health suggestion for the first subject, based on the determined health condition.

[0014]

[0009] In an embodiment, the method further comprises the steps of receiving information about the first subject, wherein said information comprises a dietary status of the first subject, and at least one of gender, age, disease record, body weight, and body length of the first subject, and using said information in determining a health condition.

[0015]

[0010] In an embodiment, the health suggestion comprises at least one nutrition guidance for improving consistency of the first excreta and at least one immunity system indicator.

[0016]

[0011] In an embodiment, the first subject is a baby that is breastfed from a female subject, and wherein the health suggestion further comprises a nutrition guidance for the female subject.

[0017]

[0012] In an embodiment, the microbiome comprises at least one of Enterobacteriaceae, Lachnospiraceae, Ruminococcaceae, Bifidobacteriaceae, Bifidobacterium, and lactobacilli.

[0018]

[0013] In an embodiment, the identifying at least one region of interest within the first visual information comprises the step of pre-processing the first visual information by at least one of flipping, adjusting lightening, and resizing the first visual information.

[0019]

[0014] In an embodiment, applying a trained machine learning model to the determined at least one region of interest in order to identify characteristics of the first excreta comprises the steps of outputting an image of at least one feature extracted from the first visual information and a scale information on the characteristics of said at least one feature, based on the determined at least one region of interest.

[0020]

[0015] In an embodiment, the method further comprises the steps of; receiving an input for adjusting the characteristics of the at least one feature, adjusting the scale information and the image of the at least one feature; and outputting the adjusted scale information and the adjusted image of the at least one feature.

[0021]

[0016] In an embodiment, the adjusted scale information is stored to retrain the trained machine learning model.

[0022]

[0017] In an embodiment, the first excreta comprises stool of the first subject, and wherein the scale information comprises a colour scale and a consistency scale of the at least one feature extracted from the first visual information of the stool.

[0023]

[0018] In an embodiment, the method further comprises the step of storing information on the identified characteristics of the first excreta.

[0024]

[0019] In an embodiment, the method further comprises the step of; receiving a second visual information of a second excreta of a second subject, comparing the first visual information and the second visual information; and determining a similarity level of the first visual information and the second visual information.

[0025]

[0020] In an embodiment, the first subject is fed with a first nutritional composition, and the second subject is fed with a second nutritional composition.

[0026]

[0021] In an embodiment, the method further comprises the steps of; identifying at least a part of characteristics of the second excreta based on the similarity level; and determining a microbiome profile of the second excreta based on the identified part of the characteristics of the second excreta.

[0027]

[0022] In an embodiment, the identifying at least a part of characteristics of the second excreta based on the similarity level, comprises identifying a first part of the characteristics of the second excreta based on the similarity level, identifying at least one region of interest within the second visual information; and applying the trained machine learning model to the determined at least one region of interest in order to identify a second part of the characteristics of the second excreta.

[0028]

[0023] In an embodiment, each of the first and the second visual information comprises at least one of a video data, an image data, a three dimensional data and a thermal image data.

[0029]

[0024] In another aspect, the invention provides a non-transitory computer readable medium for analysing at least one excreta, the non-transitory computer readable medium comprising instructions that, when executed by a processor, cause the processor to perform the afore mentioned method.

[0030]

[0025] In another aspect, the invention provides a system for analysing at least one excreta, the system comprising: one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the system to perform operations the method.

[0031] BRIEF DESCRIPTION OF THE DRAWINGS

[0032]

[0026] Further features and advantages of the invention will become apparent from the description of the invention by way of non-limiting and non-exclusive embodiments. These embodiments are not to be construed as limiting the scope of protection. The person skilled in the art will realize that other alternatives and equivalent embodiments of the invention can be conceived and reduced to practice without departing from the scope of the present invention.

[0033]

[0027] Embodiments of the invention will be described with reference to the accompanying drawings, in which like or same reference symbols denote like, same or corresponding parts, and in which

[0034] Figure 1 shows a computer-implemented method of analysing excreta condition,

[0035] Figures 2 shows an overview of an embodiment of the present invention,

[0036] Figures 3A and 3B show different dispositions of a diaper comprising a stool according to the present invention,

[0037] Figure 4 schematically shows a portable device according to an embodiment of the present invention, and

[0038] Figure 5 schematically shows a portable device and a server according to an embodiment of the present invention,

[0039] Figure 6A is a graph showing aspects of enterobacteriaceae associated with color of recorded stool samples,

[0040] Figure 6B is a graph showing aspects of enterobacteriaceae associated with consistency of the recorded stool samples,

[0041] Figure 7 is a graph showing aspects of lachnospiraceae associated with consistency of the recorded stool samples, and

[0042] Figure 8 is a graph showing aspects of ruminococcaceae associated with consistency of the recorded stool samples.

[0043] DETAILED DESCRIPTION OF EMBODIMENTS

[0044]

[0028] Terms used in the claims and specification are defined as set forth below unless otherwise specified.

[0045]

[0029] The terms “subject” or “patient” are used interchangeably and encompass a cell, tissue, or organism, human or non-human, whether in vivo, ex vivo, or in vitro, male or female.

[0046]

[0030] The terms “treating,” “treatment,” or “therapy” may be used interchangeably.

[0047]

[0031] The terms “excreta”, “stool”, “stool sample”, “urine”, or “feces” may be used interchangeably. The term stool refers to stool (feces) expelled by a subject during a bowel movement session. The stool is the total stool expelled during the bowel movement session (regardless of number of pieces, texture, liquid / solid ratio, etc.).

[0048]

[0032] The term “bowel movement” or “bowel movement session” may be used interchangeably. The term bowel movement refers to a passing of stool during a given period. For example, a subject may have a bowel movement in the morning, and another bowel movement in the night.

[0049]

[0033] It must be noted that, as used in the specification, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise.

[0034] The phrase “and / or,” as used in the specification and in the claims, should be understood to mean “either or both” of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases. Multiple elements listed with “and / or” should be construed in the same fashion, i.e., “one or more” of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the “and / or” clause, whether related or unrelated to those elements specifically identified. Thus, as a nonlimiting example, a reference to “A and / or B”, when used in conjunction with open- ended language such as “comprising” can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements).

[0050]

[0035] As used in the specification and in the claims, “or” should be understood to have the same meaning as “and / or” as defined above. For example, when separating items in a list, “or” or “and / or” shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as “only one of or “exactly one of,” or, when used in the claims, “consisting of,” will refer to the inclusion of exactly one element of a number or list of elements. In general, the term “or” as used shall only be interpreted as indicating exclusive alternatives (i.e. “one or the other but not both”) when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.”

[0051]

[0036] Fig. 1 shows a computer-implemented method of analysing excreta. Some steps in fig. 1 may be merged (e.g., steps 102 - 104), or may be omitted (e.g., step 102).

[0052]

[0037] In step 101, visual information of excreta is obtained. The visual information may be at least one of the video information, image information, three- dimensional information and thermal image information. In the whole document, an image may be used as an example of the visual information, but it should be understood that the image is only an example and other forms of visual information are included in the present invention as well. In step 101, for example, an image or a video may be captured (either displayed or not displayed, may be stored in the memory or not), e.g., by a camera, thermal imager, fetched from a memory, received from an external device via a telecommunication unit, or via other means. The visual information may have the excreta to be analysed in it.

[0053]

[0038] Excreta may be waste from an animal body, e.g., any one of a baby, an adult, a person, a dog, a cat, and other animals. The excreta may comprise at least one of urine and stool.

[0054]

[0039] The visual information may be one of diaper visual information, nappy visual information, vessel visual information (e.g., bedpan, potty etc.), litter visual information, flushing toilet visual information, grass field visual information, ground visual information, etc, where excreta may be on or in.

[0055]

[0040] In step 102, it may be determined whether there are excreta in the visual information. The determination may be by a pretrained artificial intelligence (Al) model, or by an image / visual information recognition algorithm.

[0056]

[0041] If it is determined that there are no excreta in the visual information, then the method may stop. Or, an alert to the user may be outputted (via screen or speaker) to remind the user to capture or change the visual information, and then step 102 may be performed again with the new visual information. The determination step 102 may be omitted, e.g., it may be assumed that there are always excreta in the visual information.

[0057]

[0042] In step 102, if it is determined that there are excreta in the visual information, the method may further determine the composition of the excreta, e.g., whether the excreta are only with stool or urine, or with both. This determination of the composition may be omitted.

[0058]

[0043] In step 103, the method provides the visual information to an artificial intelligence (Al) model. The Al model may process the visual information and determine an excreta condition by processing the visual information in step 104. In some embodiments, determining the excreta condition includes performing excreta assessment to characterize the excreta, and a microbiome profile of the excreta based on the identified characteristics of the excreta. The microbiome may comprise at least one of Lachnospiraceae, Ruminococcaceae, Bifidobacteriaceae, Bifidobacterium, and lactobacilli.

[0059]

[0044] For example, if the excreta comprise stool, the identified characteristics of the excreta may comprise at least one of a colour scale and a consistency scale of the stool. The colour scales may include standard colour scales or a subset of standard colour scales. The consistency scales may include watery, soft, formed and hard based on the BSS or any other stool consistency standards.

[0060]

[0045] If the excreta comprise urine, the identified characteristics of the excreta may comprise a colour scale of the urine. The colour scales may include standard colour scales or a subset of standard colour scales.

[0061]

[0046] The Al model may be pretrained by training visual information (e.g., images), wherein the training visual information may be processed by a loss function. The loss function may comprise mixing two original visual information (e.g., images) together according to different transparency, generating mixing coefficients randomly for a plurality of times and letting the Al model learn according to a data distribution during the pre-training. Such a pre-training method can increase the prediction accuracy of the Al model. Furthermore, automated data augmentation may be performed to the training visual information before being used to train the Al model, such augmented training visual information can improve the prediction accuracy as well. The Al model may also be trained with a semi-supervised training method.

[0062]

[0047] Before processing the visual information using the Al model in step 104, the visual information may be pre-processed, e.g., with at least one of flipping, lightening, darkening, and cropping. Such pre-processes can help the Al model to predict the scales more accurately.

[0063]

[0048] In an example, step 104 may include determining an efficacy and / or impact on the excreta condition based on at least one of an existing diet, a change in diet, an existing lifestyle (e.g., exercise, sleep), a change in lifestyle, medications, a change in medication, and any combination thereof.

[0064]

[0049] In some embodiments, based on the identified characteristics of the excreta, associated with the excreta image, the Al model may determine an intervention to help alleviate any symptoms related to the excreta condition experienced by the first subject, and / or to help reduce the risk of the subject experiencing any symptoms related to the excreta condition.

[0065]

[0050] In step 105, the determined excreta condition by the Al model is outputted, e.g., displayed or with sounds to indicate the scales with the identified characteristics of the excreta. For example, the indication to a colour scale may be text, coloured image and / or coloured text corresponding to the colour scale. As examples, text “dark brown” may be displayed correspond to dark brown stool as determined by the Al model, or text “stool” with a dark brown colour is displayed, or text “dark brown” with a dark brown colour is displayed, or an overlying image with a dark brown colour. Similarly, the indication to a consistency scale may be texts or an overlaying image indicating the consistency scale.

[0066]

[0051] Augmented reality (AR) technologies may be used in the present invention. For example, when outputting the excreta condition, at least one virtual excreta overlaying object (e.g., a virtual excreta overlaying, an overlaying virtual stool image, virtual stool icon, a three-dimensional image, multimedia information, etc.) may be displayed, and a plurality of scales of the virtual excreta may be displayed according to the determined characteristics of the excreta. The virtual excreta (i.e., the virtual excreta overlaying object) may be displayed next to the excreta in the visual information according to the determined characteristics of the excreta.

[0067]

[0052] For example, if the excreta in the visual information includes stool, an overlaying virtual stool object / icon (e.g., a stool shaped icon, a stool image with a predetermined trenchancy level, a copied image of the identified stool, or the combination thereof) may be displayed next to the captured visual information, e.g., next to the identified stool. In this example, the overlaying virtual stool object / icon may be displayed according to the determined characteristics of the stool, i.e., the plurality of scales of the virtual excreta may be displayed according to the determined excreta condition.

[0068]

[0053] Taking colour scale as an example, if the determined colour scale of the stool in the visual information (e.g., image) is dark brown, the overlaying virtual stool object may be displayed in the corresponding dark brown colour. For consistency scale, if the determined consistency of the stool is wet, the overlaying virtual stool object may indicate that the stool is wet, e.g., include a water drop icon in the overlaying virtual stool object, or with smooth stool surface in the overlaying virtual stool object.

[0069]

[0054] The AR technology, especially with the virtual excreta overlaying object, may help the user (e.g., caregivers) to check the difference between scales of the excreta in the real visual information (i.e., the captured image that may be displayed) and the determined characteristics of the excreta by the Al model (via the virtual excreta overlaying object).

[0070]

[0055] In some embodiments, step 105 may further include determining and / or monitoring an excreta condition for the first subj ect, and for communicating to the subject and / or a healthcare provider (e.g., physician, nurse, or any other medical professional) the excreta condition, excreta assessments, preferred subject conditions for a stool condition in case that the excreta is a stool, and / or interventions based on the stool condition. With reference to FIG. 1, as described herein, the excreta image(s) (e.g., obtained via an image capture device) may be received by an analysing tool, which then determines a corresponding condition. In some embodiments, the excreta condition may be output onto a display interface (e.g., a monitor, screen, smart device screen, etc.).

[0071]

[0056] In some embodiments, the method of FIG.1 may be performed by an analysing tool which can be embodied as a smart phone (e.g., see FIG. 2, reference character 10). For example, in some embodiments, the analysing tool 120 may be configured to apply one or more artificial intelligence (“Al”) engines (e.g., trained models, decision trees, analytical expressions, etc.) so as to determine the excreta condition. In some embodiments, the one or more Al engines each apply an algorithm, such as a machine learning algorithm (as described herein), to the one or more excreta images obtained. In some embodiments, the image capture device and the analysing tool may be provided by the same electronic device (for e.g., same mobile device, laptop, etc.).

[0072]

[0057] In an embodiment the Al engine for determining microbiome profiles from pre-processed digital images of excreta is trained as follows. The training method involves the use of N sets of training data, where each set is denoted as n, ranging from 1 to N. Each set, n, contains M(n) examples of pre-processed digital images of excreta. These images are labelled with corresponding microbiome profiles, providing a comprehensive dataset for training the Al engine. The images in the training set should have various backgrounds that can occur during training, e.g. diaper, bed pan, toilet, etc. That way, the algorithm will learn to ignore the surroundings in the image, and focus on the stool image data itself.

[0073]

[0058] In a further embodiment of this method, the training set uses multi-labels. That is, each example within M(n) is not labelled with a singular, definitive microbiome profile. Instead, they are labelled with multiple microbiome profiles, reflecting the complex and varied nature of the microbiota. For instance, an image could be labelled as 20% microbiome X, 70% microbiome Y, and 10% microbiome Z. This multi -label approach allows the Al engine to learn the nuances and variations within the microbiome profiles, leading to more accurate and detailed analysis.

[0074]

[0059] The Al engine employed in this method is based on deep learning algorithms. Suitable algorithms for this task include Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Deep Belief Networks (DBNs). Each of these algorithms has characteristics that make them suitable for processing and learning from image data and sequential patterns in data.

[0075]

[0060] The training process involves several stages, starting with an optional preprocessing step of the excreta images to enhance features relevant for microbiome profiling. Following this, the images are fed into the Al engine, where they undergo multiple iterations of learning. During these iterations, the engine adjusts its parameters to minimize the error in predicting the microbiome profiles. In an embodiment, the multi-label approach is integrated into the training, allowing the engine to understand and predict the proportional composition of different microbiomes in a single image.

[0076]

[0061] In optimizing the Al engine's performance, techniques like cross- validation, regularization, and hyperparameter tuning are employed. These techniques prevent overfitting and ensure the model generalizes well to new, unseen data.

[0077]

[0062] The output of this trained Al engine may be a set of probabilities corresponding to each microbiome profile, indicating the likelihood of each microbiome's presence in the analysed excreta sample. This training method results in an Al engine that allows for a nuanced and detailed understanding of the microbiome composition.

[0078]

[0063] Example models that can be employed in this method are: AlexNet, VGGNet, ResNet, and Inception (GoogleNet). More recently, the following models have been introduced: EfficientNet-B7, ViT-Large / 16, Swin Transformer Large.

[0079]

[0064] In a variant of this method, a pre-trained model is fine-tuned for the specific task of determining microbiome profiles from images of excreta. This approach involves taking a model that has been pre-trained on a large, diverse dataset (such as ImageNet) and then further training it (fine-tuning) on the more specialized dataset of preprocessed excreta images. This fine-tuning process allows the model to adapt its already learned features to the specific characteristics and nuances of the new dataset, thereby improving its accuracy and efficiency in the target task.

[0080]

[0065] More information about the models can be found in the following papers:

[0081]

[0066] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). "ImageNet Classification with Deep Convolutional Neural Networks". This paper introduces the AlexNet model.

[0082]

[0067] Simonyan, K., & Zisserman, A. (2014). "Very Deep Convolutional Networks for Large-Scale Image Recognition". This paper provides insights into the architecture of VGGNet.

[0083]

[0068] He, K., Zhang, X., Ren, S., & Sun, J. (2016). "Deep Residual Learning for Image Recognition". This paper details the development of ResNet.

[0084]

[0069] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. (2015). "Going Deeper with Convolutions". This paper discusses the Inception model.

[0085]

[0070] Tan, M., & Le, Q. (2019). "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks" introduces EfficientNet.

[0086]

[0071] Dosovitskiy, A., et al. (2020). "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale" describes the Vision Transformer architecture.

[0087]

[0072] Finally, Liu, Z., et al. (2021). "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" describes the Swin Transformer.

[0088]

[0073] In some embodiments, the electronic device may be in operative communication with a remote computing device (including a remote server). In some embodiments, subjective, self-assessments of excreta characterization may result in inconsistent and / or inaccurate determinations of the excreta condition. Accordingly, using an Al engine helps increase the accuracy and consistency in determining the excreta condition, and microbiome status of the excreta, as described herein.

[0089]

[0074] In some embodiments, the microbiome profile of the excreta is determined based on a plurality of characteristics identified with the excreta in the image(s) 110. In some embodiments, the plurality of characteristics of the excreta comprise at least one of colour, consistency, and texture.

[0090]

[0075] According to some embodiment, the Al engine may assess the excreta condition by using the trained machine learning model. In that way, said assessment for the excreta condition 130 may determine a health condition of the first subject based on the determined microbiome profile of the excreta, and data indicating at least one health suggestion for the subject can be output based on the determined health condition of the first subject.

[0091]

[0076] The health condition of the first subject may be determined based on estimating the abundance of at least one microbiomes based on the determined microbiome profile of the excreta. For example, while the abundance of Enterob acteriaceae may result in loose consistency of the excreta, which means a risk of illness, the abundance of Lachnospiraceae may increase the consistency of the excreta.

[0092]

[0077] Therefore, in an exemplary embodiment, the at least one health suggestion may comprise the risk of illness based on the determined microbiome profile of the excreta. In that way, the healthcare professionals can have efficient information for deciding the most appropriate treatment for the subject.

[0093]

[0078] In an exemplary embodiment, the Al engine may receive information about the first subject, for increasing the accuracy and the efficiency of the health suggestion. Said information may comprise a dietary status of the first subject and at least one of gender, age, disease record, body weight, and body length of the subject. Accordingly, the data indicating at least one health suggestion for the subject may be output based on the determined health condition and the received information about the first subject.

[0094]

[0079] The data indicating the health suggestion may comprise at least one nutrition guidance for improving consistency of the first excreta and at least one immunity system indicator.

[0095]

[0080] For example, in case that the first subject is a baby / an infant that is breastfed from a female subject, the health suggestion may comprise a nutrition guidance for the female subject.

[0096]

[0081] In some embodiments, the Al engine may store information on the identified characteristics of the excreta that has been assessed by the analysing tool 120, for re-training the trained machine learning model.

[0097]

[0082] The analysing tool can predict selected microbes in the excreta image and output relevant information to caregivers and healthcare professionals on managing excreta microbiology. This may allow to control and monitor a feeding behaviour of the first subject. For example, in case that the subject is an infant, with the result of the assessment of the excreta condition, appropriate product / formula and recipe can be provided to caregivers to educate / enhance the microbiome profile of the first subject.

[0098]

[0083] In some embodiments, the Al engine and / or tool may receive another visual information, e.g. image, including excreta of a second subject. In that case, the analysing tool may use the Al engine for comparing the image from the first subject which has been assessed for identifying characteristics of its excreta, and the new image from the additional subject. For determining the stool condition of the second subject, the analysing tool may determine a similarity level of these two images.

[0099]

[0084] According to some embodiments, these two subjects may be fed with different nutritional compositions.

[0100]

[0085] In such case, the Al engine of the analysing tool may apply the trained machine learning model which has already saved the identified characteristic of the first subject for identifying at least a part of characteristics of the excreta from the additional subject based on the similarity level. This will reduce the operation to be performed by the Al engine for determining a microbiome status of the excreta from the additional subject, since it can be determined based on the identified part of the characteristics of the excreta of the second subject.

[0086] In some embodiments, the Al engine may identify a first part of the characteristics of the excreta of the additional subject based on the similarity level, and continue to perform series of operations for identifying at least one region of interest within the image of said excreta, which contains different features than the features of the previous image of the previous excreta of the first subject. Accordingly, the Al engine may apply the trained machine learning model to the determined at least one region of interest in order to identify a second part of the characteristics of said excreta. In that way, the Al engine may determine the microbiome status of the new excreta more accurately, based on both the first and the second part of the identified characteristics of said excreta. This method will reduce the burden on the analysing tool by performing less series of operations and will allow to output the result of the assessment on the excreta condition within a shorter time.

[0101]

[0087] Fig. 2 depicts an overview of an embodiment of the present invention. Although embodiments of the present invention comprise excreta being placed in a diaper, a potty, a bed pan, a toilet chair, a toilet with plateaus, or other similar locations, in the embodiment of Fig. 2 , an example is depicted in which the excreta is placed in a diaper.

[0102]

[0088] In the exemplary embodiment of Fig. 2, a diaper 20 comprising infant stool 30 is placed in an open position on a surface 40, and an image capturing device implemented as the electronic device 10 is used by a user, to capture a visual information e.g. an image of the open diaper 20 with the stool 30. The captured image (e.g. the image 110 of Figure 1) can be used by an app running on the electronic device, and the app may input the captured image to the Al engine which will perform a series of operations in order to identify at least one region of interest within the captured image and apply a trained machine learning model for identifying characteristics of the stool, and in order to determine the microbiome profile of the stool based on the identified characteristics of the stool.

[0103]

[0089] In order to obtain the best results from the classification process, it is desirable that the captured image has good characteristics, and therefore, that certain predetermined conditions are met. As an example of the conditions to be met, the stool 30 should be as recent as possible, so that its properties (colour, consistency) have not yet changed due to for instance that a part of the stool is absorbed by the diaper. The image should hence be captured shortly after the stool has been deposited, and in the embodiment of Fig. 2 , shortly after the diaper 20 has been filled with stool, and a suitable period to capture the image is up to ten minutes from the time the diaper has been filled. The app of the electronic device may automatically detect the colour of the stool from the captured image, and analyse it in order to provide information. This allows to also provide information related to whether the colour is normal or not, which together with the consistency, can allow for a better determination of possible anomalies.

[0104]

[0090] Another example of a condition which should be met is that the image is captured with sufficient light to clearly distinguish the features in the image, but not with too much light as this may modify the real colour and appearance of the image features. An example of how this can be achieved is by using natural light (daylight) or light from a ceiling lamp. Using the flash of the camera is however less preferred, as it could change the appearance of the image.

[0105]

[0091] Another example of a condition is that the background of the image should preferably be regular, such as following a pattern, or uniform. If the stool is placed in a potty, a bed bad a toilet chair, or a toilet with plateaus, it is desirable that the background is uniform. If the stool is placed in a diaper, as represented in the embodiment of Fig. 2, it is preferable that the diaper 20 is in the foreground of the image. A surface 40 of a table which is uniform is an example of a suitable background.

[0106]

[0092] Another example of a condition that should be met is that no other object should be present in the image. If the stool is placed in a potty, a bedpan, a toilet chair, or a toilet with plateaus, it is desirable that only the stool and the background are present in the image, no other objects or parts of a body. If the stool is placed on a diaper, it is desirable that no other object other than the diaper 20 containing the stool 30 is present. Parts of the baby’s body, or other objects, should not be present in the image. If, however, the captured image has undesirable objects surrounding the stool, it may be possible to cut the captured image to eliminate the undesirable objects before the image is input to the Al engine. Other pre-processing steps may be performed such as modifying the resolution of the captured image, changing the format of the image, or other steps to remove noise in the image.

[0107]

[0093] It should be noted that embodiments of the invention may use only one of the predetermined conditions, or any combination of them. It should also be noted that other predetermined conditions can be used, as long as they help the user obtain an image which is suitable for being input to the Al engine. Which predetermined conditions are to be used may be determined and modified in the settings of the app, or may be predefined, and a controller of the portable device may control which predetermined conditions are used, and how they are determined.

[0108]

[0094] Other examples of conditions that should be met can be seen in Figs. 3a and 3b, which show different dispositions of a diaper comprising a stool according to the present invention. Again, the example of a stool in a diaper is represented, but the skilled person will understand that the features may similarly be applied to other embodiments in which the stool is provided in a potty, a bed bad, a toilet chair, a toilet with plateaus, or the like. In Fig. 3a, an open diaper 20 comprising stool 30 can be seen, with a uniform background 40, which in this case corresponds to the top of a uniform-surface table. In Fig. 3a, no other objects other than the diaper are present in the image. An image like the one in Fig. 3a could be considered suitable for the method of the present invention.

[0109]

[0095] According to an embodiment, the portable device 10, before capturing the image, for example before pointing the camera to the stool, or when the user is pointing the camera to the stool, may give indications to the user to remind him / her of some conditions that should be met ( the light should be enough, and the like). These conditions may be some or all of the predetermined conditions described above, or different conditions. The portable device may additionally guide the user in order to bring the portable device 10 closer or further to the stool, in Fig. 3a, in the diaper 20, to change the angle or the light source in the room in order to achieve better light in the image, and so on, and provide an indication of when the image could be suitable. This may allow the user to capture an image which has good characteristics and will improve the success rate of the classification. According to an embodiment, the electronic device 10 may automatically capture the image when it detects that the electronic device 10 is located at a suitable distance from the stool, or the diaper 20, the light conditions are suitable, or when any other desirable condition is met. The condition or conditions to be met in order for the portable device to decide to automatically capture the image may be previously defined.

[0110]

[0096] In Fig. 3b, the diaper 20 is not in a suitable position for capturing the image, as the diaper is not completely open and hence the stool 30 cannot be sufficiently identified. According to an embodiment of the invention, by seeing the guidance displayed in the portable device the user can be made aware that this is an unsuitable position and can open the diaper. According to another embodiment, the app will provide the guidance before pointing the camera to the stool, and the user will be aware that for example the diaper needs to be open, so that an image like the one represented in Fig. 3b is not captured.

[0111]

[0097] In some embodiments, the Al engine may apply the trained machine learning model for adjusting the captured image in accordance with the above mentioned requirements. More specifically, the Al engine may pre-process the captured image of the stool by at least one of flipping, adjusting lightening, and resizing the captured image of the stool, in accordance with the above mentioned requirements.

[0112]

[0098] According to an embodiment, the Al engine may apply the trained machine learning model for outputting an image of at least one feature extracted from the captured image of the stool and a scale information on the characteristics of said feature of the captured image of the stool, based on the determined region of interest. The scale information comprises a colour scale and a consistency scale of the at least one feature extracted from the captured image of the stool.

[0113]

[0099] In an exemplary embodiment, in the case that the excreta comprises urine (not shown in Fig. 2), the scale information may comprise a colour scale of the at least one feature extracted from the visual information of the urine.

[0114]

[0100] In some embodiments, the app running on the electronic device 10 may receive an input from the user for adjusting the characteristics of said feature that is identified and output by the Al engine. Based on said input, the scale information of said feature of the stool may be adjusted and output.

[0115]

[0101] The adjusted scale information of said feature of the stool can be stored in a memory of the Al engine, for re-training the trained machine learning model.

[0116]

[0102] Figure 4 schematically shows the electronic device 10 according to an embodiment of the present invention. The electronic device 10 has a display unit 501, which may be touch screen suitable for displaying information and handling user input. The device 10 further has a camera 502 for recording the visual information of the excreta, a processor 503 for processing recorded visual information, a memory 504 for storing the visual information, program data, the Al engine, and the like, and a communication unit 505 for communication with other devices over wired or wireless connections. In an embodiment the processor 503 is programmed to process recorded images using the Al engine and to generally implement the processes as described in this application.

[0117]

[0103] Figure 5 schematically shows the electronic device 10 and a server 100 according to an embodiment of the present invention. The electronic device 10 and server 100 can communicate over a wired or wireless link. In an embodiment, the device 10 sends recorded visual information to the server. The server has a processor, a memory, and a communication unit. The server 100 can be programmed to process received visual information using the Al engine and to send back the results to the device 10. In addition, the server may store the obtained results and / or the received images and / or any intermediate calculation results. The server may be further arranged to implement the afore mentioned trained machine learning method.

[0118]

[0104] Figure 6A is a graph showing spectrograms that were recorded from stool samples. In particular, the graph shows the presence and the abundance of Enterob acteriaceae associated with the colour of recorded stool samples, in an experiment. In this experiment a number of 104 stool samples are recorded.

[0119]

[0105] In shown Figure 6A, ratio 1 corresponds the abundance distribution of Enterob acteriaceae associated with the brown tone colours, ratio 2 corresponds the merged distribution of the abundance of Enterobacteriaceae associated with the brown tone colours, ratio 3 corresponds the abundance distribution of Enterobacteriaceae associated with the yellow tone colours, and ratio 4 corresponds merged distribution of the abundance of Enterobacteriaceae associated with the yellow tone colours.

[0120]

[0106] The results shown in Figure 6A, indicate that the abundance of Enterobacteriaceae is increased when the colour of the stool sample has yellow tones.

[0107] Figure 6B is a graph showing spectrograms that were recorded from stool samples. In particular, the graph shows the presence and the abundance of Enterobacteriaceae associated with the consistency of recorded stool samples, in an experiment. In this experiment a number of 104 stool samples are recorded.

[0121]

[0108] In shown Figure 6B, ratio 1 corresponds the abundance distribution of Enterobacteriaceae associated with formed stools, ratio 2 corresponds the merged distribution of the abundance of Enterobacteriaceae associated with the formed stools, ratio 3 corresponds the abundance distribution of Enterobacteriaceae associated with loose stools, and ratio 4 corresponds merged distribution of the abundance of Enterobacteriaceae associated with the loose stools.

[0122]

[0109] The results shown in Figure 6B, indicate that the abundance of Enterobacteriaceae is increased when the stool has a loose consistency.

[0123]

[0110] Figure 7 is a graph showing spectrograms that were recorded from stool samples. In particular, the graph shows the presence and the abundance of lachnospiraceae associated with the consistency of recorded stool samples, in an experiment. In this experiment a number of 104 stool samples are recorded.

[0124]

[0111] In shown Figure 7, ratio 1 corresponds the abundance distribution of lachnospiraceae associated with formed stools, ratio 2 corresponds the merged distribution of the abundance of lachnospiraceae associated with the formed stools, ratio 3 corresponds the abundance distribution of lachnospiraceae associated with loose stools, and ratio 4 corresponds merged distribution of the abundance of lachnospiraceae associated with the loose stools.

[0125]

[0112] The results shown in Figure 7, indicate that the abundance of lachnospiraceae is relatively increased when the stool has a formed consistency.

[0126]

[0113] Figure 8 is a graph showing spectrograms that were recorded from stool samples. In particular, the graph shows the presence and the abundance of ruminococcaceae associated with the consistency of recorded stool samples, in an experiment. In this experiment a number of 104 stool samples are recorded.

[0127]

[0114] In shown Figure 8, ratio 1 corresponds the abundance distribution of ruminococcaceae associated with hard stools, ratio 2 corresponds the abundance distribution of ruminococcaceae associated with formed stools, ratio 3 corresponds the abundance distribution of ruminococcaceae associated with loose stools, and ratio 4 corresponds the abundance distribution of ruminococcaceae associated with watery stools.

[0128]

[0115] The results shown in Figure 8, indicate that the abundance of ruminococcaceae is increased when the stool has a formed consistency.

[0129]

[0116] Various other embodiments of the invention will be apparent to the skilled person when having read the above disclosure in connection with the drawings, all of which are within the scope of the invention and accompanying claims.

Claims

CLAIMS1. A computer implemented method of analysing at least one excreta, comprising the steps of;- receiving a first visual information of a first excreta of a first subject;- identifying at least one region of interest within the first visual information;- applying a trained machine learning model to the identified at least one region of interest in order to identify characteristics of the first excreta; and- determining a microbiome profile of the first excreta based on the identified characteristics of the first excreta, wherein said characteristics comprise at least one of colour, consistency, and texture.

2. The method according to any one of the preceding claims further comprising the steps of;- determining a health condition of the first subject based on the determined microbiome profile of the first excreta; and- outputting data indicating at least one health suggestion for the first subject, based on the determined health condition.

3. The method according to claim 2, further comprising the steps of: receiving information about the first subject, wherein said information comprises a dietary status of the first subject and at least one of gender, age, disease record, body weight, and body length of the first subject, using said information about the first subject in the determining the health condition of the first subject.

4. The method according to claim 2, wherein the health suggestion comprises at least one nutrition guidance for improving consistency of the first excreta and / or at least one immunity system indicator.

5. The method according to claim 4, wherein the first subject is a baby that is breastfed from a female subject, and wherein the health suggestion further comprises a nutrition guidance for the female subject.

6. The method according to any one of the preceding claims, wherein the microbiome comprises at least one of Enterobacteriaceae, Lachnospiraceae, Ruminococcaceae, Bifidobacteriaceae, Bifidobacterium, and lactobacilli.

7. The method according to any one of the preceding claims, wherein the identifying at least one region of interest within the first visual information comprises the step of;- pre-processing the first visual information by at least one of flipping, adjusting lightening, and resizing the first visual information.

8. The method according to any one of the preceding claims, wherein the applying a trained machine learning model to the determined at least one region of interest in order to identify characteristics of the first excreta comprises the steps of;- outputting an image of at least one feature extracted from the first visual information and a scale information on the characteristics of said at least one feature, based on the determined at least one region of interest.

9. The method according to claim 4 further comprising the steps of;- receiving an input for adjusting the characteristics of said at least one feature;- adjusting the scale information and said image of the at least one feature; and- outputting the adjusted scale information and the adjusted image of the at least one feature.

10. The method according to claim 9, wherein the adjusted scale information is stored to train the trained machine learning model.

11. The method according to any one of claims 7 to 9; wherein the first excreta comprises stool of the first subject, and wherein the scale information comprises a colour scale and a consistency scale of the at least one feature extracted from the first visual information of the stool.

12. The method according to any one of claims 7 to 9, wherein the first excreta comprises urine, and wherein the scale information comprises a colour scale of the at least one feature extracted from the first visual information of the urine.

13. The method according to any one of the preceding claims, further comprising the step of; storing information on the identified characteristics of the first excreta.

14. The method according to any one of the preceding claims, further comprising the steps of;- receiving a second visual information of a second excreta of a second subj ect,- comparing the first visual information and the second visual information; and- determining a similarity level of the first visual information and the second visual information.

15. The method according to claim 14, wherein the first subj ect is fed with a first nutritional composition, and the second subject is fed with a second nutritional composition.

16. The method according to any one of claims 14 and 15 further comprising the steps of;- identifying at least a part of characteristics of the second excreta based on the similarity level; and- determining a microbiome profile of the second excreta based on the identified part of the characteristics of the second excreta.

17. The method according to claim 16, wherein the identifying at least a part of characteristics of the second excreta based on the similarity level, comprises;- identifying a first part of the characteristics of the second excreta based on the similarity level;- identifying at least one region of interest within the second visualinformation; and- applying the trained machine learning model to the determined at least one region of interest in order to identify a second part of the characteristics of the second excreta.

18. The method according to any one of the preceding claims, wherein each of the first and the second visual information comprises at least one of a video data, an image data, a three dimensional data and a thermal image data.

19. A non-transitory computer readable medium for analysing at least one excreta, the non-transitory computer readable medium comprising instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 18.

20. A system (100) for analysing at least one excreta, the system comprising: one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the system to perform operations the method according to any one of claims 1 to 18.

Images