Evaluation system, information processing device, evaluation method, and control program

Abstract

To provide a technique that can objectively evaluate a way to eat or feed a baby food.SOLUTION: An evaluation system (100) comprises: a monitoring device (3) for monitoring an infant eating a baby food; an information processing device (1) for evaluating or determining a way of the infant to eat the baby food or a way to feed the infant with the baby food by using data obtained by the monitoring; and an output device (2) for outputting feedback information generated based on a result of the evaluation or determination.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an evaluation system, an information processing device, an evaluation method, and a control program. [Background technology]

[0002] Conventionally, attempts have been made to obtain useful knowledge by analyzing the human act of eating. For example, Patent Document 1 listed below discloses a technique for quantitatively estimating the food texture of a sample by measuring the myoelectric potential when a subject chews a sample and analyzing the measured myoelectric potential. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-52516 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional technologies are aimed at adults, and there was no technology to analyze and objectively evaluate how infants eat and feed solid food. As a result, even if there was room for improvement in feeding methods, there was no opportunity to notice this. Furthermore, solid food is introduced in stages as infants grow physically and their eating functions improve, but many people were unsure whether to proceed to the next stage.

[0005] One aspect of the present invention has been made in consideration of the above-mentioned problems, and one example of its purpose is to provide technology that can objectively evaluate how a baby eats and feeds baby food. [Means for solving the problem]

[0006] In order to solve the above problems, an evaluation system according to one aspect of the present invention includes a monitoring device that monitors an infant eating baby food, an information processing device that uses data obtained from the monitoring to evaluate or judge how the infant eats or is being fed, and an output device that outputs feedback information generated based on the results of the evaluation or judgment.

[0007] In order to solve the above problem, an information processing device according to one aspect of the present invention includes a monitoring device that monitors an infant eating baby food, a control device that uses data obtained from the monitoring to evaluate or judge how the infant eats or is being fed, and an output device that outputs feedback information generated based on the results of the evaluation or judgment.

[0008] In order to solve the above problems, an evaluation method according to one aspect of the present invention is an evaluation method executed by an evaluation system including a monitoring device, an information processing device, and an output device, and includes a monitoring step in which the monitoring device monitors an infant eating baby food, an evaluation and judgment step in which the information processing device evaluates or judges the way the infant eats or is being fed using data obtained by the monitoring, and an output step in which the output device outputs feedback information generated based on the results of the evaluation or judgment.

[0009] The information processing device according to each aspect of the present invention may be realized by a computer. In this case, the control program of the information processing device that causes the computer to operate as each part (software element) of the information processing device to realize the information processing device on the computer, and the computer-readable recording medium on which the control program is recorded, also fall within the scope of the present invention. [Effects of the Invention]

[0010] According to one aspect of the present invention, it is possible to objectively evaluate how a baby eats and feeds baby food. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a main part of an information processing device according to the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an overview of an evaluation system according to the present disclosure. [Figure 3] 10A and 10B are diagrams showing a specific example of an image and pattern information obtained when feeding is not appropriate. [Figure 4] 10A and 10B are diagrams showing a specific example of an image and pattern information obtained when the feeding method is appropriate. [Figure 5] 5A and 5B are diagrams showing a specific example of feedback information presented to a user in the display device according to the first embodiment. [Figure 6] 4 is a flowchart showing an example of an evaluation method executed by the evaluation system according to the first embodiment. [Figure 7] 10 is a diagram showing a specific example of detected feature points and analysis result information of mastication movements obtained based on the feature points. FIG. [Figure 8] FIG. 10 is a diagram showing a specific example of feedback information presented to a user in the display device according to the second embodiment. [Figure 9] 10 is a flowchart showing an example of an evaluation method executed by the evaluation system according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] [Embodiment 1] <Summary> 2 is a diagram showing an overview of the evaluation system according to the present embodiment. The evaluation system 100 according to the first embodiment is a system for evaluating at least one of how an infant eats baby food and how a server who provides the baby food feeds the infant.

[0013] The evaluation system 100 of the present disclosure includes a monitoring device that monitors infants and young children eating baby food, an information processing device that uses data obtained from the monitoring to evaluate or judge how the infants and young children eat or are fed, and an output device that outputs feedback information generated based on the results of the evaluation or judgment.

[0014] In the illustrated example, the above-mentioned information processing device is shown as an information processing device 1, which may be realized as a smartphone, for example. In the illustrated example, the above-mentioned monitoring device is shown as a photographing device 3 that photographs an infant being fed and eating baby food, which may be realized as a camera included in the information processing device 1, which is a smartphone, for example. In the illustrated example, the photographing device 3 is provided on the back of the information processing device 1. In the illustrated example, the above-mentioned output device is shown as a display device 2 that displays and outputs feedback information as visualized information, which may be realized as a liquid crystal display panel or an organic EL display panel integrated with a touch panel, which is included in the information processing device 1, which is a smartphone, for example. That is, in the illustrated example, one information processing device 1 functions as the evaluation system 100 of the present disclosure.

[0015] 2 shows an example in which the information processing device 1 is a smartphone, but any one or more computers can function as the information processing device 1. For example, the information processing device 1 may be realized by a tablet terminal, a personal computer, a laptop PC, a server device, cloud computing, or the like.

[0016] 2 shows an example in which the evaluation system 100 is realized by a single information processing device 1 (smartphone), but the evaluation system 100 can be configured by combining any number of devices. For example, the evaluation system 100 may be configured to include an imaging device 3 as a monitoring device, the information processing device 1, and a display device 2 as an output device. In this case, the imaging device 3 may be a video camera for capturing moving images that is separate from the information processing device 1. The display device 2 may be a liquid crystal display device or an organic EL display device that is separate from the information processing device 1.

[0017] Hereinafter, the subject to be observed to evaluate how an infant eats baby food or how an infant is fed baby food will be referred to as a subject. In this embodiment, the subject is an infant eating baby food.

[0018] In this embodiment, for example, the evaluation system 100 evaluates how an infant eats or is fed as follows.

[0019] A server (for example, a caregiver such as a mother, a nursery teacher, a nurse, etc.) feeds baby food to a subject, and the subject eats the baby food. A user photographs the waiter feeding the baby food and the subject eating the baby food with the photographing device 3 of the information processing device 1 (a monitoring step of monitoring the infant eating the baby food). The user refers to a person who uses the evaluation system 100 to obtain an evaluation of how an infant eats or is fed, and the user may be the server himself or herself, or someone other than the server. The other person may be, for example, an instructor who instructs the server on how to feed the baby food. The server may take the photographs by, for example, fixing his or her own information processing device 1 to a tripod, or someone other than the server may take the photographs.

[0020] The information processing device 1 obtains time-series images obtained by photography as data obtained by monitoring. The information processing device 1 evaluates or judges how the infant eats or is fed using the time-series images showing the subject's eating behavior (evaluation and judgment step).

[0021] The information processing device 1 causes the output device to output the feedback information generated based on the result of the evaluation or judgment (output step). For example, the feedback information may be displayed on the display device 2.

[0022] According to the above-described configuration and method, an infant eating baby food is monitored and the data obtained from the monitoring is used to evaluate how the infant eats or is fed the baby food, thereby enabling an objective evaluation of how the infant eats or is fed the baby food. Furthermore, the objective evaluation results can be presented to the server or a user other than the server. Therefore, if there is room for improvement in the server's feeding method, the server or a user other than the server can be made aware of this, motivating the server to improve the feeding method.

[0023] For example, the display device 2 of the information processing device 1 shown in Figure 2 displays, as feedback information (hereinafter referred to as FB (Feedback) information as appropriate), an image 161 symbolizing the results of the evaluation performed by the information processing device 1 and a message 162 for the user regarding how to eat or feed infants.

[0024] By displaying image 161, the user (e.g., the server) can intuitively recognize from the image whether the way the subject is eating the baby food or the way the server is feeding the baby is appropriate. In this way, the user has the opportunity to notice if there is room for improvement in the way the baby is being fed, and is motivated to improve the way they feed the baby.

[0025] Message 162 may also include a sentence explaining the subject's current eating habits and a sentence explaining how the waiter should improve the way he or she feeds the baby. Message 162 can specifically communicate to the user measures to improve the subject's eating of baby food or the waiter's feeding habits. Therefore, the waiter is motivated to improve the way he or she feeds the baby, just like image 161. Furthermore, message 162 allows the user to understand specifically how to improve, making message 162 even more useful information for the user.

[0026] 2 is merely an example, and the content of the FB information is arbitrary as long as it makes the user aware that there is room for improvement in the feeding method. Furthermore, feedback to the user is not limited to visual feedback, but may be provided by sound, vibration, or a combination thereof. Feedback may also be provided via another device, such as by displaying the FB information on a display device external to the information processing device 1.

[0027] <Configuration of information processing device> FIG. 1 is a diagram illustrating an example of a configuration of essential parts of an information processing device 1 according to the present disclosure. As illustrated, the information processing device 1 includes a control unit 10 (control device) that controls all parts of the information processing device 1, a storage unit 11 that stores various data used by the information processing device 1, a photographing device 3 (monitoring device), and a display device 2 (output device). The photographing device 3 is a monitoring device that monitors an infant eating baby food, and in this embodiment, for example, is a device that photographs an infant eating baby food. The control unit 10 is a control device that uses data obtained by monitoring to evaluate or judge how the infant eats or is being fed. The display device 2 is an output device that outputs FB information generated based on the results of the evaluation or judgment by the control unit 10, and in this embodiment, for example, is a device that presents the FB information to a user as visualized information.

[0028] In this embodiment, the image capturing device 3 may be a device that captures images with a single eye, or may be a device that captures multiple images of the same subject at the same time from multiple viewpoints. Multiple image capturing devices 3 may be provided to capture multiple images of the same subject at the same time from multiple viewpoints. Alternatively, various well-known types of 3D cameras, such as infrared (IR) sensor type, stereo type, ToF (Time Of Flight) type, and structured lighting type, may be used as the image capturing device 3.

[0029] Although not shown, the information processing device 1 may include a communication unit that enables the information processing device 1 to communicate with other devices, and a sound output unit such as a speaker that outputs sound. Furthermore, the information processing device 1 may include an input unit that accepts input of various data to the information processing device 1. Note that the functions of the input unit and the display device 2 may be realized by a touch panel.

[0030] The control unit 10 also includes an image acquisition unit 101, a detection unit 102, an evaluation / determination unit 103, and an output control unit 105. These processing units in the control unit 10 can also be realized by, for example, installing a control program in the information processing device 1 to cause the information processing device 1 to function as these processing units. The storage unit 11 also stores an image 111, action information 112, result information 113, and FB information 114.

[0031] The image acquisition unit 101 acquires images captured in chronological order of scenes in which the subject eats. Scenes in which the subject eats include the subject's action of putting baby food into their mouth (hereinafter referred to as "eating action") and the action of moving various oral organs from the time the subject puts the baby food in their mouth until swallowing (hereinafter referred to as "chewing action"). It is assumed that in the early stages of baby food intake, infants swallow the baby food by moving various oral organs such as the upper lip, lower lip, tongue, jaw, and muscles of mastication, without "crushing" the baby food. Hereinafter, the action of moving various oral organs from the time the baby puts the food in their mouth until swallowing will be collectively referred to as "chewing action," regardless of whether or not such "crushing" action is performed.

[0032] In this embodiment, the image acquisition unit 101 acquires images captured by the image capture device 3 built into the information processing device 1. In another example, the image acquisition unit 101 may receive images from the separate image capture device 3 by communicating with the image capture device 3 via a communication unit (not shown). Alternatively, the image acquisition unit 101 may acquire images stored in a removable external memory or the like in the information processing device 1 via a terminal (not shown).

[0033] Furthermore, the image acquisition unit 101 stores the acquired images as images 111 in the storage unit 11. The images 111 may be moving images or time-series still images captured continuously. The image acquisition unit 101 may also acquire moving images and store frame images extracted from the moving images as images 111.

[0034] The detection unit 102 analyzes images 111 taken in time series of scenes in which the subject eats, and detects a particular behavior of the subject while eating. In this embodiment, the detection unit 102 analyzes the images to detect an eating behavior performed when the subject puts baby food in their mouth.

[0035] The detection unit 102 stores the detection result of the movement in the storage unit 11 as the movement information 112. In this embodiment, the detection unit 102 stores the detection result of the eating movement in the storage unit 11 as the movement information 112.

[0036] For example, the detection unit 102 may analyze the image 111 to identify a pattern of change over time in the movements of the head and mouth of an infant eating baby food. More specifically, the detection unit 102 may quantify movements related to the subject's eating behavior, detect the subject's eating behavior, and generate pattern information indicating a pattern of the eating behavior, which may be stored in the storage unit 11 as movement information 112. The pattern information may include, for example, movements related to the eating behavior; (1) The amount of forward and backward movement of the subject's face, (2) the pitch of the subject's head (an angle for expressing the forward or backward tilt of the head, which may be expressed, for example, as Euler angles); and (3) Opening and closing of the subject's mouth The information may include numerical information.

[0037] The evaluation / determination unit 103 evaluates or determines how the infant eats or is being fed, based on the behavior information 112 generated by the detection unit 102. In this embodiment, the evaluation / determination unit 103 may evaluate how the infant is being fed baby food, for example, based on the difference between the pattern of the subject's eating behavior identified by the detection unit 102 and a standard pattern of changes over time in the eating behavior when the infant is being fed in an appropriate manner.

[0038] For example, the evaluation and determination unit 103 compares the subject's pattern indicated by the pattern information generated by the detection unit 102 with a standard pattern. If the comparison result shows that the difference between the two patterns is smaller than a predetermined value, the evaluation and determination unit 103 evaluates that the way the subject eats or feeds the baby food is appropriate, and if the difference is larger than a predetermined value, it can evaluate that there is room for improvement in the way the subject eats or feeds the baby food. The evaluation and determination unit 103 stores the evaluation result, which indicates whether the way the subject eats or feeds the baby food is appropriate, in the memory unit 11 as result information 113.

[0039] The method for generating the standard pattern and the method for determining which pattern to use as the standard pattern are arbitrary. For example, the storage unit 11 may store a database that stores, as standard patterns, typical patterns of changes over time in eating behavior when the feeding method is performed appropriately, which have been prepared in advance. The evaluation determination unit 103 may evaluate the appropriateness of the feeding method by comparing it with the standard patterns in the database.

[0040] In another example, a pattern of changes over time in feeding behavior when a subject infant or toddler is able to eat properly may be stored in a database as a standard pattern for that subject. When evaluating the subject's feeding method, the evaluation determination unit 103 may evaluate the appropriateness of the subject's feeding method by comparing it with the standard pattern associated with that subject.

[0041] In yet another example, multiple standard patterns may be used when evaluating a subject. For example, by observing the subject's eating habits multiple times, several patterns of changes in the subject's eating behavior over time are accumulated. Therefore, in the first to several evaluations, the evaluation and determination unit 103 may use a standard pattern generated based on a typical pattern as a comparison target. Next, the information processing device 1 may learn the standard pattern of the subject from the multiple patterns of the subject obtained through these several observations, and generate a standard pattern for the subject. Then, in evaluations after a predetermined number of times, the evaluation and determination unit 103 may use the learned standard pattern of the subject as a comparison target.

[0042] The output control unit 105 outputs feedback information generated based on the result information 113. In this embodiment, the output control unit 105 generates FB information 114 for providing feedback to the user about how to eat or feed baby food in accordance with the result information 113 as visualized information to be displayed on the display device 2, and outputs the FB information 114 to the display device 2.

[0043] In this embodiment, the FB information 114 includes an evaluation result indicating whether the way of eating or feeding the baby food is appropriate. The FB information 114 may also include advice information that instructs on the appropriate way of feeding the baby food. The output control unit 105 may generate the evaluation result and advice information as an image 161, or may generate them as text information such as a message 162.

[0044] For example, FB information 114 to be presented when the way of eating or feeding baby food is appropriate, and FB information 114 to be presented when the way of eating or feeding baby food is inappropriate, may be stored in advance in the storage unit 11. When the result information 113 output by the evaluation determination unit 103 is an evaluation result indicating that the way of eating or feeding baby food is appropriate, the output control unit 105 can select FB information 114 corresponding to the evaluation result and output it to the display device 2. On the other hand, when the result information 113 output by the evaluation determination unit 103 is an evaluation result indicating that the way of eating or feeding baby food is inappropriate, the output control unit 105 can select FB information 114 corresponding to the evaluation result and output it to the display device 2.

[0045] <Detecting eating behavior> An example of the process for detecting eating behavior will be described with reference to Figures 3 and 4. Figure 3 is a diagram showing a specific example of an image 111 of a subject eating baby food, obtained when the baby food is eaten or fed in an inappropriate manner. Figure 4 is a diagram showing a specific example of an image 111 of a subject eating baby food, obtained when the baby food is eaten or fed in an appropriate manner.

[0046] Each figure shows one frame image extracted from a moving image, but before and after that frame image, successive frame images capturing a series of eating actions of the subject are acquired by the image acquisition unit 101, and these successive frame images are stored in the memory unit 11 as a moving image.

[0047] The process of detecting eating behavior is the same whether or not the way the baby is eating or feeding the baby food is appropriate, so below, the process of detecting eating behavior will be described using the example of a video including the frame images shown in Fig. 3 input to the detection unit 102. When a video including the frame images shown in Fig. 4 is input, the detection unit 102 can similarly perform the process of detecting eating behavior.

[0048] When a moving image including the frame images shown in Fig. 3 is input to the detection unit 102, the detection unit 102 analyzes the moving image, digitizes the movement related to the eating behavior, and generates pattern information. (1) A graph 301 showing the pattern of change over time in the amount of forward and backward movement of the subject's face; (2) a graph 302 showing the pattern of change over time in the roll, pitch, and yaw of the subject's head; and (3) Graph 303 showing the pattern of change in the size of the subject's mouth opening over time and output it as the operation information 112.

[0049] In another example, the detection unit 102 may further generate (4) a graph 304 showing a pattern of change over time in color in the inner lip region as one piece of pattern information.

[0050] More specifically, the detection unit 102 may track facial feature points (e.g., the tip of the nose) in the moving image and generate the graph 301 based on the time-series position information of the feature points in the depth direction. The detection unit 102 may generate the graph 301 by grasping the distance between the image capturing device 3 and the face of the subject based on the size of the face or head area of ​​the subject.

[0051] The detection unit 102 may observe the angle of the center of gravity of the head (for example, Euler angles) in the moving image, and generate the graph 302 based on the change in the angle over time.

[0052] The detection unit 102 may track two or more feature points around the mouth in the video and generate graph 303 based on the position information of the feature points in the vertical and horizontal (up, down, left, right) directions over time. Referring to Fig. 7, the feature points around the mouth may be, for example, any of point 313 corresponding to the tip of the nose, point 314 corresponding to the center of the upper lip, point 315 corresponding to the center of the lower lip, point 317 corresponding to the right corner of the mouth, and point 318 corresponding to the left corner of the mouth.

[0053] Furthermore, if necessary, the detection unit 102 may observe color changes in the inner lip region of the subject (hereinafter referred to as the inner lip region) included in the moving image, and generate the graph 304 based on the color changes observed in the inner lip region. For example, the detection unit 102 may identify the region between the upper lip and the lower lip and between the left corner and the right corner of the mouth as the inner lip region. The detection unit 102 may then identify the opening and closing of the mouth during a period in which a predetermined or greater color change is observed in the inner lip region as the opening and closing of the mouth when taking in baby food, i.e., the opening and closing of the mouth related to feeding.

[0054] The horizontal axis of each graph representing the pattern information shown in Figure 3 represents the passage of time. Detection unit 102 can synchronize and arrange each graph, and by understanding the time series of events observed from the waveforms of each graph, can detect the occurrence of an eating behavior within the bold frame shown in the figure and output pattern information for the eating behavior period as behavior information 112. Specifically, when detection unit 102 detects event 311 representing the opening and closing of the mouth based on graph 303, it may detect several seconds before and after the timing of the occurrence of event 311 as eating behavior period 313 (bold frame in Figure 3).

[0055] The detection unit 102 detects one or more eating movements from the video image, and stores pattern information for the period of the eating movements in the storage unit 11 as movement information 112.

[0056] The detection unit 102 may observe a color change around the subject's mouth (for example, the inner lip region described above) and detect an event 312 in which a large color change is observed. In another example, the detection unit 102 may perform image recognition and track a color region (assuming a spoon or the like) of a specific color different from the color that constitutes the subject, and detect an event 312 in which the tracked object comes into frame, approaches the area around the mouth, and then goes out of frame again.

[0057] When detecting event 312 at the same timing as event 311, detection unit 102 may determine that event 311 is a mouth opening and closing action associated with eating. On the other hand, detection unit 102 may determine that event 311 not accompanied by event 312 is a mouth opening and closing action unrelated to eating, and exclude the period before and after event 311 from the period of eating. With the above-described configuration, it is possible to identify the timing when a spoon or the like crosses, and accurately detect eating actions. For example, mouth opening and closing actions unrelated to eating actions can be excluded from the evaluation target as noise.

[0058] <Evaluation of eating behavior> The evaluation / determination unit 103 evaluates the way the subject eats or feeds the baby food based on the operation information 112 output by the detection unit 102 (for example, each graph included in the pattern information shown in FIGS. 3 and 4).

[0059] As an example, the evaluation and determination unit 103 may evaluate the feeding method as appropriate if the identified pattern information indicates a change in which the head moves forward and then rotates downward as the mouth opens and closes during the feeding action of taking in baby food. Alternatively, the evaluation and determination unit 103 may evaluate the feeding method as inappropriate if the pattern information does not indicate such a change. This allows a feeding method that does not encourage the infant to eat spontaneously to be correctly evaluated as "inappropriate." As an example, a feeding method in which the provider brings food to the infant's mouth with a spoon or the like, resulting in the infant's head not moving forward spontaneously, such as a feeding method similar to that used in nursing care, may be evaluated as "inappropriate."

[0060] More specifically, the proper way to feed a baby solid food is thought to involve the following series of steps: Step 1: Scoop some baby food onto the tip of the spoon, lightly place the spoon on the baby's lower lip, and wait for the baby to take it in with their upper lip. Step 2: The infant opens their mouth, moves their head, places the baby food at the tip of the spoon into their mouth, closes their mouth, and then pulls the spoon out horizontally.

[0061] The inventors noticed that when the above steps 1 and 2 are performed correctly, the subject (infant)'s feeding action when taking baby food into his or her mouth involves moving the face forward and tilting the head forward to bring the mouth closer to the spoon.

[0062] The inventors then considered that if the following events are detected in the video in the following order, it can be determined that the subject's eating behavior is accompanied by the above-mentioned movements and that the baby food has been given according to the above-mentioned procedure. Event 1: Move your face forward as if moving towards the spoon in front of you (movement of the subject's face forward and backward) Event 2: The subject tilts his head forward and looks down in an attempt to take the baby food from the tip of the spoon into his mouth (change in head pitch). Event 3: The lower lip moves downward and the mouth opens (change in the distance between the upper and lower lips). Event 4: The spoon crosses the inner lip area between the upper and lower lips and the left and right corners of the mouth (color change in the inner lip area). Event 5: The lower lip moves upward and the mouth closes (change in the distance between the upper and lower lips). Event 6: Returning the head to its original forward tilt position (change in head pitch) Event 7: The face returns to its original position, i.e., the face moves backward (forward and backward movement of the subject's face) After the above events 1 to 7, the subject moves on to the chewing behavior. The eating behavior, which is the stage before the chewing behavior, can be evaluated as appropriate by detecting the above events 1 to 7. The order in which events 1 to 7 occur may also be taken into consideration when evaluating the appropriateness of the feeding method.

[0063] Therefore, the evaluation and determination unit 103 refers to graph 301 during the eating movement period 313 detected by the detection unit 102 and determines whether or not event 1, in which the face moves forward, has been detected. Furthermore, the evaluation and determination unit 103 refers to graph 302 during the eating movement period 313 and determines whether or not event 2, in which the head tilts forward, has been detected. Furthermore, the evaluation and determination unit 103 refers to graph 303 during the eating movement period 313 and determines whether or not event 3, in which the mouth opens, has been detected. Furthermore, the evaluation and determination unit 103 may refer to graph 304 during the eating movement period 313 and determine whether or not event 4, in which the spoon crosses the inner lip region, has been detected. Furthermore, the evaluation and determination unit 103 refers to graph 303 during the eating movement period 313 and determines whether or not event 5, in which the mouth closes, has been detected. Furthermore, the evaluation and determination unit 103 refers to graph 302 during the eating movement period 313 and determines whether or not event 6, in which the head returns to its original forward tilted posture, has been detected. Furthermore, the evaluation / determination unit 103 refers to the graph 301 during the eating behavior period 313 and determines whether or not an event 7 in which the face moves backward has been detected.

[0064] When it is determined that events 1 to 7 have been detected in order during eating behavior period 313, evaluation determination section 103 evaluates that the way the subject eats the baby food or the way the waiter feeds the baby food is appropriate.

[0065] The frame image shown in Figure 3 depicts a scene of the subject's eating behavior when the subject's way of eating or feeding baby food is inappropriate. An example of an inappropriate case would be when, in addition to opening and closing the mouth, the subject does not move their face forward and backward or tilt their head forward and backward during the eating behavior period 313. Therefore, evaluation determination unit 103 may evaluate the way the subject eats or feeds baby food as inappropriate if at least one of the forward and backward movement of the face and the forward and backward tilt of the head does not occur during the eating behavior period 313.

[0066] A specific description will be given with reference to FIG. 3. In the illustrated example, during feeding behavior period 313, which includes event 311 of a mouth opening and closing behavior and event 312 of a spoon passing by, graph 302 does not show any events indicating forward and backward tilting of the head. Furthermore, during feeding behavior period 313, graph 301 does not show any events indicating forward and backward movement of the face. This means that, as shown in the frame image in the same figure, when feeding baby food, the subject maintains the original sitting position without moving the head, and simply opens the mouth and waits for the baby food to be delivered to the mouth. Alternatively, when the subject opens the mouth and waits for the baby food to be delivered to the mouth, the head may not be observed to tilt forward, but only the head may be observed to tilt backward when the mouth is opened.

[0067] The above indicates that the waiter was not feeding the subject in a way that encouraged them to actively participate in the meal. More specifically, the waiter brought the baby food on a spoon to the subject, who was looking up with his mouth open, and then, without waiting for the subject to take the food into his mouth, rubbed the spoon against his palate before removing it. Feeding the baby food in this way by forcing it into the subject's mouth does not foster the awareness or attitude of eating food independently, which may contribute to the underdevelopment of oral function. Furthermore, if the subject places the food in their mouth while looking up, they may develop the habit of swallowing it whole, which may lead to aspiration. Furthermore, the underuse of jaw muscles may also contribute to the underdevelopment of oral function.

[0068] As described above, evaluation determination unit 103 evaluates that the subject's way of eating or feeding the baby food is inappropriate if eating behavior that is unfavorable for the development of an infant is observed during eating behavior period 313. Evaluation determination unit 103 stores the evaluation result indicating inappropriateness in memory unit 11 as result information 113.

[0069] The frame image shown in Figure 4 depicts a scene of the subject's eating behavior when the subject's way of eating or feeding baby food is appropriate. A possible example of an appropriate case would be when, in addition to opening and closing the mouth, the subject moves their face back and forth and tilts their head back and forth during the eating behavior period 413. Therefore, evaluation determination unit 103 may evaluate the way the subject eats or feeds baby food as appropriate if the face moves back and forth and the head tilts back and forth during the eating behavior period 413.

[0070] A specific description will be given with reference to FIG. 4. The evaluation / determination unit 103 focuses on graphs 401 and 402 included in the pattern information for an eating behavior period 413, which includes event 411 of mouth opening and closing behavior in graph 403 and event 412 of spoon crossing in graph 404. According to graph 401, event 414 of moving the face forward is observed immediately before event 411 of mouth opening behavior. Next, according to graph 402, event 415 of tilting the head forward is observed. This means that, as shown in the frame image in the figure, when eating baby food, the subject put their face forward, looked down, and moved to grab the tip of the spoon with their mouth. In other words, this means that the server was able to feed the subject in a way that encouraged them to actively participate in the meal.

[0071] As described above, evaluation determination unit 103 evaluates that the subject's way of eating or feeding baby food is appropriate if eating behaviors that are favorable for the development of an infant are observed during eating behavior period 413. Evaluation determination unit 103 stores the evaluation result indicating appropriateness as result information 113 in memory unit 11.

[0072] <Feedback> 5 is a diagram showing a specific example of the FB information 114 presented to the user on the display device 2 according to the first embodiment. A first display screen example 501 shows an example of the FB information 114 displayed when a negative evaluation is given on how to eat or feed baby food. A second display screen example 502 shows an example of the FB information 114 displayed when a positive evaluation is given on how to eat or feed baby food.

[0073] The output control unit 105 selects or generates FB information 114 corresponding to the evaluation result output by the evaluation determination unit 103, configures a display screen, and displays it on the display device 2. For example, the output control unit 105 (1) Images (images 511, 521) that symbolize the evaluation results output by the evaluation determination unit 103, and (2) Messages providing the results of the evaluation and guidance on proper feeding or feeding methods for infants and young children (Messages 512 and 522), The display screen may be configured based on the FB information 114 including the above.

[0074] Furthermore, the output control unit 105 may include (3) link information (link information 513, 523) for guiding the user to a screen that provides more detailed information on proper feeding methods for infants and young children, in the FB information 114. The output control unit 105 may also place the link information 523 on the display screen when a positive evaluation is made.

[0075] As described above, by displaying the FB information 114 on the display device 2, when there is room for improvement in the way a server feeds food, the server or other users can be made aware of this, motivating the server to improve the way they feed food. Furthermore, the message 512 and the link information 513 enable the server to understand how to improve the way they feed food and actually make improvements. Furthermore, the message 522 and the link information 523 can also make servers who have received positive feedback aware of proper feeding methods. This encourages the server to consciously continue to feed food in an appropriate manner.

[0076] <Processing flow> 6 is a flowchart showing an example of the evaluation method executed by the evaluation system 100 according to embodiment 1. A series of processes of the evaluation method is started, for example, when a moving image captured by the image capturing device 3 is input to the control unit 10.

[0077] In S11, the image acquisition unit 101 acquires an image of the subject eating, and stores the image in the storage unit 11 as the image 111. For example, if a video of the subject eating is captured by the imaging device 3, the image acquisition unit 101 may acquire a frame image extracted from the video as the image 111.

[0078] In S12, the detection unit 102 detects the subject's eating behavior. As a specific example, the detection unit 102 may detect feature points indicating the subject's mouth movement from the group of images 111 and generate pattern information (e.g., graph 303) in which the vertical and horizontal position information of the detected feature points is arranged in time series. The detection unit 102 may also detect facial feature points, such as the tip of the nose, and generate pattern information (e.g., graph 301) in which the depth position information of the detected feature points is arranged in time series. The detection unit 102 may also observe the angle of the center of gravity of the head (particularly, pitch) and generate graph 302 based on the change in the angle over time. If necessary, the detection unit 102 may further detect the inner lip region, observe color changes in the detected inner lip region, and generate graph 304 representing the change in color over time. The inner lip region may be defined, for example, by detecting the feature points of the center of the upper lip, the center of the lower lip, the left corner of the mouth, and the right corner of the mouth.

[0079] In S13, the detection unit 102 stores pattern information of various movements related to the detected eating movements in the storage unit 11 as movement information 112.

[0080] In S14, evaluation determination unit 103 determines whether a movement of moving the face forward and backward and a movement of tilting the head forward and backward were observed during a period in which eating behavior was detected (for example, eating behavior period 313). If the two movements were observed, evaluation determination unit 103 proceeds to YES in S14, and if at least one of the two movements was not observed, proceeds to NO in S14.

[0081] In S15, the evaluation determination unit 103 evaluates that the subject's way of eating the baby food or the waiter's way of feeding the baby food is appropriate.

[0082] In S16, the evaluation determination unit 103 evaluates that the subject's way of eating the baby food or the waiter's way of feeding the baby food is not appropriate.

[0083] In S17, the evaluation determination unit 103 stores the evaluation result, which indicates whether the evaluation is appropriate or inappropriate, in the storage unit 11 as result information 113.

[0084] In S18, the output control unit 105 selects or generates FB information 114 corresponding to the evaluation result indicated by the result information 113, and causes the output device to output the FB information 114. For example, the output control unit 105 configures a display screen including the FB information 114, and causes the display device 2 to display the screen.

[0085] <Effects> According to the above-described configuration and method, the image capture device 3 monitors an infant eating baby food, and the information processing device 1 uses the data obtained through the monitoring to identify a pattern of change over time in the head and mouth movements of the infant while eating baby food. The information processing device 1 evaluates the baby food feeding method based on the difference between the identified pattern and the pattern of change over time in the movements when the infant is fed properly. This allows for an objective evaluation of the baby food eating and feeding method. Furthermore, the objective evaluation results can be presented to the server or a user other than the server. This allows the server or a user other than the server to be made aware of any room for improvement in the server's feeding method, motivating the server to improve the feeding method.

[0086] [Embodiment 2] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0087] <Summary> The evaluation system according to this embodiment may be, for example, as shown in Fig. 2, similar to embodiment 1. The evaluation system 100 according to embodiment 2 is a system for evaluating how an infant eats baby food and determining the progress stage of the infant's baby food.

[0088] In this embodiment, for example, the evaluation system 100 determines the progress stage of an infant's weaning food in the following manner.

[0089] The photographing device 3 photographs the subject eating the baby food (monitoring step).

[0090] The information processing device 1 obtains time-series images obtained by photography as data obtained by monitoring. The information processing device 1 determines the progress stage of the baby food using the time-series images showing the subject's eating behavior (evaluation and determination step). In this embodiment, the information processing device 1 particularly detects, from the eating behavior, chewing movements that move various oral organs from the time the subject puts the baby food in his / her mouth until he / she swallows it, and determines the progress stage of the baby food of the subject based on the detected chewing movements.

[0091] The display device 2 as an output device outputs FB information generated based on the result of the evaluation or judgment (output step). For example, the display device 2 displays, as the FB information, an image 161 showing the result of the judgment made by the information processing device 1 and a message 162 for the user regarding the judged progress stage of the baby food.

[0092] This allows the user to recognize the progress of the baby's weaning food, which is determined according to the developmental stage of feeding function. In other words, the FB information is useful information that helps the user determine whether or not to advance the baby's weaning food to the next stage. The user refers to a person who uses the evaluation system 100 to obtain the assessment result of the progress of the baby's weaning food, and the user may be the server himself or herself, or someone other than the server.

[0093] 2 is merely an example, and the content thereof is arbitrary as long as it allows the user to understand the progress of the baby's weaning food. Feedback to the user is not limited to visual feedback, and may be provided by sound, vibration, or a combination thereof. Feedback may also be provided via another device, such as by displaying the FB information on a display device external to the information processing device 1.

[0094] <Configuration of information processing device> The information processing device 1 according to this embodiment may be, for example, as shown in FIG. 1, similarly to the first embodiment.

[0095] In this embodiment, the detection unit 102 analyzes the image to detect the chewing action that occurs following the eating action of taking the baby food into the mouth. The detection unit 102 stores the detection result of the chewing action in the memory unit 11 as action information 112.

[0096] Specifically, the detection unit 102 detects the magnitude of the movement of the subject's mouth while chewing, based on images 111 captured in time series of the chewing behavior of the subject. As an example, the detection unit 102 may detect the magnitude of each of the vertical (up-down) and horizontal (left-right) movement of the mouth. More specifically, the detection unit 102 may be configured to have a function as a feature point detection unit that detects feature points and a function as a movement analysis unit that analyzes the movement of the mouth based on position information of the feature points.

[0097] The detection unit 102 detects the magnitude of the mouth movement of the subject while chewing based on the time-series images obtained by photographing. The detection unit 102 may detect the vertical and horizontal sizes of the mouth as distances, or may detect the magnitude of the mouth movement as an area. Specifically, the distance may be the vertical and horizontal movement distances of the mouth or feature points around it. The area may be the area of ​​a region surrounded by a trajectory drawn by the mouth or feature points around it.

[0098] It is also possible to use images captured by an imaging device 3 such as a 3D camera that can capture the movement of the mouth in the depth direction. In this case, the detection unit 102 may detect the vertical, horizontal, and depth dimensions of the mouth as distances, or may detect the size of the mouth movement as a volume. Specifically, the volume may be the volume of a space surrounded by the trajectory of the mouth or its surrounding feature points.

[0099] First, the detection unit 102 detects predetermined feature points on the face of the subject from the image 111. The detection unit 102 detects a chewing motion based on time-series position information of the detected feature points. The detection unit 102 may store the time-series position information of the feature points corresponding to the detected chewing motion as motion information 112 in the storage unit 11.

[0100] Furthermore, the detection unit 102 may monitor the distance between the apex of the baby's upper lip and the apex of the baby's lower lip observed in each frame image of the video (for example, the distance between points 314 and 315 in FIG. 7). A series of mouth opening and closing actions, including a mouth opening action for which the distance is equal to or greater than a predetermined threshold, may be excluded from the chewing actions to be detected. In this way, the mouth opening and closing actions when taking in baby food can be distinguished and excluded from the chewing actions required to determine the baby's stage of progress in baby food. This prevents the opening action of putting a spoon in the mouth, which is not chewing, from being mistaken for the opening action during chewing, making it possible to accurately determine the baby's stage of progress in baby food based on the nature of the chewing actions.

[0101] Next, the detection unit 102 analyzes the movement of the mouth during the chewing action using the above-mentioned time-series position information. The detection unit 102 outputs information indicating the magnitude of the mouth movement identified as a result of the analysis. The detection unit 102 may store the information indicating the magnitude of the mouth movement in the storage unit 11 as movement information 112. Specifically, the detection unit 102 may calculate an index value indicating the magnitude of the mouth movement as the information indicating the magnitude of the mouth movement. The detection unit 102 may generate visualization information that visualizes the magnitude of the mouth movement as the information indicating the magnitude of the mouth movement. The visualization information may be a graph, a figure, an illustration, etc. that represents the magnitude of the mouth movement. The detection unit 102 generates the above-mentioned graph, figure, illustration, etc. based on the time-series position information and the index value.

[0102] The detection unit 102 may store at least one of the index value indicating the magnitude of the mouth movement and the visualization information in the storage unit 11 as the movement information 112.

[0103] The evaluation and determination unit 103 evaluates the chewing action based on the detected chewing action information 112. Specifically, the evaluation and determination unit 103 identifies the manner of the chewing action. Then, the evaluation and determination unit 103 determines the progress stage of the baby's weaning food from the identified manner of the chewing action. The evaluation and determination unit 103 may store the determination result indicating the determined progress stage as result information 113 in the storage unit 11.

[0104] The output control unit 105 selects or generates FB information 114 corresponding to the result information 113 output from the evaluation determination unit 103, and causes the output device to output it. For example, the output control unit 105 may configure a display screen including FB information 114 corresponding to the determination result indicated by the result information 113, and cause the display device 2 to display it.

[0105] <Detection of chewing movements> (Feature point detection) A method for detecting feature points by the detection unit 102 will be described with reference to Fig. 7. Fig. 7 is a diagram showing a specific example of feature points detected by the detection unit 102 and analysis result information (motion information 112) of mastication movements obtained based on the detected feature points.

[0106] The schematic diagram 31 of a face in FIG. 7 shows examples of feature points to be detected. The feature points detected by the detection unit 102 are used to analyze the movement of the mouth during mastication, and therefore may be parts that move during mastication in a manner that reflects the mastication style. For example, the detection unit 102 may detect parts around the mouth on the center line L of the face as feature points, such as point 314 (the apex of the upper lip), point 315 (the apex of the lower lip), point 316 (the center position of the jaw), or the tip of the jaw (not shown) in the schematic diagram 31. Detecting parts on the center line L as feature points makes it easier to identify the magnitude of lateral movement of the mouth during mastication.

[0107] Furthermore, the detection unit 102 may detect points 317 and 318 (corners of the mouth) in the schematic diagram 31 as feature points. In this case, the detection unit 102 may analyze the movement of the mouth by regarding the movement of the midpoint between point 317 (right corner of the mouth) and point 318 (left corner of the mouth) as the movement of the mouth. Alternatively, for example, the detection unit 102 may detect the cheek area near the masseter muscle as a feature point, and in this case, the detection unit 102 may analyze the movement of the mouth from the change in the area of ​​that area.

[0108] Furthermore, the detection unit 102 may detect a part that does not move due to chewing. For example, the detection unit 102 may detect point 313 (tip of nose). In this case, the detection unit 102 can represent the positions of feature points captured in the image 111 using the position of point 313 (tip of nose) as a reference. This makes it possible to accurately identify the time-series movement of each feature point even if the position at which the subject appears shifts between the time-series images 111.

[0109] Furthermore, the detection unit 102 may detect multiple parts of the face that do not move due to chewing in order to determine a distance on the image 111 that serves as a reference for evaluating the magnitude of mouth movement. For example, the detection unit 102 may detect points 311 and 312 (the inner corners of the eyes). This allows the evaluation determination unit 103 to evaluate the magnitude of mouth movement based on the distance between point 311 (the inner corner of the right eye) and point 312 (the inner corner of the left eye). The detection unit 102 may use points 311, 312, 313, 317, 318, etc. to determine the center line L of the face.

[0110] In this way, the detection unit 102 may detect the magnitude of vertical and horizontal movement of the mouth of the subject while chewing based on the distance between predetermined feature points on the face of the subject that do not change position due to chewing, detected from the image 111. This makes it possible to accurately detect the magnitude of mouth movement whether the subject appears large or small in the image 111.

[0111] The reference distance for evaluating the magnitude of mouth movement may be input by the subject, etc. Also, if the distance to the subject is known (for example, when photographing with a 3D camera that outputs distance information), the magnitude of mouth movement can be determined according to that distance, so part detection for calculating the reference distance can be omitted.

[0112] The method for detecting the feature points as described above is not particularly limited, and for example, an image analysis method, a machine learning method, or a combination of these may be applied. Furthermore, when detecting multiple feature points, a detection unit 102 may be provided for each feature point to be detected. Furthermore, detection markers, two-dimensional codes, or three-dimensional codes may be attached to the detection target parts of the subject's face, and these may be detected from the image.

[0113] (Movement analysis) The detection unit 102 analyzes the movement of the mouth during mastication from the time-series positional changes of the feature points detected as described above, and calculates an index value indicating the movement. This will be explained based on the graph 32 shown in FIG.

[0114] Graph 32 is a graph showing time-series positional changes of feature points on images 111. Graph 32 can be generated from positional information of feature points detected in each of a plurality of images 111 in time series.

[0115] For example, the positions of the apex of the lower lip (point 315 in schematic diagram 31) and the jaw (point 316 in schematic diagram 31) change during mastication as shown in graph 32. In graph 32, initial position 321 indicates the position of the feature point at the start of mastication (just before starting to open the mouth). As shown in graph 32, after mastication begins, the feature point moves downward and left, then moves upward and right, and then moves leftward. It should be noted that the left and right described here are left and right as seen from the side photographing (observing) the subject, and left and right are reversed from the subject's perspective.

[0116] As described above, the graph 32 has a feature that the feature points move from the initial position 321 at the start of mastication and then return to the vicinity of the initial position 321. Therefore, the detection unit 102 can use this feature to identify one mastication action from the graph 32 (more precisely, the position information of the feature points detected from each of the multiple images 111 in time series). This also makes it possible to calculate an index value indicating the number of mastications from the start of chewing the object to the time it is swallowed.

[0117] Furthermore, from graph 32, it is possible to identify the position when the feature point moves furthest to the left and the position when it moves furthest to the right during one chewing action. Therefore, the detection unit 102 can also identify that the magnitude of the horizontal movement of the feature point is Mox from the position information of the feature point detected from each of the multiple time-series images 111. Similarly, the detection unit 102 can also identify the position when the feature point is at the top (for example, the initial position 321 at the start of mastication) and the position when the feature point moves furthest to the bottom from the position information of the feature point detected from each of the multiple time-series images 111, and can also identify that the magnitude of the vertical movement of the feature point is Moy. Here, the detection unit 102 may determine the movement distance when the movement of the feature point is represented on a two-dimensional plane (on the captured image) as Mox and Moy, or may determine the three-dimensional movement distance of the feature point as Mox and Moy.

[0118] Specifically, the detection unit 102 extracts position information included in one chewing movement from the position information of feature points detected from each of the multiple time-series images 111. Note that one chewing movement can also be detected based on the movement of the corners of the mouth, the thickness of the lips, the change in the area of ​​the cheeks near the masseter muscles, etc. Then, the detection unit 102 calculates the distance Mox between the pieces of position information indicating the leftmost position and the rightmost position, and also calculates the distance Moy between the pieces of position information indicating the uppermost position and the lowermost position, as index values ​​indicating the size of the mouth in one chewing movement.

[0119] At this time, the detection unit 102 may calculate index values ​​indicating the magnitude of vertical and horizontal movement of the subject's mouth based on the distance between predetermined feature points on the subject's face detected from the image 111, the positions of which do not change due to chewing. For example, the detection unit 102 may calculate the distance Ex between point 311 (the inner corner of the right eye) and point 312 (the inner corner of the left eye) in the schematic diagram 31, calculate the value Mox / Ex as an index value indicating the magnitude of horizontal movement of the mouth, and calculate the value Moy / Ex as an index value indicating the magnitude of vertical movement of the mouth. This makes it possible to non-dimensionalize the values ​​Mox and Moy and cancel out differences in the size of the subject appearing in the image 111.

[0120] Note that the index value indicating the movement of the mouth during mastication is not limited to the above example. For example, the area of ​​the portion surrounded by graph 32 (hereinafter referred to as Mos) can be said to indicate the magnitude of the movement of the mouth, so the area of ​​this portion may be used as the index value indicating the magnitude of the movement of the mouth during mastication. Mos may also be made dimensionless by dividing it by the distance between predetermined feature points whose position does not change due to mastication (for example, the above-mentioned Ex).

[0121] Furthermore, detection unit 102 may calculate the length of the diagonal distance found from Mox and Moy by Pythagoras' theorem as an index value representing the magnitude of the vertical and horizontal movements of the mouth. To explain this more specifically with reference to graph 32 shown in Fig. 7, detection unit 102 may calculate the length of the diagonal line of a rectangle (shown by a dashed line) whose horizontal length is Mox and whose vertical length is Moy as an index value representing the magnitude of the vertical and horizontal movements of the mouth.

[0122] The detection unit 102 may also calculate index values ​​other than the index value indicating the magnitude of mouth movement. For example, the detection unit 102 may use the number of chews per unit time, the number of chews from putting one chewing object in the mouth until swallowing, the chewing rhythm, etc. as index values. In addition, the detection unit 102 may determine which side is being chewed, left or right, based on the movement of the corners of the mouth, for example, and calculate the number of chews on the left and right or the ratio thereof as index values. Furthermore, for example, the detection unit 102 may determine whether the movement of the feature points is within a normal range or not, and output an index value indicating the determination result.

[0123] In obtaining the index value, the detection unit 102 may use the average value of Mox and Moy, may use the total value of Mox and Moy, or may use either Mox or Moy.

[0124] (Output of operation information) As described above, the detection unit 102 can output analysis result information of the chewing movement as the movement information 112. The analysis result information of the chewing movement specifically refers to various index values ​​and visualized information (graphs) related to the movement of the subject's mouth. The detection unit 102 may select, from the various analysis results related to the chewing movement, information required by the evaluation determination unit 103 to determine the progress stage of the weaning food of the subject, who is an infant, and output the selected information as the movement information 112.

[0125] In this embodiment, for example, the detection unit 102 detects one chewing motion or multiple chewing motions. (1) A graph like graph 32 showing the time-series positional change of the feature point 315 corresponding to the apex of the lower lip (hereinafter referred to as the lower lip trajectory graph during chewing), (2) A graph showing the time series position change of the feature point 318 corresponding to the left corner of the mouth (hereinafter referred to as the left corner of the mouth trajectory graph during chewing), (3) A graph showing the time-series positional change of the feature point 316 corresponding to the center of the lower jaw (hereinafter referred to as a lower jaw trajectory graph during mastication) may be output as analysis result information of the mastication movement (movement information 112).

[0126] The evaluation and determination unit 103 may refer to the lower lip trajectory graph during mastication, for example, to evaluate the magnitude of vertical and horizontal movement of the subject's mouth during mastication. The evaluation and determination unit 103 may refer to the left mouth corner trajectory graph during mastication, for example, to evaluate the magnitude of vertical and horizontal movement of the corners of the mouth during mastication. The evaluation and determination unit 103 may refer to the mandibular trajectory graph during mastication, for example, to evaluate the masticatory cycle (chewing cycle) that represents the movement path of the mandible during mastication.

[0127] The detection unit 102 may output, instead of the lower lip trajectory graph during mastication, the Mox and Moy of one mastication movement or the average values ​​of Mox and Moy of multiple mastication movements for the feature point 315 corresponding to the apex of the lower lip.

[0128] The detection unit 102 may output, instead of the left mouth corner trajectory graph during mastication, the Mox and Moy of one mastication movement or the average values ​​of Mox and Moy of multiple mastication movements for the feature point 318 corresponding to the left corner of the mouth.

[0129] If necessary, the detection unit 102 may further output, as the operation information 112, (4) a graph showing the change in the distance between the apex 314 of the upper lip and the apex 315 of the lower lip (hereinafter referred to as a chewing mouth width graph), corresponding to one detected chewing action or multiple detected chewing actions.

[0130] In another example, the detection unit 102 may output, as the motion information 112, a graphic showing the length of the diagonal of a rectangle (shown by a dashed line) whose horizontal length is Mox and whose vertical length is Moy for the graph 32 shown in FIG.

[0131] <Assessing the progress of weaning food> The period from the start of weaning (eating) to the completion of weaning (eating) is generally thought of as being divided into four stages according to the developmental stage of the infant's feeding function, and the progress stages of weaning food are also set out in accordance with these four stages. Specifically, the progress stages of weaning food can be divided into the following four stages. Stage 1 (early weaning): In terms of age, this corresponds to infants aged 5 to 6 months. In terms of feeding function, infants at this stage are able to close their mouths and take in and swallow. However, after taking in food, the chewing movement leading up to swallowing may not be sufficient, and the mouth may remain slightly open. This is commonly referred to as the "gulping stage." Stage 2 (mid-weaning): In terms of age, this corresponds to infants aged 7-8 months. In terms of feeding function, infants at this stage are able to crush food with their tongue and upper palate. This is commonly referred to as the "chewing stage." Stage 3 (late weaning): In terms of age, this corresponds to infants aged 9 to 11 months. In terms of feeding function, infants at this stage are able to crush food with their gums. This is commonly referred to as the "chewing stage." Stage 4 (Complete weaning): In terms of age, this corresponds to infants aged 12 to 18 months. In terms of feeding function, infants at this stage are able to chew food using their teeth. This is commonly referred to as the "munching stage."

[0132] The inventors have noticed that infants in the early weaning stage exhibit vertical movement at the corners of their mouths as they open and close their mouths during chewing. Therefore, the evaluation and determination unit 103 can be configured to determine the progress stage of the subject's weaning food as early weaning when the evaluation and determination unit 103 identifies a chewing pattern in which vertical movement at the corners of the mouth is present. As an example, the evaluation and determination unit 103 may determine the progress stage as early weaning when the index value Moy indicating the magnitude of vertical movement of the apex of the lower lip (point 315) is equal to or greater than a predetermined threshold and the index value Moy of the left corner of the mouth (point 318) is equal to or greater than a predetermined threshold.

[0133] Furthermore, the evaluation and determination unit 103 may identify a chewing pattern in which the insides of the upper and lower lips remain open during chewing when the shortest distance between the apex of the upper lip (point 314) and the apex of the lower lip (point 315) of the infant during the chewing action is equal to or greater than a predetermined threshold. When such a chewing pattern is identified, the evaluation and determination unit 103 may determine that the progress stage of weaning food is the early weaning stage. This makes it possible to more accurately determine whether the infant is in the early weaning stage. This also contributes to early detection and improvement of tongue thrusting habits.

[0134] The inventors noticed that infants in the intermediate weaning stage exhibit horizontal movement at the corners of the mouth in addition to vertical movement during chewing. Therefore, the evaluation and determination unit 103 can be configured to determine the progress stage of the subject's weaning food as the intermediate weaning stage when the evaluation and determination unit 103 identifies a chewing pattern in which the corners of the mouth exhibit horizontal movement in addition to vertical movement. As an example, the evaluation and determination unit 103 may determine the progress stage as the intermediate weaning stage when the index value Moy of the apex of the lower lip (point 315) is equal to or greater than a predetermined threshold and the index value Mox indicating the magnitude of the horizontal movement of the left corner of the mouth (point 318) is equal to or greater than a predetermined threshold.

[0135] The inventors have noticed that in infants in the later weaning stage, lateral movements are also observed in the lips and jaw during chewing. Therefore, the evaluation and determination unit 103 can be configured to determine the progress stage of the subject's weaning food as the later weaning stage when the evaluation and determination unit 103 identifies a chewing pattern in which lateral movements are also observed in the lips and jaw. As an example, the evaluation and determination unit 103 may determine the progress stage as the later weaning stage when the index value Mox of the apex of the lower lip (point 315) is equal to or greater than a predetermined threshold.

[0136] The inventors have noticed that the chewing cycle observed in the chewing movement of infants in the weaning completion stage changes from a vertically long rod shape to a pointed teardrop shape, becoming rounder and closer to a circle. Therefore, the evaluation determination unit 103 can be configured to determine the progress stage of the subject's weaning food as the weaning completion stage when a chewing mode in which a chewing cycle that is more rounded than a predetermined value is observed in the mandibular trajectory graph during mastication is identified. Alternatively, the evaluation determination unit 103 may determine the weaning completion stage when a chewing mode in which the ratio of the index value Mox of the mandibular center (point 316) to the index value Moy is more than a predetermined threshold value.

[0137] The evaluation and determination unit 103 stores the determination result indicating one of the above-mentioned stages of progress of the baby food in the storage unit 11 as result information 113.

[0138] The information processing device 1 may receive information related to the subject from a user (e.g., the subject's caregiver) via an input unit (not shown) in advance and store it in the storage unit 11. The information related to the subject may include, for example, the subject's age in months and the state of tooth eruption. The information related to the subject may further include information that can be used as a reference when determining the progress stage of weaning food, such as height, weight, duration of each feeding, number of times fed per day, results of a questionnaire regarding the subject's behavior, and results of a questionnaire regarding the hardness of the weaning food the subject is currently eating.

[0139] The evaluation and determination unit 103 may determine the progress stage of the subject's weaning food by taking into account at least one of the subject's age in months and the state of tooth eruption, thereby making it possible to more accurately determine the progress stage of the subject's weaning food.

[0140] <Feedback> 8 is a diagram showing a specific example of FB information 114 presented to a user on the display device 2 according to embodiment 2. The example display screen shown in the figure shows an example of FB information 114 displayed when the progress stage of the subject's weaning food is determined to be the "mid-weaning stage."

[0141] The output control unit 105 selects or generates FB information 114 corresponding to the judgment result output by the evaluation judgment unit 103, configures a display screen, and displays it on the display device 2. For example, the output control unit 105 (1) an image 801 that symbolizes the judgment result output by the evaluation / judgment unit 103; and (2) A message 803 instructing the user on the result of the assessment and the hardness and size of food to be fed to infants at the progress stage corresponding to the result of the assessment; The display screen may be configured based on the FB information 114 including the above.

[0142] Furthermore, the output control unit 105 may include in the FB information 114 (3) a name 802 of the weaning food stage that succinctly expresses the stage of progress corresponding to the determination result.

[0143] Furthermore, the output control unit 105 may include link information (not shown) in the FB information 114 to guide the user to a screen that provides more detailed guidelines such as the hardness and size of the baby food to be given at the determined progress stage (4).

[0144] When presenting the image 801 or name 802, the output control unit 105 may arrange the images and names of all four stages of weaning food on the display screen in chronological order, and then place a cursor 804 to clearly indicate the image and name of the weaning food stage that applies to the subject. This display allows the user to not only understand the subject's current situation, but also to understand which stage the subject is at in the weaning period from the start to the end of weaning. For example, if a user has been feeding their baby food equivalent to the early weaning stage, but is presented with a weaning food stage that indicates the subject is in the middle weaning stage, the user can realize that the subject can move on to the next weaning food stage. This motivates the user to step up the weaning food by changing the hardness and size of ingredients.

[0145] Furthermore, the information processing device 1 may receive in advance from a user via an input unit (not shown) a current baby food progress stage of the subject predicted by the user and store the same in the storage unit 11. The evaluation and determination unit 103 can compare the baby food progress stage of the infant predicted in advance by the user with the baby food progress stage determined from the identified chewing behavior, and determine whether they are different or similar. The evaluation and determination unit 103 may then store the determination result indicating the difference or similarity in the storage unit 11 as result information 113.

[0146] As a result, the output control unit 105 can present the user with FB information 114 that informs the user whether or not the progress stage that the user thinks is consistent with the progress stage determined by the evaluation determination unit 103. For example, if the two do not match, the output control unit 105 can display in the FB information 114 a message or the like that recommends the user to feed the baby food that is appropriate for the determined progress stage.

[0147] As a result, the user can proceed with weaning food in a timely manner in line with the baby's physical growth and improvement in feeding function without having to worry about their own judgment.

[0148] <Processing flow> 9 is a flowchart showing an example of an evaluation method executed by the evaluation system 100 according to embodiment 2. A series of processes of the evaluation method is started, for example, when a moving image captured by the image capturing device 3 is input to the control unit 10.

[0149] In S31, the image acquisition unit 101 acquires an image of the subject eating, and stores the image in the storage unit 11 as the image 111. For example, if a video of the subject eating is captured by the imaging device 3, the image acquisition unit 101 may acquire a frame image extracted from the video as the image 111.

[0150] In S32, the detection unit 102 detects the chewing behavior of the subject and analyzes the chewing movement. To give a specific example, the detection unit 102 may output Mox and Moy of the lower lip trajectory graph during chewing or point 315 at the apex of the lower lip. The detection unit 102 may output an index value calculated from Mox and Moy of the left mouth corner trajectory graph during chewing or point 318 at the left mouth corner. Note that, in obtaining the index value, the detection unit 102 may use the values ​​of Mox and Moy for one chewing behavior, the average values ​​of Mox and Moy for any number of chewing behaviors, or the total values ​​of Mox and Moy for any number of chewing behaviors. Here, the detection unit 102 may take the movement distance of the feature point when it is represented on a two-dimensional plane (on a captured image) as Mox and Moy, or may take the three-dimensional movement distance of the feature point as Mox and Moy. The detection unit 102 may output a mandibular trajectory graph during mastication or a chewing cycle of the mandibular center point 316.

[0151] In S33, the detection unit 102 stores the above-mentioned various trajectory graphs obtained as a result of analyzing the detected chewing movement in the storage unit 11 as analysis result information (movement information 112).

[0152] In S34, the evaluation and determination unit 103 attempts to identify a chewing mode in which the corners of the mouth move horizontally in addition to a chewing mode in which the corners of the mouth move vertically for the detected chewing movements. If the evaluation and determination unit 103 identifies a chewing mode in which the corners of the mouth move vertically but cannot identify a chewing mode in which the corners of the mouth move horizontally, the process proceeds from NO in S34 to S35. On the other hand, if the evaluation and determination unit 103 can identify a chewing mode in which the corners of the mouth move horizontally in addition to a chewing mode in which the corners of the mouth move vertically, the process proceeds from YES in S34 to S36.

[0153] In S35, the evaluation determination unit 103 determines that the subject's stage of progress in weaning food is the early weaning stage based on the fact that the chewing manner in which the corners of the mouth move vertically has been identified.

[0154] In S36, the evaluation and determination unit 103 further attempts to identify a chewing pattern in which the lips or jaw move sideways from the detected chewing movement. If the evaluation and determination unit 103 is unable to identify a chewing pattern in which the lips or jaw move sideways, the process proceeds from NO in S36 to S37. On the other hand, if the evaluation and determination unit 103 is able to identify a chewing pattern in which the lips or jaw move sideways, the process proceeds from YES in S36 to S38.

[0155] In S37, the evaluation determination unit 103 determines that the progress stage of the baby food of the subject is the middle stage of weaning, based on the fact that the chewing manner in which the corners of the mouth move in the lateral direction has been identified.

[0156] In S38, the evaluation and determination unit 103 attempts to identify a chewing mode in which the chewing cycle of the lower jaw is rounded for the detected chewing movement. If the evaluation and determination unit 103 cannot identify a chewing mode in which the chewing cycle of the lower jaw is rounded, the process proceeds from NO in S38 to S39. On the other hand, if the evaluation and determination unit 103 can identify a chewing mode in which the chewing cycle of the lower jaw is rounded, the process proceeds from YES in S38 to S40.

[0157] In S39, the evaluation determination unit 103 determines that the subject's stage of progress in weaning food is the late weaning stage based on the fact that the chewing manner in which the lips and jaw move sideways has been identified.

[0158] In S40, the evaluation determination unit 103 determines that the progress stage of the weaning food of the subject is the weaning completion stage based on the fact that the chewing manner in which the chewing cycle of the lower jaw is rounded has been identified.

[0159] In S41, the evaluation / determination unit 103 stores the determination result indicating any one of the progress stages determined for the subject in the storage unit 11 as result information 113.

[0160] In S42, the output control unit 105 selects or generates FB information 114 corresponding to the determination result indicated by the result information 113, and causes the output device to output the FB information 114. For example, the output control unit 105 configures a display screen including the FB information 114, and causes the display device 2 to display the screen.

[0161] <Effects> According to the above-described configuration and method, the image capture device 3 monitors an infant eating baby food, and the information processing device 1 uses the data obtained from the monitoring to identify the chewing behavior of the infant and determine the infant's progress in baby food from the identified behavior. Therefore, the progress of the infant's baby food can be accurately determined based on the object of the infant's chewing. Furthermore, the determination result can be presented to the server or a user other than the server.

[0162] In the past, the only way to refer to the weaning stage was to refer to the baby's age and the developmental stage related to feeding function. However, the evaluation system 100 and evaluation method according to this embodiment make it possible for a server or a user other than a server to recognize the progress stage of an infant's weaning food, determined based on the infant's chewing behavior. The FB information thus presented, indicating the progress stage, is very useful information that helps the user decide whether or not to advance the baby's weaning food to the next stage.

[0163] [Embodiment 3] The evaluation system 100 of the present disclosure can be configured to evaluate at least one of the way an infant eats baby food and the way a server feeds the infant, and to determine the infant's progress in baby food.

[0164] In this case, the detection unit 102 of the information processing device 1 is configured to detect eating behavior and generate pattern information of the eating behavior (for example, the graphs shown in FIGS. 3 and 4) as described in embodiment 1, and to detect chewing behavior and analyze the chewing behavior as described in embodiment 2. The detection unit 102 generates, for example, the index values ​​and graph shown in FIG. 7 as analysis result information of the chewing behavior. The detection unit 102 stores the pattern information of the eating behavior and the analysis result information of the chewing behavior in the storage unit 11 as behavior information 112.

[0165] Based on the eating behavior pattern information, the evaluation and judgment unit 103 executes the evaluation and judgment steps shown in Fig. 6 to evaluate the appropriateness of the way the infant eats the baby food or the way the waiter feeds the infant the baby food. Furthermore, based on the chewing behavior analysis result information, the evaluation and judgment unit 103 executes the evaluation and judgment steps shown in Fig. 9 to judge the progress stage of the infant's baby food.

[0166] The evaluation and determination unit 103 may use information on the analysis results of the chewing behavior to improve the accuracy of evaluating the appropriateness of the eating or feeding method. The evaluation and determination unit 103 may also use information on the pattern of the eating behavior to improve the accuracy of determining the progress stage of the baby food. For example, the evaluation and determination unit 103 may identify the subject's chewing behavior after excluding the opening and closing of the mouth during the eating behavior period identified based on the pattern of color change in the inner lip area from the chewing behavior for determining the progress stage of the baby food.

[0167] The output control unit 105 causes the display device 2 to display the FB information 114 shown in Fig. 5 and the FB information 114 shown in Fig. 8. The output control unit 105 may configure the display screen so that each piece of FB information 114 is displayed on one screen, or may configure the display screen so that two display screens displaying each piece of FB information 114 are switched between.

[0168] [Embodiment 4] The evaluation or judgment process executed by the evaluation and judgment unit 103 of the information processing device 1 may be executed by AI (Artificial Intelligence).

[0169] In this case, the evaluation and judgment unit 103 is configured to use data obtained by monitoring an infant eating baby food as explanatory variables, and the evaluation results of how the infant eats or is fed baby food, or the judgment results of the infant's baby food progress stage as objective variables, and to make an evaluation or judgment using an evaluation model that has learned the relationship between the explanatory variables and the objective variables.

[0170] As an example, a first evaluation model may be constructed that learns the relationship between the explanatory variables and the objective variables, with data obtained by monitoring an infant eating baby food as explanatory variables and the evaluation results of how the infant eats or is fed the baby food as objective variables. The evaluation determination unit 103 is configured to input the data obtained by monitoring into the first evaluation model and evaluate how the infant eats or is fed the baby food by obtaining output data output from the first evaluation model.

[0171] The data obtained by monitoring an infant eating baby food may be, for example, moving images or still images obtained by photographing the infant eating baby food using the photographing device 3. Alternatively, the data may be pattern information of feeding behavior (behavior information 112) obtained by analyzing the moving images or still images by the detection unit 102. Furthermore, the output data output from the first evaluation model may be an evaluation result indicating whether the feeding method is appropriate or inappropriate.

[0172] In another example, a second evaluation model may be constructed that learns the relationship between the explanatory variables and the objective variables, with data obtained by monitoring an infant eating baby food as explanatory variables and the judgment result of the infant's baby food progress stage as objective variables. The evaluation judgment unit 103 is configured to input the data obtained by monitoring to the second evaluation model and judge the baby food progress stage by obtaining output data output from the second evaluation model.

[0173] The data obtained by monitoring an infant eating baby food may be, for example, moving images or still images obtained by photographing the infant eating baby food using the photographing device 3. Alternatively, the data may be analysis result information (movement information 112) of the chewing action obtained by analyzing the moving images or still images by the detection unit 102. Furthermore, the output data output from the second evaluation model may be a determination result indicating any one of the baby food progress stages applicable to the infant.

[0174] [Modification] (Another example of a monitoring device) The monitoring device for monitoring an infant eating baby food is not limited to the imaging device 3. The monitoring device may be a position sensor for determining the position of the infant's head, an electromyograph for measuring the amount of muscle activity of the infant's masticatory muscles, or a combination of these.

[0175] (Another example of FB information) If there is any behavior that was not observed in S14 of FIG. 6, the evaluation determination unit 103 of the information processing device 1 according to the first embodiment may generate an evaluation result in S16 that includes information indicating the behavior that was not observed.

[0176] The output control unit 105 may cause the display device 2 to display FB information 114 indicating what movements were not observed. For example, if a movement of moving the face forward and a movement of tilting the head forward were not observed, the output control unit 105 may cause the display device 2 to display, as the FB information 114, a sentence explaining that these movements were not observed.

[0177] Furthermore, the evaluation determination unit 103 may classify the way of eating or feeding baby food into several types depending on the type of observed or unobserved behavior. For example, the evaluation determination unit 103 may classify into types such as "pressing the spoon against the upper lip or upper jaw," "putting the spoon too far back in the mouth," and "tilting the head back."

[0178] The output control unit 105 may select a message indicating points to improve in the way of eating or feeding according to the type classified by the evaluation determination unit 103, and display the message as FB information 114 on the display device 2. For example, if the type is classified as "pressing the spoon against the upper lip or upper jaw," the output control unit 105 may select a message such as "place the spoon lightly on the lower lip and wait until the upper lip takes the food in," and display the message as FB information 114 on the display device 2.

[0179] Alternatively, the output control unit 105 may cause the display device 2 to display, as the FB information 114, a moving image or animation showing points to be improved, instead of or in addition to a message.

[0180] In the first embodiment, if the detection unit 102 does not detect an eating action of the subject taking baby food into their mouth from the video, the evaluation determination unit 103 may determine that the appropriateness of the eating or feeding method cannot be evaluated. In this case, the output control unit 105 may cause the output device to output, as FB information 114, an error message to the effect that an eating action for evaluating the appropriateness of the eating or feeding method was not detected.

[0181] In the first to third embodiments, the detection unit 102 may analyze the time-series positions of both the left and right corners of the subject's mouth or the movement path of the lower jaw in the moving image. If the analysis result information indicates that there is a bias in the amount of movement of the motor organs between the left and right sides of the face during mastication, the evaluation and determination unit 103 may identify the mastication pattern as a bad oral habit of uneven mastication. In this case, the output control unit 105 may cause the output device to output a warning message indicating that there is a suspicion of uneven mastication as FB information 114.

[0182] [Additional Notes] Some or all of the above-described embodiments can also be described as follows, although the present disclosure is not limited to the aspects described below.

[0183] (Appendix 1) An information processing device (1) according to one embodiment of the present disclosure includes an image acquisition unit (101) that acquires video images of an infant eating baby food, a detection unit (102) that detects from the video images the feeding behavior of the infant when taking in baby food or the chewing behavior of the infant until swallowing the baby food that the infant has taken in their mouth, an evaluation and judgment unit (103) that evaluates or judges the way the infant eats or is being fed based on the analysis results of the detected feeding behavior or chewing behavior, and an output control unit (105) that outputs feedback information generated based on the results of the evaluation or judgment.

[0184] According to the above configuration, infants and young children eating baby food are monitored, and the data obtained from the monitoring is used to evaluate how the baby food is eaten or fed, so that how the baby food is eaten or fed can be objectively evaluated.

[0185] (Appendix 2) In the information processing device of Appendix 1, the detection unit detects the feeding behavior based on changes over time in the head and mouth movements of an infant eating baby food, and generates pattern information indicating a pattern of the feeding behavior by quantifying the changes over time in the head and mouth movements of the infant during the feeding behavior, and the evaluation and determination unit may evaluate the way the infant eats or is fed based on the pattern information.

[0186] The method of feeding baby food is reflected in the pattern of changes over time in the head and mouth movements of the infant eating the baby food. In other words, when baby food is fed in an appropriate manner, the head and mouth movements change over time in a predetermined pattern, and if the feeding method is inappropriate, the changes over time in the head and mouth movements will differ from the predetermined pattern. Therefore, with the above configuration, the method of feeding baby food can be appropriately evaluated.

[0187] (Appendix 3) In the information processing device of Supplementary Note 1 or Supplementary Note 2, the detection unit may detect the chewing action based on changes over time in the positions of each feature point on the face of an infant eating baby food, and analyze the magnitude of vertical and horizontal movement of the infant's mouth during the chewing action, as well as the chewing cycle, and the evaluation and determination unit may determine the progress stage of the infant's baby food based on at least one of the magnitude of the movement and the chewing cycle.

[0188] The baby's progress in weaning food is determined in stages as the baby's physical growth and feeding function improve, and the progress is reflected in the chewing behavior. Therefore, the above configuration makes it possible to appropriately determine the progress of the baby's progress in weaning food.

[0189] (Appendix 4) A control method for an information processing device (1) according to one embodiment of the present disclosure includes an image acquisition step (S11, S31) of acquiring video images of an infant eating baby food, a detection step (S12, S32) of detecting from the video images the feeding behavior of the infant when taking in baby food or the chewing behavior of the infant until swallowing the baby food that the infant has taken in their mouth, an evaluation and judgment step (S14 to S17, S34 to S41) of evaluating or judging the way the infant eats or is being fed based on the analysis results of the detected feeding behavior or chewing behavior, and an output control step (S18, S42) of outputting feedback information generated based on the results of the evaluation or judgment.

[0190] (Appendix 5) A control program for causing a computer to function as the information processing device described in Appendix 1, which causes a computer to function as the image acquisition unit, the detection unit, the evaluation and judgment unit, and the output control unit, is also included in the technical scope of the present disclosure.

[0191] 〔effect〕 The configuration of the present disclosure can promote the development of motor organs related to oral function in growing infants and enable early detection of bad oral habits. Chewing food, swallowing behavior, saliva secretion, and other activities have a significant impact on the brain and the entire body, and are factors that significantly affect physical and mental health and healthy lifespan. Therefore, the configuration of the present disclosure is expected to contribute to maintaining the physical and mental health of infants and young children, promoting their development, and preventing oral dysfunction caused by bad oral habits. This will ultimately contribute to achieving Goal 3 of the Sustainable Development Goals (SDGs), "Good health and well-being for all."

[0192] [Software implementation example] The functions of the evaluation system 100 or the information processing device 1 (hereinafter referred to as the "device") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (particularly each part included in the control unit 10).

[0193] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0194] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0195] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0196] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0197] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0198] 1. Information processing equipment 2 Display device (output device) 3. Imaging equipment (monitoring equipment) 10 Control unit (control device) 11 Storage section 100 rating system 101 Image acquisition unit 102 Detection unit 103 Evaluation and Judgment Department 105 Output control section

Claims

1. a monitoring device for monitoring an infant eating baby food; an information processing device that uses the data obtained by the monitoring to evaluate or judge how the infant eats or is fed; an output device that outputs feedback information generated based on the result of the evaluation or judgment, The information processing device identifies the pattern of change over time in the head and mouth movements of the infant while eating baby food, and evaluates the way in which the infant is fed baby food based on the difference between the identified pattern and the pattern of change over time in the movements when the infant is fed in an appropriate manner.

2. The information processing device is configured to: When the baby opens and closes its mouth to take in baby food, if the head moves forward and then rotates downward, the feeding method is evaluated as appropriate. The evaluation system according to claim 1 , wherein if the change is not observed, the feeding method is evaluated as being inappropriate.

3. the monitoring device is an imaging device that images the infant eating baby food, the information processing device analyzes the captured video and evaluates or judges how the infant eats or is being fed, The information processing device includes: The evaluation system according to claim 2 , wherein the opening and closing movements of the mouth are identified based on a color change in an inner region of the baby's lips included in the moving image.

4. The evaluation system according to claim 1 , wherein the information processing device identifies a chewing behavior pattern of the infant and determines a progress stage of the infant's weaning food from the identified behavior pattern.

5. A monitoring device for monitoring an infant eating baby food; an information processing device that uses the data obtained by the monitoring to evaluate or judge how the infant eats or is fed; an output device that outputs feedback information generated based on the result of the evaluation or judgment, the information processing device identifies a chewing behavior of the infant and determines a progress stage of the infant's weaning food from the identified behavior; the monitoring device is an imaging device that images the infant eating baby food, the information processing device analyzes the captured video and evaluates or judges how the infant eats or is being fed, The information processing device includes: An evaluation system that monitors the distance between the apex of the infant's upper lip and the apex of the infant's lower lip contained in the video, and excludes from the chewing action a series of mouth opening and closing actions that includes a mouth opening action in which the distance is equal to or greater than a predetermined threshold.

6. The information processing device includes: monitoring the distance of the infant during the chewing activity; The evaluation system according to claim 5 , wherein, when the shortest distance among the distances is equal to or greater than a predetermined threshold, a state in which the inner sides of the upper and lower lips remain open during mastication is identified.

7. The evaluation system according to claim 4 , wherein the information processing device determines the progress stage of the baby's weaning food by taking into account at least one of the baby's age in months and the state of tooth eruption.

8. The information processing device includes:

8. The evaluation system according to claim 4, wherein the feedback information indicates whether or not the baby's weaning food progress stage predicted in advance by a user is different from the baby's weaning food progress stage determined from the identified chewing behavior.

9. The information processing device includes:

9. The evaluation system according to claim 1, wherein data obtained by monitoring an infant eating baby food is used as an explanatory variable, and the evaluation results of how the infant eats or is fed the baby food, or the judgment results of the infant's baby food progress stage, are used as a dependent variable, and the evaluation or judgment is performed using an evaluation model that has learned the relationship between the explanatory variables and the dependent variable.

10. a monitoring device for monitoring an infant eating baby food; a control device that uses the data obtained by the monitoring to evaluate or judge how the infant eats or is fed; an output device that outputs feedback information generated based on the result of the evaluation or judgment, The control device is an information processing device that identifies patterns of change over time in the head and mouth movements of the infant while it is eating baby food, and evaluates how the infant is being fed baby food based on the difference between the identified patterns and the patterns of change over time in the movements when the infant is being fed in an appropriate manner.

11. An evaluation method executed by an evaluation system including a monitoring device, an information processing device, and an output device, a monitoring step in which the monitoring device monitors an infant eating baby food; an evaluation and judgment step in which the information processing device evaluates or judges how the infant eats or is fed using the data obtained by the monitoring; an output step in which the output device outputs feedback information generated based on a result of the evaluation or judgment, In the evaluation and judgment step, the information processing device identifies a pattern of change over time in the head and mouth movements of the infant while eating baby food, and evaluates the way in which the infant is fed baby food based on the difference between the identified pattern and the pattern of change over time in the movements when the infant is fed in an appropriate manner.

12. A control program for causing a computer to function as the information processing device according to claim 1.

Images