LIFELOG PROVIDING SYSTEM AND LIFELOG PROVIDING METHOD

Abstract

To make it possible to provide a user with a life log in which a child's growth status can be grasped by selecting a child to be allowed to be viewed.SOLUTION: It is configured to: detect a specific event related to a level of growth of a child in images captured by a camera 1, by using image recognition; extract a scene image including the detected specific event, from the images captured by the camera; select a scene image that is allowed to be viewed on the basis of child's identification information included in login information of a user device 5; and generate, as a life log, a growth map in which a thumbnail of the scene image is overlaid on a map image including at least a timeline of child growth so as to correspond to date and time of the detection of the specific event related to the scene image.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a life log providing system and a life log providing method that provide a user with images of children in a facility taken by a camera as a life log. [Background technology]

[0002] In recent years, due to an increase in dual-income households, children are often placed in daycare centers from a young age, and parents and other guardians have fewer opportunities to see their children's "opportunities for growth." This causes parents to hesitate to place their children in daycare centers from a young age, to regret missing opportunities for growth, and to be dissatisfied with the fact that they can only check on their children's growth through childcare workers, and technology that can eliminate such dissatisfaction is desirable.

[0003] In response to such demands, there is a known technology that recognizes memorable scenes of a child from images captured by a camera, such as the child's smile on that day, the child's first time standing, the child's first time walking, etc., and extracts and outputs images of the child's memorable scenes from the images captured by the camera (see Patent Document 1). This technology allows parents to view memorable scenes of their child that they cannot see directly, and can alleviate the parents' frustration. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-125870 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the conventional technology, images of impressive scenes of a child can be presented to a guardian, but since whether an image is impressive or not depends on the guardian's subjective opinion, the presented images are not necessarily what the guardian wants. Also, from the guardian's point of view, the child's growth status can only be confirmed through the childcare worker. For this reason, when presenting images of a child taken at a childcare facility to a guardian as a life log of the child, it is desirable to be able to select a child to be allowed to view and to understand the growth status.

[0006] Therefore, a main object of the present invention is to provide a life log providing system and a life log providing method that can provide a user with a life log that allows the user to select which children are allowed to view the life log and to understand the growth status of the children. [Means for solving the problem]

[0007] The life log providing system of the present invention is a life log providing system in which a processor executes a process of providing images of a child in a facility taken by a camera to a user device as a life log, wherein the processor detects specific events related to the child's growth status by image recognition of the images taken by the camera, extracts scene images in which the detected specific events are captured from the images taken by the camera, selects the scene images that are permitted to be viewed based on the child's identification information included in the login information of the user device, and generates a growth map as the life log in which thumbnails of the scene images are superimposed on a map image including at least a timeline of the child's growth so as to correspond to the date and time of detection of the specific event related to the scene image.

[0008] In addition, the life log providing method of the present invention is a method for providing a life log to a user device by using a processor to execute a process of providing images of a child in a facility taken by a camera as a life log, in which the processor detects a specific event related to the child's growth status by image recognition of the images taken by the camera, extracts a scene image in which the detected specific event is captured from the images taken by the camera, selects the scene image that is permitted to be viewed based on the child's identification information included in the login information of the user device, and generates a growth map as the life log in which thumbnails of the scene images are superimposed on a map image including at least a time axis of the child's growth so as to correspond to the date and time of detection of the specific event related to the scene image. Effect of the Invention

[0009] According to the present invention, scene images that are permitted to be viewed can be selected based on the child's identification information included in the login information of the user device. This makes it possible to provide a life log that allows a user, such as a guardian, to grasp the developmental status of a child related to the user. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is an overall configuration diagram of a life log providing system according to an embodiment of the present invention. [Diagram 2] An explanatory diagram showing an overview of the life log provision system [Diagram 3] FIG. 13 is an explanatory diagram showing a transition state of a screen displayed on a user terminal 5. [Figure 4] FIG. 13 is an explanatory diagram showing a growth map screen displayed on a user terminal 5. [Diagram 5] A block diagram showing the schematic configuration of an edge computer 3 and a cloud computer 4. [Figure 6] FIG. 1 is an explanatory diagram showing management information processed by a cloud computer 4. [Figure 7] Flow diagram showing the procedure of processing performed by edge computer 3 [Figure 8] A flow diagram showing the procedure of face matching processing performed by the cloud computer 4. [Figure 9] A flow diagram showing the procedures of the login process, the growth map generation process, and the distribution process performed by the cloud computer 4. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The first invention made to solve the above problem is a life log providing system in which a processor executes a process of providing images taken by a camera of a child in a facility to a user device as a life log, wherein the processor detects specific events related to the child's growth status by image recognition of the images taken by the camera, extracts scene images in which the detected specific events are captured from the images taken by the camera, selects the scene images that are permitted to be viewed based on the child's identification information included in the login information of the user device, and generates a growth map as the life log in which thumbnails of the scene images are superimposed on a map image including at least a timeline of the child's growth so as to correspond to the date and time of detection of the specific event related to the scene image.

[0012] According to this, scene images that are permitted to be viewed can be selected based on the child's identification information included in the login information of the user device. This makes it possible to provide a life log that allows a parent or other user to grasp the developmental status of the child.

[0013] In addition, a second invention includes an edge computer installed in a facility and a cloud computer connected to the edge computer via a network, wherein a first processor provided in the edge computer performs a process of detecting the specific event and a process of extracting the scene image, and transmits the scene image to the cloud computer, and a second processor provided in the cloud computer generates the growth map based on the scene image received from the edge computer, and distributes the growth map to a user device.

[0014] This makes it possible to reduce the amount of data sent from the edge computer to the cloud computer, thereby reducing the communication load.

[0015] In a third invention, the processor detects the specific event by image recognition processing including at least one of a skeleton detection processing, a behavior recognition processing, and a facial expression estimation processing.

[0016] This allows specific events to be detected with high accuracy.

[0017] In addition, a fourth invention is configured such that the processor displays, in response to a user operation of selecting the thumbnail, time information regarding the date and time when a specific event corresponding to the selected thumbnail occurred on the user device.

[0018] This allows the user to easily check the date and time when a particular event that caught the user's attention occurred.

[0019] In a fifth aspect of the present invention, the processor is configured to reproduce, in the user device, the scene image corresponding to the selected thumbnail, in response to a user operation of selecting the thumbnail.

[0020] This allows the user to easily view scene images relating to a particular event that has caught the user's attention.

[0021] In a sixth aspect of the present invention, the processor is configured to register a specific event as a favorite in response to a user's operation to instruct registration of the event as a favorite.

[0022] According to this, by registering a particular event that the user finds interesting as a favorite, the user can easily view scene images of the particular event repeatedly later.

[0023] In a seventh aspect of the present invention, the processor is configured to display, in response to a user operation to instruct viewing of favorites, a list of information related to specific events registered in favorites on the user device.

[0024] This allows the user to easily check information about specific events registered in the favorites on the user device. The information about the specific events displayed in the list includes, for example, the name of the specific event, the date and time when the specific event occurred, and the age of the target child (months after birth).

[0025] In addition, an eighth invention is a life log providing method in which a processor executes a process of providing images of a child in a facility taken by a camera to a user device as a life log, wherein the processor detects a specific event related to the child's growth status by image recognition of the images taken by the camera, extracts scene images in which the detected specific event is captured from the images taken by the camera, selects the scene images that are permitted to be viewed based on the child's identification information included in the login information of the user device, and generates a growth map as the life log in which thumbnails of the scene images are superimposed on a map image including at least a time axis of the child's growth so as to correspond to the date and time of detection of the specific event related to the scene image.

[0026] According to this, like the first invention, it is possible to provide a life log that enables a user such as a guardian to grasp the growth status of a child related to the user.

[0027] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0028] Fig. 1 is a diagram showing the overall configuration of a lifelog providing system according to this embodiment, and Fig. 2 is a diagram showing an overview of the lifelog providing system.

[0029] This life log providing system provides images taken of children (infants) left at a childcare facility such as a nursery school as a life log to users such as guardians such as parents who leave their children at the childcare facility and facility staff such as childcare workers involved in childcare work at the childcare facility, and is equipped with a camera 1, a recorder 2, an edge computer 3, a cloud computer 4, and a user terminal 5 (user device).

[0030] A camera 1, a recorder 2, and an edge computer 3 are installed in a childcare facility. The camera 1, the recorder 2, and the edge computer 3 are connected via a network such as a LAN. The edge computer 3, the cloud computer 4, and the user terminal 5 are connected via a network such as the Internet.

[0031] The camera 1 takes pictures of the inside of the childcare facility. This camera 1 constantly takes pictures of the daily life of children in the childcare facility.

[0032] The recorder 2 accumulates (records) images captured by the camera 1.

[0033] The edge computer 3 acquires images captured by the camera 1 from the recorder 2, detects specific events related to the child's growth status by image recognition of the captured images, extracts scene images including the specific events from the images captured by the camera 1 based on the detection results, and transmits the scene images and information such as the event ID and the detection date and time of the specific event to the cloud computer 4. Here, a specific event is an event that is a target (evaluation item) when judging the child's growth status, among various events that appear in the child (behavior, facial expression, state, etc.).

[0034] The cloud computer 4 identifies the child appearing in the scene image received from the edge computer 3 by face matching, and associates the preregistered information of the child with the scene image. The cloud computer 4 also generates a growth map that visualizes the child's growth status (degree of growth). The cloud computer 4 also manages logins to the system from the user terminal 5, and delivers the child's growth map and scene images related to the user to the user terminal 5 as a life log.

[0035] The user terminal 5 is a PC or a smartphone, and is operated by a guardian (such as a parent) or a facility staff member (such as a childcare worker) as a user. In this embodiment, it displays a growth map and scene images distributed as a life log from the cloud computer 4. This allows the guardian or facility staff member as a user to view the growth map and scene images of a child that the guardian is raising or a child that the facility staff member is in charge of.

[0036] In this embodiment, two information processing devices, an edge computer 3 and a cloud computer 4, are provided. However, the functions of the edge computer 3 and the cloud computer 4 may be realized by a single information processing device, that is, only one of the edge computer 3 and the cloud computer 4 may be provided.

[0037] In addition, in this embodiment, a scene image in which a specific event is captured is extracted from an image captured by a camera 1 installed in a childcare facility, but a scene image may also be extracted from an image captured by a smartphone or the like of a situation when going out to a park or the like.

[0038] In this embodiment, the edge computer 3 detects a specific event and extracts a scene image (video image) in which the specific event is captured from the images recorded in the recorder 2, but the scene image may be extracted by a facility staff member or the like performing an operation to select the scene image in which the specific event is captured. Also, the edge computer 3 may extract candidates for the scene image, and the facility staff member or the like may select the candidate for the scene image.

[0039] Next, a description will be given of the screens displayed on the user terminal 5. Fig. 3 is an explanatory diagram showing the transition status of the screens displayed on the user terminal 5.

[0040] When the user terminal 5 accesses the cloud computer 4, first, a login screen shown in Fig. 3(A) is displayed. When the user inputs a user ID and a password into the input units 11 and 12 on this login screen and operates the login button 13, the screen transitions to a person selection screen shown in Fig. 3(B).

[0041] The person selection screen shown in Fig. 3(B) displays person selection sections 15, 16 for each of a plurality of children. A person's image, name, and age in months are displayed in these person selection sections 15, 16. When the user performs an operation to select one of the plurality of person selection sections 15, 16 on this person selection screen, the screen transitions to a growth map screen shown in Fig. 3(C).

[0042] The person selection screen is displayed when a guardian or facility staff member who has multiple children at the childcare facility logs in. When a guardian who has only one child at the childcare facility logs in, the person selection screen is omitted. If the user is a guardian such as a parent, the children they are raising are displayed, and if the user is a facility staff member such as a childcare worker, the children they are in charge of are displayed.

[0043] A growth map 21 relating to the child selected by the user is displayed on the growth map screen shown in Fig. 3(C). A thumbnail 22 of a scene image showing the child's action corresponding to a specific event is displayed on this growth map 21. When the user performs an operation to select the thumbnail 22, the screen transitions to a video playback screen shown in Fig. 3(D). In addition, a favorites viewing mark 23 is displayed on the growth map screen. When the user operates the favorites viewing mark 23, the screen transitions to a favorites list screen shown in Fig. 3(E).

[0044] A video viewer 25 is displayed on the video playback screen shown in FIG. 3(D). In this video viewer 25, a scene image (video) relating to a specific event corresponding to a thumbnail 22 selected on the growth map is played. The video playback screen also displays the name of the specific event, the date and time when the specific event was detected (photographed date and time), and the child's age in months at the time when the specific event was detected (photographed time). The video playback screen also displays a favorite registration mark 26. When the user operates the favorite registration mark 26, a process of registering the corresponding specific event in favorites is performed.

[0045] The favorites list screen shown in Fig. 3(E) displays a list of information about specific events registered in favorites among specific events detected in the past regarding the target child. Specifically, the name of the specific event ("Event"), the date and time when the specific event was detected ("Date of occurrence"), and the child's age in months at the time when the specific event was detected are displayed. When the user performs an operation to select the name of a specific event on this favorites list screen, the screen transitions to a video playback screen shown in Fig. 3(D).

[0046] Next, a description will be given of the growth map screen displayed on the user terminal 5. FIG.

[0047] The growth map screen displays a growth map 21 that visualizes the child's growth status (growth level). In this growth map 21, thumbnails 22 of scene images showing the child's actions corresponding to specific events are superimposed on a map image 28.

[0048] The map image 28 has an item column 31 for specific events related to the development of motor skills ("Movement"), an item column 32 for specific events related to the development of dexterity ("Hand movements"), and an item column 33 for specific events related to mental development ("Mind, Language, and Sociality"), which are items related to the areas of specific events that are the subject of judgment regarding a child's growth status.

[0049] In the example shown in Figure 4, specific events related to the development of motor skills ("Movement") include actions such as sitting, standing up, rolling over, crawling, crawling, walking alone, etc. Additionally, specific events related to the development of dexterity ("Hand Movements") include actions such as shaking a rattle, shaking a rattle around, hitting objects (blocks) together with both hands, holding objects in both hands, and putting objects in and taking them out of a box. Additionally, specific events related to mental development ("Mind, Language, and Sociality") include actions such as enjoying playing peek-a-boo, saying bye-bye, and pointing.

[0050] In this embodiment, growth primarily refers to growth (development) in terms of physical ability and mental ability, but may also include growth in terms of body size, such as height and weight.

[0051] Furthermore, the map image 28 is provided with an item field 34 for age in months. This item field 34 for age in months indicates a time axis that is the standard for the child's growth.

[0052] Furthermore, in the map image 28, a standard period mark 35 (growth status index) indicating a standard period that serves as a guide for judging the growth status is drawn for each specific event.

[0053] In addition, the map image 28 displays an event detection mark 36 indicating the time when a specific event was detected (time when the image was taken) based on the child's age in months at the time when the specific event was detected (time when the image was taken). This event detection mark 36 is displayed at the position of the corresponding age in months on the standard period mark for each specific event. In addition, a thumbnail 22 of a scene image corresponding to the event detection mark 36 is displayed adjacent to the event detection mark 36. This allows the user, i.e., a guardian such as a parent or a facility staff member such as a childcare worker, to grasp the child's growth situation at a glance while comparing it with the standard growth situation, and to easily check whether the child is growing normally or not. Furthermore, information useful for future childcare and care can be obtained, and appropriate childcare and care can be provided according to the child's growth situation.

[0054] If the time point at which a specific event is detected for the target child falls outside the standard period, an event detection mark 36 and a thumbnail 22 are displayed to the left or right of the standard period mark 35. When the user performs an operation to select the thumbnail 22, the screen transitions to a video playback screen shown in FIG. 3(D).

[0055] Furthermore, when the user places the pointer (mouse over) on the thumbnail 22, a speech bubble 37 is displayed. In this speech bubble 37, a timestamp of the specific event corresponding to the thumbnail 22, that is, time information regarding the date and time when the specific event occurred, is displayed.

[0056] In addition, the growth map screen is provided with a scroll button 38. When the user operates this scroll button 38, the growth map 21 is scrolled horizontally, and the growth map 21 of a month age that does not fit within one screen can be displayed. Note that a page forward button may be provided to switch to the growth map 21 of the next page.

[0057] The growth map screen also displays a favorites viewing mark 23. When the user operates the favorites viewing mark 23, the screen transitions to a favorites list screen shown in FIG.

[0058] Next, a description will be given of a schematic configuration of the edge computer 3 and the cloud computer 4. Fig. 5 is a block diagram showing a schematic configuration of the edge computer 3 and the cloud computer 4. Fig. 6 is an explanatory diagram showing management information processed by the cloud computer 4.

[0059] The edge computer 3 includes a communication unit 51, a storage unit 52, and a processor 53 (first processor).

[0060] The communication unit 51 communicates with the recorder 2 via the network. In this embodiment, the communication unit 51 receives images captured by the camera 1 and stored in the recorder 2 from the recorder 2. The communication unit 51 also communicates with the cloud computer 4 via the network. In this embodiment, the communication unit 51 transmits images generated by the processor 53 to the cloud computer 4.

[0061] The storage unit 52 stores programs executed by the processor 53 and the like.

[0062] The processor 53 performs various processes related to providing a life log by executing the programs stored in the storage unit 52. In this embodiment, the processor 53 performs a specific event detection process, a scene image extraction process, and the like.

[0063] In the specific event detection process, the processor 53 performs image recognition processing on the images captured by the camera 1 stored in the recorder 2, and detects specific events related to the child's growth status based on the recognition results. In the image recognition process, at least one of skeletal detection processing, behavior recognition processing, and facial expression estimation processing is performed. The skeletal detection processing makes it possible to recognize the movements of each part of the child. The behavior recognition processing makes it possible to recognize the behavior of the child. The facial expression estimation processing makes it possible to recognize the child's facial expressions, such as a smile.

[0064] This specific event detection process can be performed using a recognition model constructed by machine learning (deep learning, etc.). In addition, the image recognition process recognizes people and objects around the target child. For example, when detecting the behavior of shaking a rattle, the object held by the person is recognized. When detecting the behavior of enjoying playing peek-a-boo, the person playing peek-a-boo (the nursery teacher) is recognized.

[0065] In the scene image extraction process, the processor 53 extracts a scene image (video image) in which a detected specific event is captured from the images (video images) captured by the camera 1 and stored in the recorder 2, based on the detection results of the specific event detection process.

[0066] The scene image extracted by this scene image extraction process is transmitted to the cloud computer 4. In addition, as specific event detection result information, the detection date and time of the specific event, the recording time of the scene image, the camera ID of the camera 1 that captured the scene image, the event ID of the detected specific event, and the event detection score (a score indicating the likelihood of the detected specific event) are transmitted to the cloud computer 4.

[0067] In addition to extracting video images during a period in which a specific event was captured, the scene image extraction process may also extract a person image, i.e., an image area in which a target person appears, from the image captured by camera 1. Specifically, a detection frame for the person or a rectangular area including the detection frame is extracted.

[0068] The cloud computer 4 includes a communication unit 61, a storage unit 62, and a processor 63 (second processor).

[0069] The communication unit 61 communicates with the edge computer 3 and the user terminal 5 via the network.

[0070] The storage unit 62 stores a program executed by the processor 63, etc. The storage unit 62 also accumulates scene images received from the edge computer 3. The storage unit 62 also accumulates management information. Note that a large-capacity storage device such as a hard disk may be provided to accumulate the scene images and management information.

[0071] The processor 63 performs various processes related to the provision of a life log by executing the programs stored in the storage unit 62. In this embodiment, the processor 63 performs face matching processing, login management processing, growth map generation processing, distribution processing, and the like.

[0072] In the face matching process, the processor 63 identifies a person appearing in a scene image received from the edge computer 3, that is, a person who is the subject of a specific event. Specifically, the processor 63 extracts facial feature information of a child from the scene image, compares the facial feature information of the child appearing in the scene image with facial feature information for each child previously registered in the person management information stored in the storage unit 62, obtains a face matching score, and identifies a person whose face matching score is equal to or greater than a predetermined threshold as a person appearing in the scene image. Based on the result of this face matching process, the person appearing in the scene image can be associated with previously registered person management information (person ID, name, date of birth). Specifically, the person ID and face matching score obtained in the face matching process are stored in the storage unit 62 as specific event detection result information.

[0073] In the login management process, the processor 63 performs login judgment (user authentication) based on the login management information stored in the storage unit 62. If the login is successful, that is, if the person who has requested the login is judged to be a legitimate user, the growth map 21 and scene images are permitted to be viewed. The login management information also includes the number of children (number of person IDs) and person IDs permitted to view the growth map 21 and scene images, and a person selection screen (see FIG. 3(B)) is generated based on this information.

[0074] In the growth map generation process, the processor 63 generates a growth map 21 for a child related to the logged-in user (guardian or facility staff) and selected by the user. At this time, a map image 28 (see FIG. 4) is created based on the event field management information stored in the storage unit 62. Specifically, item columns 31, 32, and 33 for each of the three fields of the specific event are drawn. In addition, a standard period mark 35 (see FIG. 4) is drawn based on the specific event management information (standard start month age, standard end month age) stored in the storage unit 62. In addition, the month age (year, month, day) of the child at the time of detection is calculated based on the detection date and time of the specific event included in the specific event detection result information and the birth date of the person included in the person management information. Then, a thumbnail 22 is arranged on the map image 28 based on the month age of the child at the time of detection.

[0075] In the distribution process, the processor 63 distributes the growth map 21 generated in the growth map generation process to the user terminal 5 in response to a user instruction operation on the user terminal 5, and causes the growth map 21 to be displayed on the user terminal 5. Also, in response to a user instruction operation on the user terminal 5, the processor 63 distributes a scene image (moving image) to the user terminal 5, and causes the user terminal 5 to play back the scene image.

[0076] In addition, processor 63 manages the favorite state of specific events that occurred in the past for each child (favorite management process). In this favorite management process, information regarding the favorite state is stored in memory unit 62 in association with specific event detection result information and face matching result information. When the user operates favorite registration mark 26 (see FIG. 3(D)), a process is performed to register the corresponding specific event in the favorites. When the user operates favorite viewing mark 23 (see FIG. 3(C)), a process is performed to display a favorite list screen (FIG. 3(E)) based on information regarding the favorite state in memory unit 62.

[0077] Next, a description will be given of the processing performed by the edge computer 3. FIG 7 is a flow diagram showing the procedure of the processing performed by the edge computer 3.

[0078] In the edge computer 3, the processor 53 first acquires the captured image of the camera 1 recorded in the recorder 2 (ST101). Then, the processor 53 recognizes the child's actions from the captured image of the camera 1 and generates action information representing the actions of each child (action recognition process) (ST102). Next, the processor 53 performs specific event detection process and scene image extraction process for all specific events (ST103 to ST113). Specifically, the processor 53 judges whether the captured image of the detected action corresponds to the specific event for each frame in sequence, and performs a process of linking the corresponding frames (usually several tens of frames) with an event ID and registering the extracted information in the detected event list. Then, when the captured image no longer corresponds to the specific event, the processor 53 judges based on the event ID whether the extracted information of the specific event has been registered in the detected event list in the past, and performs a process of integrating the extracted information (scene image) that has reached the time limit.

[0079] In this process, first, processor 53 determines whether or not a certain child's movement recognized in the movement recognition process corresponds to a specific event of interest (movement determination process) (ST104).

[0080] Here, if the detected action corresponds to a specific event, i.e., if a specific event has been detected (Yes in ST104), processor 53 then determines whether the specific event being detected has an event ID that is not registered in the detected event list, i.e., whether the specific event has been newly detected (ST105).

[0081] Here, if the detected specific event has an unregistered event ID (Yes in ST105), processor 53 registers new extraction information in the detected event list with a captured image showing a child's behavior corresponding to the specific event as a scene image (ST106).On the other hand, if the detected specific event has an event ID already registered in the detected event list (No in ST105), processor 53 updates (adds) the extraction information with a captured image showing a child's behavior corresponding to the specific event as a scene image (ST107).

[0082] On the other hand, if the detected action does not correspond to the specific event, i.e., if the specific event is not detected (including the case where the specific event has ended) (No in ST104), then processor 53 determines whether the specific event has been registered in the detected event list (ST108).

[0083] Here, if the detected specific event has already been registered in the event list (Yes in ST108), processor 53 then determines whether the registration time of the extracted information, i.e., the total recording time of the scene images (moving images) registered as the extracted information, has reached a predetermined time limit based on the event ID (registration time determination process) (ST109).

[0084] Here, if the registration time reaches the time limit (Yes in ST109), the processor 53 then integrates the multiple scene images registered as extracted information (ST110). The communication unit 51 then transmits the integrated scene image to the cloud computer 4 together with the event ID of the specific event in which the scene image appears (ST111). The extracted information linked to the event ID of the specific event is then deleted from the detected event list (ST112).

[0085] On the other hand, if the specific event is not registered in the detected event list (No in ST108) or if the registration time has not reached the limit (No in ST109), no particular process is performed and the process proceeds to the process for the next specific event.

[0086] Next, a description will be given of the face matching process performed by the cloud computer 4. FIG 8 is a flow diagram showing the procedure of the face matching process performed by the cloud computer 4.

[0087] In the cloud computer 4, first, the communication unit 61 receives the scene image from the edge computer 3 (ST201). Next, the processor 63 performs face matching processing for identifying children appearing in the scene image for all registered children (ST202 to ST208).

[0088] In this process, first, the processor 63 extracts facial feature information of the child from the scene image, and compares the facial feature information of the child in the scene image with facial feature information for each child registered in advance to obtain a face matching score (ST203).Then, the processor 63 determines whether the face matching score is equal to or greater than a predetermined threshold (face matching score determination) (ST204).

[0089] If the matching score is equal to or greater than the threshold (Yes in ST204), face matching result information including a person ID and face matching score is generated (ST206).On the other hand, if the matching score is less than the threshold (No in ST204), it is determined that there is no corresponding person, and face matching result information not including a person ID is generated (ST205).

[0090] Next, the processor 63 stores the registration information in the storage unit 62 as specific event detection result information (ST207).

[0091] Next, a description will be given of the login process, the growth map generation process, and the distribution process performed by the cloud computer 4. Fig. 9 is a flow diagram showing the procedures of the login process, the growth map generation process, and the distribution process performed by the cloud computer 4.

[0092] In the cloud computer 4, first, the processor 63 causes the user terminal 5 to display a login screen in response to a viewing request from the user terminal 5 (ST301). Next, in the user terminal 5, the user inputs login information (ID and password) to perform a login operation, and when the communication unit 61 receives the login request from the user terminal 5, the processor 63 verifies the login information and determines whether the login is successful, i.e., whether the user is a legitimate user (ST302).

[0093] Here, if the login is successful (Yes in ST302), the processor 63 causes the user terminal 5 to display a person selection screen (ST303). Next, when the user performs an operation to select a person on the user terminal 5, the processor 63 acquires specific event detection result information related to the selected person from the storage unit 62 (ST304). Next, the processor 63 generates a growth map 21 related to the selected person based on the specific event detection result information related to the selected person (ST305). Next, the processor 63 distributes the growth map 21 to the user terminal 5 and causes it to be displayed on the user terminal 5 (ST306).

[0094] Next, when the user performs an operation to select a thumbnail 22 on the growth map screen displayed on the user terminal 5 (Yes in ST307), the processor 63 acquires the event ID of the specific event corresponding to the thumbnail 22 selected by the user (ST308). Then, the processor 63 delivers a scene image (moving image) corresponding to the event ID to the user terminal 5, and causes the user terminal 5 to play back the scene image (ST309).

[0095] Furthermore, when the user performs a logout operation on the user terminal 5, the communication section 61 receives a logout request from the user terminal 5 (ST310), and performs a logout process (ST311).

[0096] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the above embodiments to create new embodiments. [Industrial Applicability]

[0097] The life log providing system and life log providing method of the present invention have the effect of being able to provide a user with a life log that allows them to systematically understand their child's growth status, and are useful as a life log providing system and life log providing method that provide a user with images taken by a camera of a child within a facility as a life log. [Explanation of symbols]

[0098] 1 Camera 2. Recorder 3. Edge Computers 4. Cloud Computer 5 User terminal (user device) 21 Growth Map 22 Thumbnails 23 Favorite View Mark 25 Map Images 26 Favorites mark 31, 32, 33 Specific event items 34 Age in months 35 Standard Period Mark 36 Event detection mark 37 Speech bubble 38 Scroll button 51 Communications Department 52 Storage section 53 Processors 61 Communications Department 62 Storage section 63 processors

Claims

1. A life log providing system that executes a process of providing an image of a child in a facility taken by a camera to a user device as a life log, The processor, Detecting specific events related to the child's growth status through image recognition of the images captured by the camera; Extracting a scene image in which the detected specific event is captured from the image captured by the camera; selecting the scene image that is permitted to be viewed based on the child's identification information included in the login information of the user device; A life log providing system characterized by generating a growth map as the life log, in which thumbnails of the scene images are superimposed on a map image including at least a time axis of a child's growth so as to correspond to the date and time of detection of the specific event related to the scene image.

2. An edge computer installed in the facility, a cloud computer connected to the edge computer via a network, A first processor provided in the edge computer, performing a process of detecting the specific event and a process of extracting the scene image, and transmitting the scene image to the cloud computer; A second processor provided in the cloud computer, The life log providing system according to claim 1 , further comprising: generating the growth map based on the scene image received from the edge computer; and distributing the growth map to a user device.

3. The processor, 2. The life log providing system according to claim 1, wherein the specific event is detected by image recognition processing including at least one of a skeleton detection processing, a behavior recognition processing, and a facial expression estimation processing.

4. The processor, The life log providing system according to claim 1, characterized in that, in response to a user's operation of selecting a thumbnail, time information regarding the date and time when a specific event corresponding to the selected thumbnail occurred is displayed on the user device.

5. The processor, 2. The life log providing system according to claim 1, wherein the scene image corresponding to the selected thumbnail is reproduced on the user device in response to a user operation of selecting the thumbnail.

6. The processor, 2. The life log providing system according to claim 1, wherein a specific event of interest is registered in the favorites in response to a user's operation to instruct registration of the favorites.

7. The processor, 2. The life log providing system according to claim 1, wherein information about specific events registered in favorites is displayed in a list on the user device in response to a user operation to instruct viewing of favorites.

8. A life log providing method for providing an image of a child in a facility taken by a camera to a user device as a life log by a processor, comprising: The processor, Detecting specific events related to the child's growth status through image recognition of the images captured by the camera; Extracting a scene image in which the detected specific event is captured from the image captured by the camera; selecting the scene image that is permitted to be viewed based on the child's identification information included in the login information of the user device; A method for providing a life log, characterized in generating, as the life log, a growth map in which thumbnails of the scene images are superimposed on a map image including at least a time axis of a child's growth so as to correspond to the date and time of detection of the specific event related to the scene image.

Images