Information processing device, information processing method, and recording medium

Abstract

Provided are an information processing device, an information processing method, and a recording medium which make it possible to improve the efficiency of processing of monitoring a plurality of persons present in a moving body.　This information processing device comprises: a person detection means for detecting persons in an image captured by a camera that captures the inside of a moving body; a selection means for selecting, on the basis of a prescribed rule, a person to be analyzed from among the persons in the image; an estimation means for analyzing a region in which the person to be analyzed appears and estimating the health condition of the person; and a transmission means for transmitting the image to a prescribed image analysis server.

Description

Information Processing Apparatus, Information Processing Method, and Recording Medium

[0001] The present invention relates to an information processing apparatus, an information processing method, and a recording medium.

[0002] Patent Document 1 discloses an in-vehicle device that can determine a wearable device possessed by a driver when a plurality of wearable devices are brought into a vehicle. The in-vehicle device described in this document detects the heart rate of the driver from the image data transmitted from an in-vehicle camera that photographs the driver. Then, the in-vehicle device compares the biometric information transmitted from one or more wearable devices possessed by one or more persons who have boarded the vehicle with the heart rate of the driver, and identifies the wearable device possessed by the driver.

[0003] Japanese Unexamined Patent Application Publication No. 2017-***

[0004] The in-vehicle device of Patent Document 1 has a problem that monitoring of other passengers is neglected on the premise of monitoring the driver. On the other hand, when trying to monitor a plurality of passengers simultaneously, there is a problem that the processing load on the vehicle (in-vehicle device) side becomes large. <****10> An object of the present disclosure is to provide an information processing apparatus, an information processing method, and a recording medium that can contribute to improving the efficiency of monitoring processing of a plurality of persons in a moving body vehicle. <****12> According to a first aspect, there is provided an information processing apparatus including: a person detection unit that detects a person shown in an image captured by a camera that captures the inside of a moving body vehicle; a selection unit that selects a person to be analyzed from the persons shown in the image based on a predetermined rule; an estimation unit that analyzes a region in which the person to be analyzed is shown and estimates the health state of the person; and a transmission unit that transmits the image to a predetermined image analysis server.

[0007] Note: The specific number in and in the original text may be incorrect in the provided content. Here, they are retained as they are. Also, ,

[0001] ,

[0002] , ,

[0003] , ,

[0004] ,

[0005] , ,

[0006] , ,

[0007] are retained without translation as they seem to be some kind of identifiers. And for , the number part "220094" in "特開2017-220094号公報" is translated as "***" as it's not clear what the exact translation rule for this kind of patent number is in this context. If there are specific requirements for translating patent numbers, it should be adjusted accordingly.From a second perspective, an information processing method is provided which detects a person in an image taken by a camera that photographs the interior of a moving vehicle, selects a person to be analyzed from among the people in the image based on predetermined rules, analyzes the area in which the person to be analyzed is pictured, estimates the health status of the person, and transmits the image to a predetermined image analysis server.

[0008] From a third perspective, a recording medium is provided which contains a program that causes a computer to perform the following steps: a process of detecting a person in an image taken by a camera that photographs the interior of a moving vehicle; a process of selecting a person to be analyzed from among the people in the image based on predetermined rules; a process of analyzing the area in which the person to be analyzed is photographed and estimating the health status of the person; and a process of transmitting the image to a predetermined image analysis server.

[0009] This disclosure makes it possible to provide an information processing device, an information processing method, and a recording medium that can contribute to improving the efficiency of monitoring multiple people inside a moving vehicle.

[0010] This is a diagram showing one configuration of this disclosure. This is a flowchart showing the operation of this disclosure. This is a diagram for explaining the operation of this disclosure. This is a diagram showing one configuration of this disclosure. This is a functional block diagram showing the configuration of the in-vehicle terminal of this disclosure. This is a sequence diagram for explaining the operation of this disclosure. This is a diagram for explaining the operation of this disclosure. This is a diagram showing one configuration of this disclosure. This is a diagram showing an example of personal information referenced by the in-vehicle terminal of this disclosure. This is another sequence diagram for explaining the operation of this disclosure. This is a diagram showing another example of personal information referenced by the in-vehicle terminal of this disclosure. This is a diagram showing one configuration of this disclosure. This is another sequence diagram for explaining the operation of this disclosure. This is a diagram showing the configuration of the computer constituting the information processing device or in-vehicle terminal of this disclosure.

[0011] First, an overview of one embodiment of this disclosure will be described with reference to the drawings. In this disclosure, the drawings are associated with one or more embodiments. The reference numerals in the drawings appended to this overview are provided for convenience as examples to aid understanding and are not intended to limit this disclosure to the illustrated embodiments. In addition, the connecting lines between blocks in the drawings and other references referred to in the following description include both bidirectional and unidirectional lines. Unidirectional arrows schematically indicate the flow of the main signal (data) and do not exclude bidirectionality. The program is executed via a computer device, which includes, for example, a processor, a storage device, an input device, a communication interface, and a display device as needed. This computer device is also configured to communicate with devices (including computers) inside or outside the device via the communication interface, whether wired or wireless. In addition, there are ports or interfaces at the input / output connection points of each block in the figures, but these are omitted from the illustration.

[0012] In one embodiment, this disclosure can be implemented by an information processing device 10, as shown in Figure 1, which includes a person detection means 11, a selection means 12, an estimation means 13, and a transmission means 14. More specifically, the person detection means 11 detects a person in the image. The selection means 12 selects a person to be analyzed from among the people in the image based on predetermined rules. The estimation means 13 analyzes the area in which the person to be analyzed is pictured and estimates the person's health status. The transmission means 14 transmits an image captured by a camera C that photographs the interior of a mobile vehicle V to a predetermined image analysis server 20.

[0013] The information processing device 10 configured as described above operates as follows. First, the information processing device 10 detects a person in the image captured by the camera C that photographs the interior of the mobile vehicle V (step S01 in Figure 2).

[0014] Next, the information processing device 10 selects a person to be analyzed from among the people in the image based on predetermined rules (step S02 in Figure 2). Possible predetermined rules include rules based on seating, such as priority for the driver's seat or the rear seats, or rules based on age, gender, etc., estimated from the camera image.

[0015] Next, the information processing device 10 analyzes the area in which the person to be analyzed is pictured and estimates the person's health status (step S03 in Figure 2). If an abnormality is found as a result of the health status estimation, the information processing device 10 may output a predetermined value.

[0016] Furthermore, the information processing device 10 transmits the image to a predetermined image analysis server 20 (step S04 in Figure 2). The image analysis server 20 analyzes the image and estimates the health status of individuals not selected as analysis targets. If an abnormality is detected as a result of the health status estimation by the image analysis server 20, the image analysis server 20 may output a predetermined message.

[0017] Figure 3 is a diagram illustrating the operation of the present disclosure. As shown in Figure 3, the person detection means 11 of the information processing device 10 detects people sitting in the driver's seat and the rear seat in images captured by the camera C that photographs the interior of the mobile vehicle V. The selection means 12 then selects a person to be analyzed from among the detected people based on predetermined rules. In the example in Figure 3, the selection means 12 selects the person sitting in the driver's seat as the target of analysis. The estimation means 13 then analyzes the area in which the person sitting in the driver's seat is visible and estimates the health status of that person. In the example in Figure 3, the estimation means 13 estimates pulse rate, respiratory rate, etc., based on changes in the color of veins visible in the image of the person sitting in the driver's seat. For example, if an abnormality is found in the health status of the person sitting in the driver's seat, the information processing device 10 can output a warning message via image or sound, or take control such as emergency stopping of the mobile vehicle V.

[0018] As described above, by selecting the person to be analyzed and processing it on the information processing device side (edge ​​side), it is possible to deal with emergencies where it would be too late to request analysis from the image analysis server 20. In addition, the analysis of the person sitting in the back seat in Figure 3 will be performed by the image analysis server 20, which is not shown in Figure 3. Therefore, there will be no omissions in monitoring people in the image.

[0019] As explained above, this disclosure makes it possible to monitor multiple individuals inside a moving vehicle, and then select the highest priority individuals from among them for monitoring.

[0020] [First Embodiment] Next, a first embodiment will be described in which the image analysis processing is efficiently shared between an in-vehicle terminal and an image analysis server, as an example of an information processing device. Figure 4 is a diagram showing one configuration of the present disclosure. Referring to Figure 5, a configuration is shown that includes a vehicle V equipped with a camera C, an in-vehicle terminal 100 mounted on this vehicle, and an image analysis server 200 capable of receiving images from the in-vehicle terminal 100 via a network N. The in-vehicle terminal 100 and the image analysis server 200 share the task of analyzing images captured by the camera C.

[0021] The image analysis server 200 analyzes the images received from the in-vehicle terminal 100 and estimates the vital sign values ​​of the people in the images. The image analysis server 200 also transmits the estimation results to the in-vehicle terminal 100.

[0022] Figure 5 is a functional block diagram showing the configuration of the in-vehicle terminal 100 of this disclosure. Referring to Figure 5, the configuration includes a person detection unit 101, a selection unit 102, an estimation unit 103, a transmission unit 104, and a control unit 105.

[0023] The person detection unit 101 detects people in images captured by the camera C mounted on the vehicle V and sends them to the selection unit 102. The person detection unit 101 can use various image processing engines capable of extracting areas containing people from images, or large-scale language models (LLM) or VLM (Vision-Language Model) capable of performing image analysis. The person detection unit 101 corresponds to the person detection means 11 described above.

[0024] The selection unit 102 selects a person to be analyzed from among the people detected by the person detection unit 101. The following rules can be used as the selection rules for the person to be analyzed in the selection unit 102: (1) Selection based on seat (boarding position) Prioritize people in the driver's seat or conductor's position involved in the operation of the vehicle V. Alternatively, a rule could be considered to prioritize the seat directly behind the driver's seat, which is the seat of honor and where the guest of honor sits. (2) Selection based on clothing Select people involved in the operation of the vehicle V or people who are likely to have health problems based on the clothing of the crew. For example, prioritize people wearing a driver's uniform or a patient's gown. (3) Selection based on age and gender Prioritize the elderly and women. Alternatively, a rule could be considered to prioritize children. Age and gender can be determined from the features in the image. In addition, the above (1) to (3) may be combined to select the person to be analyzed. The selection unit 102 corresponds to the selection means 12 described above.

[0025] The estimation unit 103 estimates various vital sign values ​​from the area (image area) in which the person selected by the selection unit 102 is pictured, and outputs them to the control unit 105 as an indicator of the person's health status. Examples of vital sign values ​​include respiratory rate, pulse rate, and oxygen saturation (SpO2). Hereinafter, these will be collectively referred to as "vital information." Respiratory rate can be measured by extracting periodic, subtle movements from an image of a person over a certain period. Pulse rate can be measured by extracting changes in the brightness value of a specific color that appears in an image of a person's skin over a certain period. Oxygen saturation can be measured by extracting changes in a specific red component that appears in an image of a person's skin over a certain period. These are just examples of items measured as vital sign values, and other items can also be added to the measurement targets. For example, concentration level, stress level, alertness level, etc., can be added to the measurement targets based on facial expressions and eye movements. The estimation unit 103 described above corresponds to the estimation means 13 described above.

[0026] The transmitting unit 104 transmits images captured by the camera C mounted on the vehicle V to the image analysis server 200 and requests analysis. The transmitting unit 104 described above corresponds to the transmitting means 14 described above.

[0027] The control unit 105 performs various controls based on the vital sign values ​​(vital information) of the person estimated by the estimation unit 103. For example, if the person's respiratory rate, pulse rate, oxygen saturation (SpO2), etc., shows abnormal values, the control unit 105 outputs a warning message via the in-vehicle display device or speaker. Alternatively, if the person's respiratory rate, pulse rate, oxygen saturation (SpO2), etc., shows abnormal values, the control unit 105 may perform direct controls such as slowing down or stopping the vehicle. Alternatively, if the person's respiratory rate, pulse rate, oxygen saturation (SpO2), etc., shows abnormal values, the control unit 105 may perform controls to notify a predetermined recipient that an abnormality has occurred in the person inside the vehicle.

[0028] Next, the operation of this embodiment will be described in detail with reference to the drawings. Figure 6 is a sequence diagram illustrating the operation of this disclosure. Referring to Figure 6, first, the in-vehicle terminal 100 detects a person from the image captured by the camera C (step S001).

[0029] Next, the in-vehicle terminal 100 selects a person to be analyzed from among the detected persons using the predetermined rules mentioned above (step S002).

[0030] Next, the in-vehicle terminal 100 analyzes the image of the selected person and obtains vital information (step S003).

[0031] Next, the in-vehicle terminal 100 performs processing according to the acquired vital information (step S004). In parallel with the above series of processes, the in-vehicle terminal 100 transmits the images taken by the camera C mounted on the vehicle V to the image analysis server 200 and requests analysis (step S005).

[0032] Upon receiving the request to analyze the aforementioned image, the image analysis server 200 analyzes the image received from the in-vehicle terminal 100 and estimates the vital sign values ​​of the person in the image (step S006). Then, the image analysis server 200 transmits the analysis results to the in-vehicle terminal 100 (step S007).

[0033] Figure 7 is a diagram illustrating the operation of the present disclosure. Figure 7 shows an image taken by a camera C mounted inside a bus at a certain point in time. In the example in Figure 7, persons P1 to P3 are detected. Also, in Figure 7, person P1 on the near side is assumed to be sitting in the driver's seat. For example, if a rule is used that prioritizes persons in the driver's seat (driver), person P1 will be selected as the target of analysis. This makes it possible to quickly detect abnormalities in the driver's health condition. Also, for example, if a rule is used that prioritizes persons wearing crew (conductor) uniforms, person P2 will be selected as the target of analysis. This makes it possible to quickly detect abnormalities in the crew (conductor)'s health condition. Of course, both rules may be used to select both persons P1 and P2 as targets of analysis. Furthermore, the importance of persons P1 to P3 may be calculated, and persons with higher importance (scores) may be selected as targets of analysis.

[0034] As explained above, this disclosure makes it possible to monitor the priority of individuals captured by camera C in vehicle V and to take prompt action if any abnormalities occur.

[0035] [Second Embodiment] Next, a second embodiment will be described in which a person identification function is added to the in-vehicle terminal 100a. Figure 8 is a functional block diagram showing another configuration of the in-vehicle terminal 100a of this disclosure. The first difference from the in-vehicle terminal 100 of the first embodiment shown in Figure 5 is that the in-vehicle terminal 100a is equipped with a person identification unit 106 and a personal information storage unit 107. The second difference from the in-vehicle terminal 100 of the first embodiment is that the selection unit 102a selects a person by referring to personal information. The other configurations are almost the same as those of the first embodiment, so the differences in operation will be explained below.

[0036] The person identification unit 106 identifies the person in the image based on the facial image of the person in the image captured by camera C.

[0037] The personal information storage unit 107 stores personal information of individuals who may board vehicle V. Figure 9 shows an example of personal information set in the personal information storage unit 107. In the example in Figure 9, age group, presence or absence of pre-existing conditions, health status, etc., are registered for each individual (user). Such personal information can be prepared by setting it in advance or by accessing an external personal information management server and downloading the necessary personal information. Alternatively, this personal information may be downloaded after identifying a person who has boarded the vehicle, and deleted after the person has disembarked from the vehicle.

[0038] When a person is identified by the person identification unit 106, the selection unit 102a retrieves the personal information of that person from the personal information storage unit 107. Based on this personal information, the selection unit 102a selects a person to be analyzed from among the people detected by the person detection unit 101. For example, based on each person's health status, the selection unit 102a selects person CCC, who is currently receiving medical treatment, from among users AAA to CCC in Figure 9, as the person to be analyzed.

[0039] FIG. 11 shows an example of additional personal information stored in the personal information storage unit 107, where the normal value range of vital sign values for each person is stored. The control unit 105a refers to the normal value range of vital sign values for each person and determines whether the estimated vital sign value is abnormal. Generally, it is said that the average values of pulse and respiratory rate change depending on age and gender, and there are also individual differences as well.FIGURE 11 shows that by setting the normal range of pulse and respiratory rate for each individual, it is possible to more accurately determine the presence or absence of abnormalities in pulse and respiratory rate. In the example of FIG. 11, the normal values of vital sign values for each person are shown as ranges, but the normal values of vital sign values for each person may also be managed by median values or average values. In this case, the control unit 105a determines whether the estimated vital sign value is abnormal based on the degree to which the vital sign value of each person deviates from these statistical values.

[0040] Next, the operation of this embodiment will be described in detail with reference to the drawings. FIG. 11 is another sequence diagram for explaining the operation of the present disclosure. Referring to FIG. 11, first, the in-vehicle terminal 100a detects a person from the image captured by the camera C (step S001).

[0041] Next, the in-vehicle terminal 100a identifies the detected person (step S101). Next, the in-vehicle terminal 100a selects a person by referring to the personal information of the person who has been successfully identified (step S102).

[0042] Next, the in-vehicle terminal 100a analyzes the image of the selected person and acquires vital information (step S003).

[0043] Next, the in-vehicle terminal 100a refers to the normal value range of vital sign values for each person, determines whether the estimated vital sign value is abnormal, and performs processing according to the result (step S004).

[0044] According to this embodiment, the in-vehicle terminal 100a can be made to select a person based on the health state of each individual. Further, according to this embodiment, it is possible to accurately determine the presence or absence of abnormalities in vital sign values for each individual.

[0045] [Third Embodiment] Next, a third embodiment in which an arousal level estimation function is added in addition to vital information will be described. FIG. 12 is a functional block diagram showing another configuration of the in-vehicle terminal 100b of the present disclosure. The difference from the in-vehicle terminal 100a of the second embodiment shown in FIG. 8 is that an arousal level estimation unit 108 that estimates the arousal level of the person selected by the selection unit 102 is added, and the control unit 105b uses the arousal level in combination to determine whether there is an abnormality in the vital sign value. Since other configurations are substantially the same as those of the second embodiment, the following description will focus on the differences in their operations.

[0046] The arousal level estimation unit 108 estimates the arousal level of the person corresponding to the image of the person selected by the selection unit 102. This arousal level is a quantification of the degree to which a person is awake, and can be represented by, for example, values in three levels: high, medium, and low. Such an arousal level can be estimated, for example, based on the image of the eye part in the image of a person. For example, when a person's eyes tend to be closed and the movement of the pupils is small, the arousal level is "low". Also, for example, when a person's eyes are open and moving appropriately, the arousal level is "high". Note that similar indicators such as the eye opening rate can be used instead of the arousal level.

[0047] The estimation unit 103b outputs the arousal level estimated by the arousal level estimation unit 108 to the control unit 105b in addition to the vital information of each person.

[0048] The control unit 105b determines whether the estimated vital sign value is abnormal based on the arousal level in addition to the normal value range of the vital sign value of each person. For example, a person's vital sign value tends to be lower during sleep than in normal times. Therefore, when the control unit 105b estimates from the arousal level that the person to be analyzed is sleeping, it corrects the normal value range of the vital sign value shown in FIG. 11 and determines whether the estimated vital sign value is abnormal. Also, when the sleeping state is dangerous due to the past medical history of the corresponding person, etc., the control unit 105b determines that it is abnormal based on the arousal level regardless of the vital sign value.

[0049] Next, the operation of this embodiment will be described in detail with reference to the drawings. Figure 13 is another sequence diagram for illustrating the operation of this disclosure. The first difference from the sequence diagram of the second embodiment shown in Figure 11 is that in step S103, the in-vehicle terminal 100b acquires (estimates) the alertness level of the selected person. The second difference from the sequence diagram of the second embodiment is that in step S104, the in-vehicle terminal 100b determines whether the estimated vital sign values ​​are abnormal based on each person's vital sign values ​​and alertness level, and performs processing according to the result. Other operations are the same as in the second embodiment, so their description will be omitted.

[0050] According to this embodiment, the in-vehicle terminal 100b can use the level of alertness to more accurately determine whether or not there are abnormalities in vital sign values.

[0051] In the embodiment described above, the level of alertness is used to determine whether or not there are abnormalities in vital signs, but the level of alertness may also be used to select the person to be analyzed. For example, if a person's level of alertness is low, they may be unwell and may be selected as a priority for analysis. Conversely, if a person other than the driver has a low level of alertness, for example, if they are sleeping, they may be excluded from the analysis. This is because, in most cases, there is not a high urgency to assess the health status of a person who is sleeping.

[0052] While the embodiments of this disclosure have been described above, this disclosure is not limited to the embodiments described above, and further modifications, substitutions, and adjustments can be made without departing from the basic technical concept of this disclosure. For example, the network configurations, element configurations, and data representations shown in the drawings are examples to aid in understanding this disclosure and are not limited to the configurations shown in these drawings.

[0053] For example, in the first to third embodiments described above, the information processing device was described as an in-vehicle terminal 100 to 100b, but other devices may also be equipped with the functions of an information processing device. For example, a roadside device installed on the roadside may be equipped with the functions of the information processing device of this disclosure. In this case, the roadside device would acquire images of the inside of a vehicle and select a person to be analyzed from among the people in the images.

[0054] (Hardware Configuration) In each embodiment of this disclosure, each component of each device represents a functional unit block. Some or all of each component of each device is realized by any combination of an information processing device 900 and a program, for example, as shown in Figure 14. Figure 14 is a block diagram showing an example of the hardware configuration of the information processing device 900 that realizes each component of each device. The information processing device 900 includes, as an example, the following configuration: ・CPU (Central Processing Unit) 901 ・ROM (Read Only Memory) 902 ・RAM (Random Access Memory) 903 ・Program 904 loaded into RAM 903 ・Storage device 905 that stores the program 904 ・Drive device 907 that reads and writes to the recording medium 906 ・Communication interface 908 that connects to a communication network 909 ・Input / output interface 910 that performs data input and output ・Bus 911 that connects each component

[0055] Each component of each device in each embodiment is realized by the CPU 901 acquiring and executing a program 904 that realizes these functions. That is, the CPU 901 in Figure 14 executes a person detection program and an analysis target selection program, and performs update processing of each calculation parameter held in RAM 903, storage device 905, etc. The program 904 that realizes the functions of each component of each device is, for example, stored in storage device 905 or ROM 902 in advance and read by the CPU 901 as needed. The program 904 may be supplied to the CPU 901 via a communication network 909, or it may be stored in a recording medium 906 in advance, and the drive device 907 may read the program and supply it to the CPU 901.

[0056] There are various variations in how each device is implemented. For example, each device may be implemented by any combination of a separate information processing device 900 and a program for each component. Alternatively, multiple components of each device may be implemented by any combination of a single information processing device 900 and a program. That is, each part (processing means, function) of the information processing device described above can be implemented by a computer program that causes a processor mounted on the device to execute the respective processes using its hardware.

[0057] Furthermore, some or all of the components of each device are realized by other general-purpose or dedicated circuits, processors, etc., or combinations thereof. These may be made up of a single chip or multiple chips connected via a bus.

[0058] Some or all of the components of each device may be realized by a combination of the circuits and programs described above.

[0059] When some or all of the components of each device are implemented by multiple information processing devices or circuits, these multiple information processing devices or circuits may be centrally located or distributed. For example, the information processing devices or circuits may be implemented in a form in which each is connected via a communication network, such as a client-and-server system or a cloud computing system.

[0060] The embodiments described above are preferred embodiments of this disclosure and do not limit the scope of this disclosure to these embodiments alone. That is, a person skilled in the art can modify or substitute the embodiments described above to construct various modified forms without departing from the gist of this disclosure.

[0061] Some or all of the above embodiments may also be described as follows, but are not limited to these.

[0062] [Note 1] An information processing device comprising: a person detection means for detecting a person in an image taken by a camera that photographs the interior of a moving vehicle; a selection means for selecting a person to be analyzed from among the people in the image based on predetermined rules; an estimation means for analyzing the area in which the person to be analyzed is photographed and estimating the health status of the person; and a transmission means for transmitting the image to a predetermined image analysis server. [Note 2] The predetermined rules of the above-described information processing device may be rules for selecting the person to be monitored based on the person's boarding position. [Note 3] The predetermined rules of the above-described information processing device may be rules for selecting the person to be monitored based on the person's clothing. [Note 4] The predetermined rules of the above-described information processing device may be rules for selecting the person to be monitored based on the person's age or gender. [Note 5] The estimation means of the above-described information processing device may be configured to estimate the health status of the person by estimating vital sign values ​​using an image of the person's skin. [Note 6] The estimation means of the information processing device described above can be configured to estimate the level of alertness from an image of the person's eyes, and to estimate the person's health condition by referring to the level of alertness in addition to the vital sign values. [Note 7] The information processing device described above can further include a person identification means for identifying the person based on an image taken by a camera that photographs the interior of the vehicle, and the selection means can be configured to select the person to be monitored using information regarding the health condition of the identified person. [Note 8] The information processing device described above can be configured to determine whether the vital sign values ​​of the person to be monitored are abnormal based on the normal values ​​of the vital sign values ​​of the identified person. [Note 9] An information processing method that detects a person in an image taken by a camera that photographs the interior of a vehicle, selects a person to be analyzed from among the people in the image based on predetermined rules, analyzes the area in which the person to be analyzed is pictured, estimates the person's health condition, and transmits the image to a predetermined image analysis server.[Note 10] A recording medium that records a program causing a computer to execute the following: a process for detecting a person in an image taken by a camera that photographs the interior of a moving vehicle; a process for selecting a person to be analyzed from among the people in the image based on predetermined rules; a process for analyzing the area in which the person to be analyzed is photographed and estimating the health status of the person; and a process for transmitting the image to a predetermined image analysis server. The forms described in each of the above notes can be combined with each other after making the necessary modifications. For example, a configuration that combines the contents described in Note 2 and the contents described in Note 3 is also included in the scope of disclosure of this specification. In this case, the information processing device will select the person to be analyzed based on both the person's position and clothing. The forms described in Notes 9 to 10 can be expanded into the forms described in Notes 2 to 7, similar to Note 1.

[0063] Furthermore, each disclosure in the above-mentioned patent documents is incorporated into this document by reference and may be used as the basis or part of this disclosure as necessary. Within the framework of this disclosure (including the claims), further modifications and adjustments to the embodiments or examples are possible based on their fundamental technical concept. Also, within the framework of this disclosure, various combinations or selections (including partial deletions) of various disclosure elements (including each element of each claim, each element of each embodiment or example, each element of each drawing, etc.) are possible. In other words, this disclosure naturally includes the entire disclosure, including the claims, and various modifications and alterations that a person skilled in the art could make in accordance with the technical concept. In particular, with respect to the numerical ranges described in this document, any numerical value or sub-range included within that range should be interpreted as being specifically described, even if not otherwise stated. Furthermore, each disclosure item of the above-mentioned cited documents may, as necessary, be used in part or in whole as part of this disclosure, in accordance with the spirit of this disclosure, and this is also considered to be included in the disclosure items of this application.

[0064] 10 Information Processing Device 11 Person Detection Means 12 Selection Means 13 Estimation Means 14 Transmission Means 20, 200 Image Analysis Server 100, 100a, 100b In-vehicle Terminal 101 Person Detection Unit 102, 102a, 102b Selection Unit 103, 103b Estimation Unit 104 Transmission Unit 105, 105b Control Unit 106 Person Identification Unit 107 Personal Information Storage Unit 108 Awakening Level Estimation Unit 900 Information Processing Device 901 CPU (Central Processing Unit) 902 ROM (Read Only Memory) 903 RAM (Random Access Memory) 904 Program 905 Storage Device 906 Recording Medium 907 Drive Device 908 Communication Interface 909 Communication Network 910 Input / Output Interface 911 Bus C Camera N Network P1-P3 V Mobile Entity (Vehicle)

Claims

1. An information processing device comprising: a person detection means for detecting a person in an image captured by a camera that photographs the interior of a moving vehicle; a selection means for selecting a person to be analyzed from among the people in the image based on predetermined rules; an estimation means for analyzing the area in which the person to be analyzed is depicted and estimating the health status of the person; and a transmission means for transmitting the image to a predetermined image analysis server.

2. The information processing device according to claim 1, wherein the predetermined rule is a rule for selecting the person to be analyzed based on the person's boarding position.

3. The information processing device according to claim 1 or 2, wherein the predetermined rule is a rule for selecting the person to be analyzed based on the person's clothing.

4. The information processing device according to any one of claims 1 to 3, wherein the predetermined rule is a rule for selecting the person to be analyzed based on the person's age or gender.

5. An information processing device according to any one of claims 1 to 4, wherein the estimation means estimates the health status of a person by estimating vital sign values ​​using images of the person's skin.

6. The information processing apparatus according to claim 5, wherein the estimation means estimates the level of alertness from an image of the person's eyes, and estimates the person's health status by referring to the level of alertness in addition to the vital sign values.

7. An information processing device comprising a person identification means for identifying a person based on an image taken by a camera that photographs the interior of the moving vehicle, wherein the selection means selects the person to be monitored using information regarding the health status of the identified person.

8. An information processing device that determines whether the vital sign values ​​of the person being monitored are abnormal, based on the normal values ​​of the vital sign values ​​of the person identified.

9. An information processing method comprising: detecting a person in an image captured by a camera that photographs the interior of a moving vehicle; selecting a person to be analyzed from among the people in the image based on predetermined rules; analyzing the area in which the person to be analyzed is depicted; estimating the health status of the person; and transmitting the image to a predetermined image analysis server.

10. A recording medium that contains a program that causes a computer to perform the following steps: a process of detecting a person in an image taken by a camera that photographs the interior of a moving vehicle; a process of selecting a person to be analyzed from among the people in the image based on predetermined rules; a process of analyzing the area in which the person to be analyzed is photographed and estimating the health status of the person; and a process of transmitting the image to a predetermined image analysis server.

Images