Information processing device and biological condition determining method

Abstract

To determine a biological state more accurately without the need for correction in consideration of differences in sensor characteristics.SOLUTION: An information processing device includes an image information generation unit for generating image information based on biological information, a model construction unit for constructing a learning model by machine learning using the image information, and a biological state determination unit for inputting the image information to the learning model and acquiring an output value that is a determination result of the biological state.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device and a biological condition determining method. [Background technology]

[0002] Patent Document 1 describes, with regard to a drowsiness estimation rule updating device, the device "includes an RRI acquisition means for acquiring RRI data, an index value calculation means for calculating index values ​​for a plurality of indexes related to autonomic nerve activity based on the RRI data, a storage means for storing drowsiness estimation rules, a drowsiness data acquisition means for acquiring information output from a drowsiness information output device that outputs information on the presence or absence of drowsiness, which is information obtained from the subject or information obtained by analyzing the subject's biological state, a reference data generation means for generating reference data using the drowsiness data for evaluating the suitability of the drowsiness estimation rules, and an update means for updating the drowsiness estimation rules stored in the storage means based on the index values ​​and the reference data." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-120062 A Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, attention has been focused on technologies that measure a driver's biometric information, such as pulse rate and heart rate, to determine their biological condition, such as drowsiness, fatigue, or level of concentration, and then perform automatic control, such as issuing an alert to prevent an accident or stopping the vehicle, depending on the results of the determination.

[0005] When determining the biological condition, the measurement data is corrected taking into account the characteristic differences that differ depending on the sensor model, the individual differences of the measurement subject, or the differences in the measurement method. However, there is a problem that the correction process and conditions must be set in advance for each sensor, which is troublesome. Therefore, there is a demand for a method to determine the biological condition more accurately without the need for correction to absorb the differences in sensor characteristics.

[0006] Patent Document 1 discloses a device for determining drowsiness using RRI (RR Interval: heart rate interval). In the technology of this document, RRI data is corrected to absorb characteristic differences due to differences in sensor models, and sensor characteristic information used for the correction is set in advance. In other words, the technology of this document does not take into consideration the determination of a biological state that does not require correction to absorb differences in sensor characteristics, etc.

[0007] Therefore, an object of the present invention is to determine a biological condition with higher accuracy without requiring correction taking into account differences in sensor characteristics and the like. [Means for solving the problem]

[0008] The present application includes a plurality of means for solving at least part of the above-mentioned problems, examples of which are as follows. People Biometric information This is an image showing a Lorentz plot graph that shows the fluctuation of the heartbeat interval using RRI (RR Interval) data. an image information generating unit that generates image information; and a model constructing unit that constructs a learning model by machine learning using the image information; Generated based on the biometric information acquired from the user of the vehicle. inputting the image information into the learning model; Regarding the presence or absence of drowsiness, fatigue, and ability to concentrate a biometric condition determining unit for obtaining an output value which is a result of determining a biometric condition; To control the in-vehicle equipment based on the determination result, The result of the determination of the biological condition The vehicle is provided with Prescribed vehicle and a communication unit that outputs the signal to the device. Effect of the Invention

[0009] According to the present invention, it is possible to determine the biological condition with higher accuracy without requiring correction taking into account differences in sensor characteristics and the like.

[0010] Problems, configurations, effects, and the like other than those described above will become apparent from the description of the following embodiments. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram showing an example of a schematic configuration of an information processing device according to a first embodiment. [Diagram 2] FIG. 2 is a functional block diagram showing an example of a functional configuration of a calculation unit in the information processing device. [Diagram 3] Fig. 3(a) is a diagram showing an example of image information of a Lorenz plot showing fluctuations in heartbeat intervals when the user is not feeling sleepy. Fig. 3(b) is a diagram showing an example of image information of a Lorenz plot showing fluctuations in heartbeat intervals when the user is feeling sleepy. [Figure 4] FIG. 11 is a flow diagram illustrating an example of a learning model construction process. [Diagram 5] FIG. 11 is a flow chart showing an example of a biological condition determining process. [Figure 6] FIG. 11 is a functional block diagram showing an example of a functional configuration of an information processing device and a server device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described.

[0013] First Embodiment 1 is a diagram showing an example of a schematic configuration of an information processing device 100. The information processing device 100 is a device that judges a biological state by using biological information of a user (e.g., a driver) riding in a moving body such as a vehicle. Specifically, the information processing device 100 generates image information of a graph (Lorenz plot) showing fluctuations in heartbeat intervals by using biological information (e.g., RRI data: RR Interval) generated from measurement results of heartbeat, pulse, etc., and inputs this to a learning model to judge a biological state such as sleepy or not sleepy.

[0014] As shown in the figure, the information processing device 100 has a calculation device 110, a sensor 120, a communication device 130, and a storage device 140. The information processing device 100 may have the calculation device 110, the communication device 130, the storage device 140, and the sensor 120 in the same housing, or the calculation device 110, the communication device 130, the storage device 140, and the sensor 120 may each be housed in a separate housing. The calculation device 110 and the sensor 120 are connected to each other so as to be able to communicate with each other via a communication cable or a predetermined wireless communication standard (e.g., Bluetooth: registered trademark).

[0015] The arithmetic device 110 is a device that performs various arithmetic processes of the information processing device 100. As shown in the figure, the arithmetic device 110 has a CPU (Central Processing Unit) 111 that executes various arithmetic processes performed in the information processing device 100, a ROM (Read Only Memory) 112 that stores programs executed by the CPU 111, etc., a RAM (Random Access Memory) 113 that temporarily stores various information read from the ROM 112, an I / F (Interface) 114 for electrically connecting the arithmetic device 110 and the sensor 120, and a bus 115 that connects them to each other.

[0016] The arithmetic device 110 acquires biological information (for example, RRI data indicating a heartbeat interval) from the sensor 120, and uses the biological information to generate image information showing a Lorentz plot graph representing the fluctuation of the heartbeat interval (hereinafter, may be referred to as "Lorentz plot image information"). In addition, the arithmetic device 110 constructs (generates) a learning model by performing machine learning using the generated image information.

[0017] Moreover, the arithmetic device 110 inputs image information generated using the biological information acquired from the sensor 120 into the learning model, thereby acquiring an output value that is a result of arithmetic processing by the learning model. Note that the output value is information indicating the result of the determination of the biological state using the image information, and indicates the biological state, for example, sleepy, not sleepy, tired, not tired, focused, or not focused.

[0018] The arithmetic device 110 also converts the output values ​​output from the learning model into a predetermined data format that can be used by the Main Unit 200 (for example, a navigation device, an in-vehicle device such as a D / A (Digital Audio), or an ECU (Engine Control Unit)), and outputs the data to these devices. The in-vehicle device, etc., uses the determination result of the biological condition obtained from the information processing device 100 to execute processing according to the user's biological condition (for example, playing music in the car, or controlling the air conditioner to change the temperature inside the car).

[0019] The sensor 120 is a measuring device that measures vital data such as the user's heart rate and pulse rate. Specifically, the sensor 120 is a contact sensor capable of detecting the user's heart rate (or pulse rate), such as a piezoelectric meter embedded in a vehicle seat or a wearable device such as a watch that is worn in contact with the body. The sensor 120 may be a non-contact sensor such as a Doppler sensor that irradiates microwaves or millimeter waves onto the human body and detects the user's heart rate based on the reflected waves. Alternatively, the sensor 120 may be a camera that can detect the heart rate based on the flow rate of hemoglobin contained in the blood.

[0020] The sensor 120 measures the user's heartbeat for a predetermined period (for example, 5 minutes, 10 minutes, or 30 minutes from the start of measurement), generates RRI data, which is biological information, as a measurement result, and outputs it to the information processing device 100. The RRI data is information that indicates the heartbeat interval (interval between R waves), which is the time interval between heartbeats.

[0021] The type of sensor 120 is not particularly limited as long as it is a measuring device capable of measuring vital data such as the user's heart rate or pulse rate. For example, sensor 120 may be a measuring device capable of calculating the user's heart rate interval in a predetermined period by inverse calculation based on the detected pulse rate and generating RRI data.

[0022] The communication device 130 is a communication module for communicating information with an external device (for example, the Main Unit 200). The communication device 130 is connected to the Main Unit 200 via a communication cable or a predetermined wireless communication standard (for example, Bluetooth) so as to be capable of communicating with the Main Unit 200, and transmits the result of the determination of the biological condition to the Main Unit 200.

[0023] The storage device 140 is a so-called hard disk drive, a solid state drive (SSD), a flash memory, or other non-volatile storage device capable of storing digital information. The storage device 140 stores, for example, various information used to construct a learning model and the constructed learning model.

[0024] 2 is a functional block diagram showing an example of a functional configuration of the arithmetic device 110 in the information processing device 100. As shown in the figure, the arithmetic device 110 has a arithmetic unit 300, a control unit 310, a storage unit 320, and a communication unit 330.

[0025] The calculation unit 300 is a functional unit that performs various processes executed by the calculation device 110. Specifically, the calculation unit 300 has an image information generation unit 301, a labeling unit 302, a model construction unit 303, a user authentication unit 304, a biological condition determination unit 305, and a data format conversion unit 306. In the following, the determination of a biological condition related to drowsiness will be described as an example.

[0026] The image information generating unit 301 is a functional unit that generates image information using biological information acquired from the sensor 120. Specifically, the image information generating unit 301 uses the RRI data acquired from the sensor 120 to generate image information of a Lorenz plot that represents fluctuations in heartbeat intervals over a predetermined period of time.

[0027] More specifically, the image information generating unit 301 uses the RRI data to plot the time interval from the start of measurement to the first (Nth (N=1)) heartbeat on the horizontal axis (X axis), the time interval from the first (Nth (N=1)) heartbeat to the next (N+1th (N=1)) heartbeat on the vertical axis (Y axis), and plots their intersections. Next, the image information generating unit 301 plots the time interval from the first (Nth (N=1)) heartbeat to the next (N+1th (N=1)) heartbeat on the horizontal axis (X axis), the time interval from the next (N+1th (N=1)) heartbeat to the next (N+1 (N=2)) heartbeat on the vertical axis (Y axis), and plots their intersections. In this way, the image information generating unit 301 generates a Lorentz plot graph by repeatedly performing the same process for all heartbeat intervals measured in a predetermined period.

[0028] Furthermore, the image information generating unit 301 generates image information by converting the generated Lorenz plot graph into a predetermined format.

[0029] FIG. 3(a) shows an example of image information of a Lorenz plot showing fluctuations in heartbeat intervals when the user is not feeling sleepy. FIG. 3(b) shows an example of image information of a Lorenz plot showing fluctuations in heartbeat intervals when the user is feeling sleepy. As shown in these figures, when the user is not feeling sleepy, the fluctuations in the heartbeat intervals are small, so the positions of the plots do not spread out and the graph shape falls within a certain range. On the other hand, when the user is feeling sleepy, the fluctuations in the heartbeat intervals are large, so the positions of the plots are scattered over a wide range. In this way, the image information of the Lorenz plot based on the RRI data shows a unique shape that represents the user's biological state (for example, sleepy, not sleepy, etc.), so that the user's biological state can be determined using such image information.

[0030] The labeling unit 302 is a functional unit that performs a labeling process. Specifically, the labeling unit 302 performs a labeling process of associating a correct answer label indicating the biological state of the user with image information of the Lorenz plot. More specifically, the labeling unit 302 obtains a correct answer label indicating whether the user felt sleepy during the measurement period, and performs a labeling process of associating the correct answer label with image information of the Lorenz plot generated using the biological information of the user during the corresponding period.

[0031] The labeling unit 302 acquires the correct label by a predetermined method. For example, after the end of the measurement period, the labeling unit 302 displays screen information for receiving an answer from the user as to whether or not the user felt sleepy during the measurement period on a display device provided in the information processing device 100 or the Main Unit 200, and acquires the correct label by receiving the answer directly from the user.

[0032] Alternatively, the labeling unit 302 captures the facial expression of the user during the measurement period via an in-car camera capable of capturing images inside the car, and performs face recognition using the image information to obtain a correct label indicating whether the user felt drowsy during the measurement period. Note that the face recognition technology may be a publicly known technology such as the evaluation method of NEDO (New Energy Industrial Technology Development Organization).

[0033] The method of obtaining the correct label is not particularly limited, and any method may be used as long as a correct label indicating whether or not the user felt drowsy during the measurement period can be obtained.

[0034] The model construction unit 303 is a functional unit that constructs the learning model 322. Specifically, the model construction unit 303 performs machine learning using the image information of the Lorenz plot generated by the image information generation unit 301 and the correct answer label associated with the image information to construct the learning model 322. The construction process of the learning model 322 will be described later.

[0035] The user authentication unit 304 is a functional unit that performs user authentication based on user information. The user information includes a user ID for identifying the user and a category to which the user belongs among predetermined categories classified based on physical features. The user authentication unit 304 searches the storage unit 320 for a learning model 322 associated with the user ID or a learning model 322 associated with the category to which the user belongs.

[0036] The biometric condition determination unit 305 is a functional unit that inputs image information of a Lorenz plot generated using biometric information (RRI data) acquired from the sensor 120 into the learning model 322 and acquires an output value indicating the determination result of the biometric condition.

[0037] The data format conversion unit 306 is a functional unit that acquires an output value (information indicating a result of determination of a biological state) output as a calculation result of the learning model 322, and converts the output value into a data format that can be used by the Main Unit 200. In addition, the data format conversion unit 306 outputs the result of determination of a biological state, which is an output value after conversion, to the Main Unit 200 via the communication unit 330. Note that the data format to be converted is not particularly limited as long as it is a data format that can be used by the Main Unit 200.

[0038] The control unit 310 is a functional unit that controls the operation of the sensor 120. Specifically, the control unit 310 outputs an instruction to measure biological information to the sensor 120. The control unit 310 also outputs instructions to the sensor 120 regarding predetermined control objects, such as recognition of the user position, the measurement time for the user, and the frequency range of the radio waves to be irradiated.

[0039] The storage unit 320 is a functional unit that stores various data. Specifically, the storage unit 320 stores various base models 321 that serve as a foundation for constructing a learning model 322. The base models 321 include, for example, a convolutional neural network (CNN) that exhibits excellent performance in image recognition. The storage unit 320 also stores the constructed learning model 322.

[0040] The learning model 322 includes a learning model 322 constructed by measurements of a user of a vehicle equipped with the information processing device 100, and a plurality of learning models 322 constructed by measurements of people (people other than the user) belonging to each category classified based on physical features. These learning models 322 are stored in the storage unit 320 in association with a user ID and the category.

[0041] The communication unit 330 is a functional unit that performs information communication with the Main Unit 200. Specifically, the communication unit 330 transmits the determination result of the biological condition to the Main Unit 200. Note that the communication unit 330 may obtain the correct label from the Main Unit 200.

[0042] The functional configuration of the information processing device 100 has been described above.

[0043] The arithmetic unit 300 and the control unit 310 are realized by a program that causes the CPU 111 of the arithmetic device 110 to perform processing. These programs are stored in, for example, the ROM 112, and are loaded onto the RAM 113 for execution and executed by the CPU 111. The control unit 310 may be realized by a predetermined control circuit. The storage unit 320 is realized by the RAM 113, the ROM 112, the storage device 140, or a combination of these. The communication unit 330 is realized by the communication device 130.

[0044] Moreover, each functional block of the information processing device 100 is classified according to the main processing content in order to facilitate understanding of each function realized in this embodiment. Therefore, the present invention is not limited by the classification method or name of each function. Furthermore, each configuration of the information processing device 100 can be further classified into more components according to the processing content. Moreover, it is also possible to classify one component so that it executes more processes.

[0045] In addition, all or part of each functional unit may be constructed by hardware (such as an integrated circuit such as an ASIC) implemented in a computer. Furthermore, the processing of each functional unit may be executed by one piece of hardware or by multiple pieces of hardware.

[0046] [Explanation of operation] Next, a learning model construction process executed by the information processing device 100 will be described.

[0047] 4 is a flow diagram showing an example of a learning model construction process. This process is started when, for example, a user (or a person other than the user) receives an instruction to execute the learning model construction process while driving. In the following, an example will be described in which a learning model 322 is constructed for a user.

[0048] When the process starts, image information to be used for machine learning is generated by the processes of steps S001 to S005 below. Specifically, the control unit 310 outputs an instruction to the sensor 120 to execute (start) measurement of biometric information of a user (e.g., a driver) (step S001). Based on the instruction, the sensor 120 executes measurement of the user for a predetermined period (e.g., 5 minutes), generates RRI data based on the measurement data, and outputs it to the image information generation unit 301.

[0049] Next, the model construction unit 303 acquires the RRI data from the sensor 120 (step S002). The model construction unit 303 temporarily stores the acquired RRI data in the storage unit 320.

[0050] Next, the model construction unit 303 performs data cleaning (step S003). Specifically, the model construction unit 303 removes outliers from the acquired RRI data and performs a process of complementing information that cannot be read as a numerical value (NaN). The complementation process is performed by a predetermined method, such as by adopting the midpoint between the values ​​before and after the data point that is NaN.

[0051] Next, the image information generating unit 301 generates a Lorenz plot graph based on the RRI data, and generates image information of the graph (step S004).

[0052] Next, the labeling unit 302 performs a labeling process on the generated image information (step S005). Specifically, the labeling unit 302 acquires, from the user, a correct answer label indicating a biological state during a measurement period of the RRI data corresponding to the generated image information, and associates the correct answer label with the image information.

[0053] Through the processing of steps S001 to S005, information for machine learning is generated that corresponds image information of a Lorenz plot generated using RRI data obtained by measuring a user for a predetermined period of time with a correct answer label indicating the biological condition during that period.

[0054] Next, the model construction unit 303 determines whether a predetermined number (e.g., 100) of pieces of machine learning information have been generated (step S006). Note that the predetermined number of pieces of machine learning information includes a predetermined ratio (e.g., approximately equal numbers) of image information of Lorenz plots showing fluctuations in heartbeat intervals in each of the cases where the user feels sleepy and where the user does not feel sleepy.

[0055] If it is determined that the predetermined number of pieces of information for machine learning have not been generated (No in step S006), the model construction unit 303 returns the process to step S001. On the other hand, if it is determined that the predetermined number of pieces of information for machine learning have been generated (Yes in step S006), the model construction unit 303 transitions the process to step S007.

[0056] In step S007, the model construction unit 303 divides the machine learning information at an arbitrary ratio. Specifically, the model construction unit 303 divides the machine learning information, i.e., image information of a Lorenz plot associated with a correct answer label indicating a biological state, into training data and validation data in order to use a hold-out method for accuracy verification described later. The division ratio of the training data and validation data is arbitrary. For example, the training data and validation data are divided at an arbitrary ratio such as 6:4 or 7:3.

[0057] Next, the model construction unit 303 constructs the learning model 322 by the processes of steps S008 to S011. Specifically, the model construction unit 303 determines the base model 321 (step S008). More specifically, the model construction unit 303 acquires a predetermined base model 321 (for example, a convolutional neural network) from the storage unit 320, and determines this as the base model 321 to be used for model construction.

[0058] Next, the model construction unit 303 executes machine learning using the training data (step S009). Here, machine learning will be described. The model construction unit 303 inputs image information of the training data to a transfer function indicated by a determined base model 321 such as a convolutional neural network, and compares an output value output from the base model 321 as a prediction result for a correct label with a correct label associated with the input image information. If the output value differs from the correct label, the model construction unit 303 feeds it back and changes the objective variable of the transfer function. The model construction unit 303 also executes such prediction and feedback processing for all training data to perform machine learning and builds a learning model 322.

[0059] Next, the model construction unit 303 executes accuracy verification of the learning model 322 using the verification data (step S010). Specifically, the model construction unit 303 inputs the verification data to the constructed learning model 322. In addition, the model construction unit 303 verifies the accuracy of the constructed learning model 322 based on a comparison between an output value from the learning model 322 and a correct answer label associated with image information of the verification data.

[0060] Next, the model construction unit 303 judges whether the accuracy of the learning model 322 is sufficient (step S011). Specifically, the model construction unit 303 makes this judgment based on whether the accuracy of the learning model 322 verified in step S010 satisfies a predetermined reference value (e.g., an accuracy rate of 80% or more) that is judged to be sufficient. If the model construction unit 303 judges that the accuracy is sufficient (Yes in step S011), the model construction unit 303 shifts the process to step S012. On the other hand, if the model construction unit 303 judges that the accuracy is not sufficient (No in step S011), the model construction unit 303 returns the process to step S001.

[0061] In addition, in the processes of steps S009 and S010 executed via step S011, the model construction unit 303 executes these processes using new training data and validation data. Specifically, in step S001, the model construction unit 303 outputs an instruction to the sensor 120 via the control unit 310 to execute a new measurement, and executes the same processes as the processes of steps S002 to S010 described above using the acquired new biological information.

[0062] Also, in step S012, the model construction unit 303 stores (preserves) in the storage unit 320 a predetermined category classified based on the user ID or physical features in association with the constructed learning model 322.

[0063] Furthermore, when the model construction unit 303 saves the learning model 322, the process of this flow ends.

[0064] When constructing a learning model 322 for a person other than the user, the model construction unit 303 constructs a learning model 322 corresponding to each category by measuring a person belonging to each category classified based on physical characteristics. The categories are classified according to the type and range of physical characteristics. For example, examples of categories include "gender = male, age = 30s, height = 170 cm to 180 cm, weight = 60 kg to 70 kg, chronic disease = none" and "gender = female, age = 40s, height = 150 cm to 160 cm, weight = 50 kg to 60 kg, chronic disease = none".

[0065] The model construction unit 303 performs measurements on people in a population that belong to the categories thus classified, and constructs a learning model 322 corresponding to each category using image information generated based on the biometric information thus obtained.

[0066] As a result, in the biometric condition determination process described below, the user can determine the biometric condition based on his or her own user authentication by applying a learning model 322 constructed by measurements on himself or herself or a learning model 322 constructed by measurements on a person belonging to the same category as the user.

[0067] Next, the biological condition determining process executed by the information processing device 100 will be described.

[0068] 5 is a flow diagram showing an example of a biological condition determination process. This process is started, for example, when the engine of the vehicle is started (when the ignition key is turned on) or when a process start instruction is received from a user. The engine start may be detected by the user authentication unit 304 acquiring information indicating that the engine has started from the ECU via the communication unit 330. The user may issue a start instruction by, for example, pressing a process start instruction button provided in the information processing device 100, and the user authentication unit 304 may detect this.

[0069] When the process starts, the user authentication unit 304 performs personal authentication of the user (step S021). Specifically, the user authentication unit 304 searches the storage unit 320 for a learning model 322 associated with the user ID input via a predetermined input device. If such a learning model 322 does not exist, the user authentication unit 304 searches the storage unit 320 for a learning model 322 associated with the category to which the user belongs.

[0070] The predetermined input device may be, for example, the Main Unit 200 of a navigation device. In this case, the user authentication unit 304 may acquire user input information from the Main Unit 200 via the communication unit 330. The input device may be provided in the information processing device 100.

[0071] In addition, the authentication method is not limited to input of user information. For example, if the vehicle is equipped with an in-vehicle camera that captures images inside the vehicle, the user authentication unit 304 may acquire image information captured by the in-vehicle camera and use the image information to perform face authentication to identify the physical features of the user. When this method is adopted, the information processing device 100 only needs to be connected to the in-vehicle camera via the communication device 130 so as to be able to communicate with the camera.

[0072] The authentication method may be fingerprint authentication. When this method is adopted, the information processing device 100 may be provided with a fingerprint authentication sensor. Alternatively, when a fingerprint authentication sensor is installed at a predetermined location of the vehicle (e.g., a steering wheel, etc.), the information processing device 100 may acquire fingerprint information from an ECU connected via the communication device 130, for example, and identify the physical features of the user by collating the fingerprint information with fingerprint information previously stored in the storage unit 320.

[0073] Next, the user authentication unit 304 judges whether or not there is a corresponding learning model 322 (step S022). Specifically, the user authentication unit 304 makes such a judgment based on whether or not a learning model 322 associated with the user ID or the category to which the user belongs is stored in the storage unit 320. If it is judged that there is a corresponding learning model 322 (Yes in step S022), the user authentication unit 304 shifts the process to step S023. On the other hand, if it is judged that there is no corresponding learning model 322 (No in step S022), the user authentication unit 304 shifts the process to step S030.

[0074] In step S030, the biometric condition determination unit 305 notifies the user that there is no corresponding learning model 322. For example, the biometric condition determination unit 305 generates a predetermined message for notifying the user that there is no corresponding learning model 322, and displays the message on a display device of the in-vehicle device, which is the Main Unit 200, via the communication unit 330. In addition, when the biometric condition determination unit 305 displays the message on the Main Unit 200, the process of this flow is terminated.

[0075] In addition, in the process of step S023 to which the process proceeds when it is determined that a corresponding learning model 322 exists, the user authentication unit 304 acquires the corresponding learning model 322 from the storage unit 320 .

[0076] Next, the biological condition determining unit 305 outputs an instruction to measure vital data such as heart rate to the sensor 120 via the control unit 310 (step S024). Next, when the biological condition determining unit 305 acquires the RRI data output from the sensor 120 (step S025), it inputs the RRI data to the acquired learning model 322 (step S026).

[0077] Next, the biological condition determining section 305 acquires the output value output as the calculation result of the learning model 322, that is, the determination result regarding the presence or absence of drowsiness of the user during the measurement period (sleepy / not sleepy) (step S027).

[0078] The data format conversion unit 306 also converts the output value into a predetermined data format (step S028) and outputs it to the Main Unit 200 via the communication unit 330 (step S029). When the data format conversion unit 306 outputs the output value, which is the determination result of the biological condition, to the Main Unit 200, it ends the processing of this flow.

[0079] The information processing device 100 according to this embodiment has been described above.

[0080] According to such an information processing device, the biological condition can be determined more accurately without the need for correction taking into account differences in sensor characteristics, etc. In particular, the information processing device uses RRI data, which is biological information, to generate image information of a Lorenz plot that visually represents fluctuations in heartbeat intervals, and determines the biological condition using such image information. For example, when RRI data is input into a learning model to determine the biological condition, differences in sensor characteristics are reflected in the numerical data, so correction processing and its conditions must be set in advance beforehand. However, according to the information processing device, the biological condition is determined using image information that allows visual discrimination of tendencies indicating the biological condition, so correction and settings for correction are not required, and the biological condition can be determined accurately.

[0081] Second Embodiment 6 is a functional block diagram showing an example of the functional configuration of the information processing device 100 and the server device 400 according to this embodiment. As shown in the figure, the information processing device 100 is connected to the server device 400 so as to be able to communicate with each other via a predetermined network N such as a LAN (Local Area Network).

[0082] The image information generation unit 411, the labeling unit 412, and the model construction unit 413, the memory unit 420, the base model 421, and the communication unit 430 in the server device 400 correspond to the image information generation unit 301, the labeling unit 302, and the model construction unit 303, the memory unit 320, the base model 321, and the communication unit 330 in the information processing device 100 of the first embodiment, respectively, and are functional units and information that perform similar processing, so detailed explanations are omitted in this embodiment.

[0083] In the first embodiment described above, the learning model 322 was constructed by the information processing device 100, but in this embodiment, the biometric information acquired from the sensor 120 is transmitted to the server device 400, and the learning model 322 is constructed by the image information generation unit 411, the labeling unit 412, and the model construction unit 413 possessed by the calculation unit 410 of the server device.

[0084] Specifically, when the calculation unit 300 of the information processing device 100 acquires biological information (RRI data) from the sensor 120 in the learning model construction process, the calculation unit 300 transmits the biological information (RRI data) to the server device 400 via the communication unit 330. When the image information generation unit 411, the labeling unit 412, and the model construction unit 413 of the server device 400 acquire the RRI data via the communication unit 430, they construct the learning model 322 in the same manner as in the above-mentioned steps S003 to S011.

[0085] Moreover, the learning model 322 constructed by the model construction unit 413 of the server device 400 is transmitted to the information processing device 100 via the communication unit 430 of the server device 400 and stored in the storage unit 320. Moreover, the information processing device 100 executes the same biological condition determination process as described above, using the learning model 322 stored in the storage unit 320.

[0086] According to the information processing device 100 and server device 400 of the second embodiment, the construction of the learning model 322 can be executed by a server device with high processing performance, so that the specifications of the information processing device 100 can be reduced, and as a result, manufacturing costs can be saved.

[0087] The present invention is not limited to the information processing device 100 of the first and second embodiments, and various modifications are possible. For example, the arithmetic device 110 (the arithmetic unit 300 and the control unit 310), the communication device 130, and the storage device 140 in the first embodiment may be built into an on-board device such as a navigation device, which is the Main Unit 200, or an ECU.

[0088] In addition, in the above-described embodiment, the output value is a judgment result such as sleepy / not sleepy, but the present invention is not limited to this, and the information processing device 100 may output a judgment result including an intensity level of the biological state, such as "slightly sleepy," "very sleepy," "low sleepiness," "medium sleepiness," and "high sleepiness."

[0089] Such a determination can be realized, for example, by subdividing the correct labels into a plurality of levels in the biological state, associating them with the image information, and constructing a learning model 322 by machine learning using them.

[0090] According to the information processing device 100 according to such a modified example, the biometric condition of the user can be determined at a more detailed level.

[0091] In addition, in the above-described embodiment, image information of a Lorenz plot is generated, but the present invention is not limited to this. For example, image information showing the user's electrocardiogram waveform may be generated and used to determine the biological condition.

[0092] Even with an information processing device according to such a modified example, the biological condition is determined using image information that allows visual identification of trends indicating the biological condition, so that correction and settings for correction are not required, and the biological condition can be determined with high accuracy.

[0093] Furthermore, the present invention is not limited to the above-mentioned embodiments and modifications, and includes various other embodiments and modifications. For example, the above-mentioned embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment or modification, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0094] 100 information processing device, 110 calculation device, 111 CPU, 112 ROM, 113 RAM, 114 I / F (interface), 115 bus, 120 sensor, 130 communication device, 140 storage device, 200 main unit, 300 calculation section, 301 image information generating section, 302 labeling section, 303 model constructing section, 304 user authentication section, 305 biological state determining section, 306 data format conversion section, 310 control section, 320 storage section, 321 base model, 322 learning model, 330 communication section

Claims

1. An image information generating unit that generates image information showing a Lorentz plot graph representing fluctuations in heartbeat intervals using RRI (R-R Interval) data, which is human biological information; A model construction unit that constructs a learning model by machine learning using the image information; a biological condition determination unit that inputs the image information generated based on the biological information acquired from the user of the vehicle into the learning model and obtains output values ​​that are biological condition determination results regarding the presence or absence of drowsiness, fatigue, and concentration; a communication unit that outputs the result of the biological condition determination to a predetermined vehicle device implemented in the vehicle so as to control in-vehicle equipment based on the result of the determination.

23. An information processing apparatus comprising:

2. 2. The information processing device according to claim 1, The present invention further includes a storage unit that stores the learning model constructed using the biometric information acquired by measuring people belonging to predetermined categories classified according to combinations of types of physical features and their ranges.

23. An information processing apparatus comprising:

3. 3. The information processing device according to claim 2, the learning model is stored in the storage unit in association with user identification information or the user identification information and the predetermined category; A user authentication unit that searches the storage unit for the learning model corresponding to the user's identification information or the category to which the user belongs.

23. An information processing apparatus comprising:

4. A biological condition determining method performed by an information processing device, comprising: The information processing device includes: A step of generating image information showing a Lorentz plot graph representing fluctuations in heartbeat intervals using RRI (R-R Interval) data, which is human biological information; constructing a learning model by machine learning using the image information; inputting the image information generated based on the biological information acquired from the user of the vehicle into the learning model, and acquiring output values ​​which are the results of the biological state determination regarding the presence or absence of drowsiness, the presence or absence of fatigue, and the presence or absence of concentration; a communication step of outputting the determination result of the biological condition to a predetermined vehicle device installed in the vehicle in order to control in-vehicle equipment based on the determination result. A biological condition determining method comprising:

Images