Information processing device and method for determining biological state

Abstract

Information processing device and method for determining biological state. The determination of a biological state with greater accuracy without the need for corrections to account for differences in sensor characteristics, etc. An information processing device (100) comprising an image information generation module (301) that generates image information based on biological information, a model building module (303) that constructs a machine learning model using said image information, and a biological state determination module (305) that inputs said image information into said learning model (322) and obtains an output value that is the result of determining said biological state. Figure for the abstract: Figure 2

Description

Title of the invention: Information processing device and method for determining biological state

[0001] The present invention relates to an information processing device and a method for determining biological state.

[0002] Patent document JP2016-120062 relates to a device for updating sleepiness estimation rules and provides the following information: "The present invention comprises an IRR data acquisition means for acquiring IRR data, an indicator value calculation means for calculating indicator values ​​for a plurality of indicators relating to the activity of the autonomic nervous system on the basis of said IRR data, a storage means in which sleepiness estimation rules are stored, a sleepiness data acquisition means for acquiring output information from a sleepiness information output device that delivers information on the presence or absence of sleepiness, which is information obtained from the subject or information obtained by analyzing the biological state of the subject,a means for generating reference data to evaluate the relevance of the drowsiness estimation rule using the drowsiness data, and an update means for updating the drowsiness estimation rules stored in said storage means based on said indicator values ​​and said reference data.

[0003] In recent years, attention has focused on technologies for measuring a driver's biological information, such as their pulse and heart rate, in order to determine their biological state, such as their level of drowsiness, fatigue or concentration, and then to automatically control the vehicle, based on the results of the determination, by issuing alerts or stopping the vehicle in order to avoid accidents.

[0004] In order to determine the biological state, the measurement data are corrected to account for differences in characteristics between the various sensor models, individual differences in the measurement target, and differences in the measurement method. However, this is complicated by the fact that the correction process and conditions must be defined in advance for each sensor. Therefore, there is a need for a more precise determination of the biological state without requiring correction to absorb differences in sensor characteristics.

[0005] Patent document JP2016-120062 discloses a device that uses 1TRR (RR interval: interval between two heartbeats) to determine the Drowsiness. According to the technology described in the document, IRR data are corrected to account for differences in characteristics between different sensor models, and the sensor characteristics used for this correction are predefined. In other words, the technology does not address the problem of determining a biological state without requiring correction to account for differences in sensor characteristics.

[0006] The objective of the present invention is therefore to determine a biological state with greater precision, without the need to make corrections to take into account differences in sensor characteristics, etc.

[0007] The present invention proposes a plurality of means for solving at least some of the aforementioned problems, some examples of which are presented below. An information processing device according to one aspect of the present invention that solves the aforementioned problem comprises an image information generation module that generates image information based on biological information, a model building module that builds a machine learning model using said image information, and a biological state determination module that inputs said image information into said learning model and obtains an output value that is the result of determining said biological state.

[0008] In one embodiment, the information processing device includes a storage module for storing at least one of said learning models built using said biological information obtained by measurement on a user and using said biological information obtained by measurement on individuals belonging to a predetermined category defined according to a combination of types and ranges of physical characteristics.

[0009] According to another aspect, the invention relates to a method for determining a biological state, performed by an information processing device, such that the information processing device performs:

[0010] a step of generating image information based on biological information;

[0011] a step of constructing a machine learning model using said image information;

[0012] an input step of said image information into said learning model, and of obtaining an output value which is the result of the determination of said biological state.

[0013] According to one embodiment, in the image information generation step, said biological information is used to generate information image showing a graph of a Lorenz curve representing the fluctuation of the heartbeat interval.

[0014] According to the present invention, a biological state can be determined more precisely without the need to make corrections to take into account differences in sensor characteristics, etc.

[0015] Problems, configurations or results other than those described above will be addressed in the description of embodiments provided below. Brief description of the drawings

[0016] [Fig-1] Fig. 1 represents an example of a schematic configuration of a information processing device according to the first embodiment.

[0017] [Fig.2] The [Fig.2] represents a functional diagram of an example of a functional configuration of the computing device of an information processing device.

[0018] [Fig. 3] Fig. 3(a) is an example of image information for a Lorenz curve, which represents the fluctuation of the interval between heartbeats when the user is not feeling drowsy. Fig. 3(b) is an example of image information for a Lorenz curve, which represents the fluctuation of the interval between heartbeats when the user is feeling drowsy.

[0019] [Fig.4] Fig.4 represents a flow diagram of an example of a process for building a learning model.

[0020] [Fig.5] The [Fig.5] represents a flow diagram of an example of a biological state determination process.

[0021] [Fig.6] The [Fig.6] represents a functional diagram of an example of a functional configuration of an information processing device and a server device according to a second embodiment.

[0022] Examples of embodiments of the present invention are provided below. Implementation method 1

[0023] Figure 1 represents an example of a schematic configuration of an information processing device 100. The information processing device 100 is intended for determining the biological state of a user (e.g., a driver) moving within a moving body, such as a vehicle, using biological information. More specifically, the information processing device 100 uses biological information (e.g., IRR:RR interval data) obtained from heart rate and pulse measurements, among other things, to generate graph image information (Lorenz curve) showing fluctuations in the heart rate interval, and between these information in a learning model to determine biological state, such as level of sleepiness.

[0024] As illustrated, the information processing device 100 comprises a computing device 110, a sensor 120, a communication device 130, and a storage device 140. Furthermore, the computing device 110, the communication device 130, the storage device 140, and the sensor 120 of the information processing device 100 may be housed in a single enclosure, or the computing device 110, the communication device 130, the storage device 140, and the sensor 120 may each be housed in separate enclosures. In addition, the computing device 110 and the sensor 120 are connected to each other by transmission cables or a predetermined wireless communication protocol (e.g., Bluetooth: registered trademark).

[0025] The calculating device 110 is a device that performs various arithmetic processing operations of the information processing device 100. As shown in the figure, the calculating device 110 comprises a central processing unit (CPU) 111 which performs various arithmetic processing operations carried out in the information processing device 100, a read-only memory (ROM) 112 which stores, among other things, a program executed by the CPU 111, a random access memory (RAM) 113 which temporarily stores various information read from the ROM 112, and an interface (I / F) 114 for electrically connecting the calculating device 110 with the sensor 120, as well as a bus 115 for interconnecting them.

[0026] The computing device 110 obtains biological information (e.g., IRR data indicating the heartbeat interval) from the sensor 120 and uses it to generate image information of a Lorenz curve graph representing fluctuations in the heartbeat interval (hereafter sometimes referred to as "Lorenz curve image information"). The computing device 110 also builds (generates) a learning model by performing machine learning using the generated image information.

[0027] The computing device 110 obtains an output value, which is the result of the arithmetic processing by the learning model, by entering into the learning model the image information generated using the biological information obtained from the sensor 120. The output value is the result of determining the biological state using the image information, and corresponds to a biological state such as, for example, drowsy, not drowsy, tired, not tired, focused, not focused.

[0028] The computing device 110 converts the output values ​​of the learning model into a predetermined data format that can be used by the main unit 200 (for example, an embedded device such as a navigation device or a a digital audio unit (D / A), or an engine control unit (ECU), and outputs them to these devices. The on-board or similar device uses the result of the biological state determination obtained from the information processing device 100, and executes a process (for example, playing music, or controlling the air conditioning to change the temperature in the car) according to the user's biological state.

[0029] Sensor 120 is a measuring device intended to measure vital data, such as heart rate and pulse, of a user. More specifically, Sensor 120 is a contact sensor capable of detecting the user's heart rate (or pulse), such as a piezometer integrated into a vehicle seat, or a wearable device worn in contact with the body, such as a watch. Alternatively, Sensor 120 can be a non-contact sensor such as a Doppler sensor that emits microwaves or millimeter waves onto the human body and detects the user's heart rate based on the reflected waves. Alternatively, Sensor 120 can be a camera capable of detecting heart rate based on the flow of hemoglobin in the blood.

[0030] The sensor 120 measures the user's heart rate over a predetermined period of time (for example, 5, 10 or 30 minutes from the start of the measurement) and generates IRR data as biological information, which is transmitted to the information processing device 100. The IRR data is information on the heartbeat interval (interval between R waves), which corresponds to the time interval between two heartbeats.

[0031] The type of sensor 120 is not limited to a particular type, but can be any measuring device capable of measuring vital data such as a user's heart rate or pulse. For example, sensor 120 can be a measuring device capable of backcalculating a user's heart rate interval over a predetermined period of time based on the detected heart rate and generating IRR data.

[0032] The communication device 130 is a communication module or analog intended to communicate information with an external device (e.g., the main unit 200). The communication device 130 is connected to the main unit 200 by a transmission cable or a predetermined wireless communication protocol (e.g., Bluetooth: registered trademark), and transmits the results of the biological status determination to the main unit 200.

[0033] The storage device 140 is a hard disk drive (HDD), a solid-state drive (SSD), a flash memory device, or any other non-volatile storage device capable of storing digital information. The storage device 140 stores, for example, various information used for building learning models as well as the learning models built.

[0034] Fig. 2 represents a functional diagram of an example of a functional configuration of the computing device 110 of an information processing device 100. As illustrated, the computing device 110 comprises a computing module 300, a control module 310, a storage module 320, and a communication module 330.

[0035] The computing module 300 is a functional module that performs various processing tasks carried out by the computing device 110. More specifically, the computing module 300 includes an image information generation module 301, a labeling module 302, a model building module 303, a user authentication module 304, a biological state determination module 305, and a data format conversion module 306. The determination of drowsiness is presented below as an example of biological state determination.

[0036] The image information generation module 301 is a functional module that generates image information using biological information obtained from sensor 120. More specifically, the image information generation module 301 uses IRR data acquired from sensor 120 to generate image information of a Lorenz curve representing the fluctuation of the heartbeat interval over a predetermined period.

[0037] More specifically, using IRR data, the image information generation module 301 takes the time interval between the start of the measurement and the first (Nth (N = 1)) heartbeat on the horizontal axis (X axis), and the time interval between the first (Nth (N = 1)) and the next (N + th (N = 1)) heartbeat on the vertical axis (Y axis), and plots their points of intersection. The image information generation module 301 then takes the time interval between the first (Nth (N = 1)) heartbeat and the next (N + th (N = 1)) heartbeat on the horizontal axis (X axis) and the time interval between the next (N + th (N = 1)) heartbeat and the next (N + 1 (N = 2)) heartbeat on the vertical axis (Y axis), and plots their points of intersection.Thus, the image information generation module 301 generates a graph of a Lorenz curve by repeating the same process for all heartbeat intervals measured within a given period.

[0038] The image information generation module 301 also generates image information by converting the generated Lorenz curve graph into a predetermined format.

[0039] Figure 3(a) is an example of image information for a Lorenz curve, which represents the fluctuation of the interval between heartbeats when the user is awake. Figure 3(b) is an example of image information for a Lorenz curve, which represents the fluctuation of the interval between heartbeats when the user is awake. As these figures show, when the user is awake, the heartbeat interval varies little, so the position of each trace in the graph is not spread out and is concentrated in a certain area. On the other hand, when the user is awake, the variation in the heartbeat interval is significant, so the position of each trace in the graph is dispersed over a wider area.Thus, Lorenz curve image information based on IRR data can be used to determine the user's biological state, thanks to the specific shape representing the user's biological state (e.g., drowsy or not, etc.).

[0040] The labeling module 302 is a functional module that performs a labeling process. More specifically, the labeling module 302 performs a labeling process that matches the correct labels indicating a user's biological state to Lorenz curve image information. More specifically, the labeling module 302 performs a labeling process that obtains the correct labels indicating whether the user felt drowsy or not during the measurement period and matches them to the Lorenz curve image information generated using the user's biological information during the corresponding period.

[0041] The labeling module 302 obtains the correct label by means of a predetermined process. For example, after the end of the measurement period, the labeling module 302 displays information on the display device provided by the information processing module 100 or the main unit 200 to receive a response from the user as to whether or not drowsiness was felt during the measurement period, and obtains the correct label by receiving the response directly from the user.

[0042] Alternatively, the labeling module 302 captures the user's facial expression during the measurement period using an on-board camera capable of capturing images of the vehicle's interior, and by performing facial authentication using this image information, obtains the correct label indicating whether the user was asleep or not during the measurement period. The facial authentication technology should be based on a known technology, such as the NEDO (New Energy Industrial Technology Development) evaluation method. Organization - Organization for the development of new energies and industrial technologies).

[0043] The method for obtaining a correct label shall in no way be limited, and any method may be used as long as the correct label indicating whether the user felt drowsy or not during the measurement period can be obtained.

[0044] The model building module 303 is a functional module that builds a learning model 322. More specifically, the model building module 303 builds the learning model 322 by performing machine learning using Lorenz curve image information generated by the image information generation module 301 and the correct answer label associated with this image information. The process of building the learning model 322 will be described below.

[0045] The user authentication module 304 is a functional module that performs user authentication based on user information. This user information includes a user ID, which identifies the user, and a category to which the user belongs from among predetermined categories defined based on physical characteristics. The user authentication module 304 retrieves the learning model 322 associated with the user ID or the user category from the storage module 320.

[0046] The biological state determination module 305 is a functional module which inputs Lorenz curve image information generated using biological information (IRR data) acquired from a sensor 120 into the learning model 322 and obtains output values ​​indicating the results of the biological state determination.

[0047] The data format conversion module 306 is a functional module that acquires the output values ​​(information representing the result of the biological state determination) resulting from the calculation of the learning model 322 and converts these output values ​​into a data format that can be used by the main unit 200. The data format conversion module 306 also sends the converted output values ​​(results of the biological state determination) to the main unit 200 via the communication module 330. The data format to be converted is not limited, as long as it is a data format that can be used by the main unit 200.

[0048] The control module 310 is a functional module that controls the operation of a sensor 120. More specifically, the control module 310 sends instructions to the sensor 120 to measure biological information. Furthermore, The control module 310 sends instructions to the sensor 120 for predetermined control objects, such as user position recognition, user measurement duration, and the frequency range of the radio wave to be emitted.

[0049] The storage module 320 is a functional module for storing various data. More specifically, the storage module 320 stores various basic models 321 that aid in building the learning model 322. A basic model 321 is, for example, a convolutional neural network (CNN), which exhibits excellent image recognition performance. The storage module 320 also stores the constructed learning models 322.

[0050] There are a plurality of learning models 322, constructed by taking measurements on a user of a vehicle in which the information processing device 100 is mounted and by taking measurements on individuals (other than the user) belonging to each category defined on the basis of physical characteristics. Furthermore, these learning models 322 are stored in the storage module 320 with user IDs and the categories of types associated with them.

[0051] The communication module 330 is a functional module that ensures information communication with the main unit 200. More specifically, the communication module 330 transmits the results of the biological state determination to the main unit 200. The communication module 330 can also obtain the correct response label from the main unit 200.

[0052] The functional configuration of the information processing device 100 has been described above.

[0053] The computing module 300 and the control module 310 are implemented by means of programs that cause the central processing unit 111 of the computing device 110 to perform a processing operation. These programs are stored, for example, in ROM 112, loaded into RAM 113 for execution, and executed by the central processing unit 111. The control module 310 can be implemented by means of a predetermined control circuit. The storage module 320 is implemented by means of RAM 113 or ROM 112 or storage device 140, or a combination thereof. The communication module 330 is implemented by means of communication device 130.

[0054] Each functional block of the information processing device 100 is classified according to the main content of the processing in order to facilitate understanding of each function performed in the present embodiment. Consequently, the present invention is not limited by the way in which each function is categorized or designated. Each of the components of the information processing device 100 can also be divided into several components according to the content of the processing. It is also possible to use a distribution such that one component can perform several processing tasks.

[0055] All or part of each functional module may be based on hardware (for example, an integrated circuit, such as an ASIC) implemented in a computer. Furthermore, the processing of each functional module may be performed by one or more hardware devices. Description of operation

[0056] The process of constructing a learning model executed by the information processing device 100 will now be described.

[0057] Figure 4 represents a flow diagram of an example of a process for building a learning model. Such a process is initiated, for example, when a user (or someone other than the user) receives an instruction to execute the learning model building process while driving. The case where a learning model 322 is built for a user is described below as an example.

[0058] Once the process has started, steps S001 to S005, described below, will generate image information that will be used for machine learning. More specifically, the control module 310 issues (step S001) instructions to the sensor 120 to perform (start) the measurement of a user's (e.g., a driver's) biological data. Based on these instructions, the sensor 120 takes measurements on the user for a predetermined period of time (e.g., 5 minutes), generates IRR data based on the measurement data, and sends the data to the image information generation module 301.

[0059] Next, the model building module 303 acquires (step S002) the IRR data from the sensor 120. In addition, the model building module 303 temporarily stores the acquired IRR data in the storage module 320.

[0060] Next, the model building module 303 performs (step S003) data cleaning. More specifically, the model building module 303 removes outliers and performs additional processing of information that cannot be read as numeric values ​​(NaN) from the acquired IRR data. The additional processing is performed in a predetermined manner, for example, by adopting the midpoint of the values ​​before and after the location of the NaN data.

[0061] Next, the image information generation module 301 generates (step S004) a graph of a Lorenz curve based on the IRR data and generates the image information of such a graph.

[0062] Next, the labeling module 302 performs (step S005) a labeling process on the generated image information. More specifically, the labeling module 302 obtains from the user a correct label indicating the biological state during the measurement period of the IRR data corresponding to the generated image information, and associates it with the image information.

[0063] The process of steps S001 to S005 generates information for machine learning by matching image information of a Lorenz curve, generated using IRR data of a user measured over a predetermined period of time, with a correct response label indicating the user's biological state during that period.

[0064] Next, the model building module 303 determines (step S006) whether a predefined number (e.g., 100) of machine learning information elements has been generated or not. It is assumed that the predetermined number of information elements for machine learning includes Lorenz curve image information showing the fluctuation of the heartbeat interval in each case where the user feels drowsy or not in a predetermined ratio (e.g., an equal number for each state).

[0065] Then, if it is determined that the predetermined number of information elements has not been generated (No at step S006), the model building module 303 returns the process to step S001. On the other hand, if it is determined that the predetermined number of machine learning information elements has been generated (Yes at step S006), the model building module 303 moves the process to step S007.

[0066] In step S007, the model building module 303 divides the information intended for machine learning into arbitrary proportions. More specifically, the model building module 303 divides the information for machine learning—that is, the Lorenz curve image information to which a correct response label indicating the biological state is assigned—into training data and validation data, using the holdout accuracy estimation method described below. The division ratio between the training data and the validation data is arbitrary. For example, the training data and the validation data are divided into arbitrary proportions such as 6:4 or 7:3.

[0067] Next, the model building module 303 constructs the learning model 322 through the processing tasks of steps S008 to S0011. More specifically, the model building module 303 determines (step S008) the base model 321. More specifically, the model building module 303 extracts a predetermined base model 321 (for example, a network neural convolutional) from the storage module 320 and defines it as a basic model 321 to be used for model building.

[0068] Next, the model building module 303 performs (step S009) machine learning using the training data. The machine learning process will now be described. The model building module 303 inputs image information from the training data into the transfer function indicated by the determined base model 321, such as a convolutional neural network, and compares the output values ​​of the base model 321 as a prediction result for the correct answer label with the correct answer label associated with the input image information. Furthermore, if there is a difference in the output value for the correct label, the model building module 303 returns the information and modifies the studied variables of the transfer function.The model building module 303 performs machine learning by running such prediction and feedback processes for all training data, and builds the learning model 322.

[0069] Next, the model building module 303 performs (step S010) a verification of the accuracy of the learning model 322 using the validation data. More specifically, the model building module 303 inputs validation data into the constructed learning model 322. Furthermore, the model building module 303 verifies the accuracy of the constructed learning model 322 by comparing the output values ​​of such a learning model 322 with the correct response label associated with the image information in the validation data.

[0070] Next, the model building module 303 determines (step SOI 1) whether the accuracy of the learning model 322 is sufficient. More specifically, the model building module 303 bases its decision on whether the accuracy of the learning model 322, verified in step S010, meets a predetermined standard value (e.g., 80% or more correct answers) that is deemed sufficient. If the accuracy is deemed sufficient (Yes in step SOI 1), the model building module 303 proceeds the process to step S012. Conversely, if it is determined that the accuracy is insufficient (No in step SOI 1), the model building module 303 returns the process to step S001.

[0071] In the processes of steps S009 and S010, which are executed after a return from step S011, the model building module 303 executes these processes using new training data and new validation data. More specifically, the model building module 303 sends instructions for performing new measurements to sensor 120 via the module command 310 at step S001, and executes the same process as in steps S002 to S010 described above using the new biological information acquired.

[0072] At step S012, the model building module 303 stores (saves) in the storage module 320 predetermined categories defined on the basis of the user ID and physical characteristics by matching them with the constructed learning model 322.

[0073] The process of this flow is completed when the model building module 303 saves the learning model 322.

[0074] In the case of constructing a learning model 322 for an individual other than the user, the model building module 303 constructs a learning model 322 corresponding to each category by taking measurements on a person belonging to each category defined on the basis of physical characteristics. The categories are classified according to the type of physical characteristics and the combination of their ranges. Two examples of categories could be: "sex = male, age = 30 years, height = 170 cm-180 cm, weight = 60 kg-70 kg, chronic disease = none" and "sex = female, age = 40 years, height = 150 cm-160 cm, weight = 50 kg-60 kg, chronic disease = none".

[0075] The model building module 303 performs measurements on people from the statistical population belonging to the defined categories, and builds a learning model 322 corresponding to each category using the image information generated on the basis of the biological information thus acquired.

[0076] Therefore, in the biological state determination process described below, the user can carry out the biological state determination by applying the learning model 322 built by measurements on himself or on individuals belonging to the same category as the user, on the basis of user authentication.

[0077] The biological state determination process carried out by the information processing device 100 will now be described.

[0078] Figure 5 shows a flow diagram of an example of a biological state determination process. This process is initiated, for example, when the vehicle's engine is started (when the ignition key is in the ON position) upon receipt of a process start instruction from the user. Engine start can be detected by the user authentication module 304, which acquires information indicating that the engine has started from the engine control unit via the communication module 330, for example. The user start instruction can be triggered, for example, by pressing a process start control button provided by the processing device. information 100, which can be detected by user authentication module 304.

[0079] When processing is initiated, the user authentication module 304 performs (step S021) a personal authentication of the user. More specifically, the user authentication module 304 retrieves the learning model 322 associated with the user ID entered via the predetermined input device from the storage module 320. If such a learning model 322 does not exist, the user authentication module 304 retrieves a learning model 322 associated with a category to which the user belongs from the storage module 320.

[0080] The predetermined input device can be a main module 200, such as a navigation device, for example. In this case, the user authentication module 304 can obtain the user's input information from the main unit 200 via the communication module 330. The input device can also be provided by the information processing device 100.

[0081] The authentication method is not limited to the input of user information. For example, when the vehicle is equipped with an on-board camera that captures images inside the vehicle, the user authentication module 304 can acquire the image information captured by the on-board camera and use it to perform facial recognition to identify the user's physical characteristics. When this method is used, the information processing device 100 must be connected to the on-board camera via the communication device 130 to enable the communication of information.

[0082] The authentication method can also be fingerprint authentication. When this method is used, it is sufficient for the information processing device 100 to be equipped with a fingerprint authentication sensor. Alternatively, when a fingerprint authentication sensor is installed at a predetermined location in the vehicle (e.g., a steering wheel, etc.), the information processing device 100 can identify the user's physical characteristics by acquiring fingerprint information from the engine control unit connected via the communication device 130, for example, and matching the fingerprint information to the fingerprint information already saved in the storage module 320.

[0083] Next, the user authentication module 304 determines (step S022) whether or not a corresponding learning model 322 exists. More specifically, the user authentication module 304 performs such a determination based on This depends on whether the learning model 322, to which the user ID or the user's category is associated, is stored in the storage module 320. If a matching learning model 322 is found (Yes at step S022), the user authentication module 304 proceeds the process to step S023. Conversely, if no matching learning model 322 is found (No at step S022), the user authentication module 304 proceeds the process to step S030.

[0084] At step S030, the biological state determination input module 305 notifies the user of the absence of a corresponding learning model 322. For example, the biological state determination module 305 generates a predetermined message notifying the absence of a corresponding learning model 322 and displays this message on an on-board equipment display device, namely the main unit 200, via the communication module 330. After displaying such a message on the main unit 200, the biological state determination module 305 terminates the process of this flow.

[0085] In the process of step S023, to which the flow passes when it is determined that there is a corresponding learning model 322, the user authentication module 304 obtains the corresponding learning model 322 from the storage module 320.

[0086] Next, the biological state determination module 305 sends (step S024) an instruction to measure vital data, such as heart rate, to the sensor 120 via the control module 310. Then, the biological state determination module 305 acquires (step S025) the IRR data emitted at the output of the sensor 120 and introduces them (step S026) into the retrieved learning model 322.

[0087] Next, the biological state determination module 305 obtains (step S027) the output value resulting from the calculation of the learning model 322, that is to say the result of the determination concerning the presence or absence of drowsiness of the user during the measurement period (drowsy / not drowsy).

[0088] In addition, the data format conversion module 306 converts (step S028) the output values ​​into a predetermined data format, and delivers them as output (step S029) to the main unit 200 via the communication module 330. The process of this flow is completed when the data format conversion module 306 sends the output values, i.e. the results of the biological status determination, to the main unit 200.

[0089] The information processing device 100 of this embodiment has been described above.

[0090] This type of information processing device does not require correction to account for differences in sensor characteristics, etc., and allows to determine a biological state more accurately. In particular, the information processing device uses IRR data, which is biological data, to generate Lorenz curve image information, which is a visual representation of heart rate interval fluctuations, and uses this image information to determine a biological state. For example, when IRR data is fed into a machine learning model to determine a biological state, it is necessary to define the correction process and conditions in advance, because differences in sensor characteristics are reflected in the numerical data.However, depending on the information processing device, the biological state can be accurately determined without requiring correction and adjustments for correction, because the biological state is determined using image information that allows a visual distinction of the trend indicating the biological state.

[0091] Embodiment 2

[0092] Figure 6 shows a functional diagram of an example of a functional configuration of an information processing device 100 and a server device 400 according to the present embodiment. As illustrated, the information processing device 100 is interconnected to the server device 400 via a predetermined N-network, such as a LAN (Local Area Network), for example.

[0093] Since the image information generation module 411, the labeling module 412, the model building module 413, the storage module 420, the base model 421, and the communication module 430 contained in the server device 400 are each functional modules and information that serve the same processing as the image information generation module 301, the labeling module 302, the model building module 303, the storage module 320, the base model 321, and the communication module 330 in the information processing device 100 of the first embodiment, their detailed description is omitted in this embodiment.

[0094] In the first embodiment described above, the learning model 322 was built by the information processing device 100, but in the present embodiment, the biological information acquired from the sensor 120 is transmitted to the server device 400, and the learning model 322 is built by the image information generation module 411, the labeling module 412, and the model building module 413 included in the computing module 410 of the server device.

[0095] More specifically, when the computing module 300 of the information processing device 100 acquires biological information (IRR data) at from sensor 120 during the process of building the learning model, it transmits the information to the server device 400 via the communication module 330. When the image information generation module 411, the labeling module 412 and the model building module 413 of the server device 400 acquire the IRR data via the communication module 430, the learning model 322 is built following the same process as that of steps S003 to S011 described above.

[0096] The learning model 322 built by the model building module 413 of the server device 400 is transmitted to the information processing device 100 via the communication module 430 of the server device 400 and stored in the storage module 320. The information processing device 100 also performs the same biological state determination process as described above using the learning model 322 stored in the storage module 320.

[0097] According to the information processing device 100 and the server device 400 of the second embodiment, since the construction of the learning model 322 can be carried out by the server device with high processing performance, the specifications of the information processing device 100 can be eliminated, and consequently, the manufacturing cost can be reduced.

[0098] The present invention is not limited to the information processing device 100 shown in the first and second embodiments, and various variations are possible. For example, the computing device 110 (computing module 300 and control module 310), the communication device 130, and the storage device 140 in the first embodiment can be integrated into an embedded device or an engine control unit, such as a navigation device, which constitutes the main unit 200.

[0099] Furthermore, although the above-mentioned embodiment uses determination results such as "drowsy / not drowsy" as output values, the present invention is not limited thereto, and the information processing device 100 can output determination results including a level of intensity of the biological state, for example, "a little sleepy", "very sleepy", "drowsy (low)", "drowsy (medium)" and "drowsy (high)".

[0100] Such a determination can be achieved, for example, by subdividing the correct labels into several levels for a biological state and matching them with image information, and then building a machine learning model using these labels.

[0101] According to this variant of information processing device 100, the biological state of the user can be determined at a more detailed level.

[0102] In the aforementioned embodiment, Lorenz curve image information has been generated, but the invention is not limited to this. Image information showing the waveform of a user's ECG can, for example, be generated and used to determine the biological state.

[0103] This variant of the information processing device also makes it possible to determine a biological state accurately, without requiring correction and adjustments for correction, because the biological state is determined using image information which allows a visual distinction of the trend indicating the biological state.

[0104] The present invention is not limited to the embodiments and variations described above, but also includes various other embodiments and variations. For example, the embodiments described above are detailed for the purpose of explaining the present invention in a way that is easy to understand, and are not necessarily limited to those that have all the configurations described. It is also possible to replace some of the configurations of an embodiment with the configurations of other embodiments or variations, and it is also possible to add the configurations of other embodiments to the configurations of an embodiment. It is also possible to add, delete, or substitute other configurations for some of the configurations of each embodiment.

[0105] Description of reference numbers

[0106] 100 Information processing device,

[0107] 110 Computing device,

[0108] 111 Central processing unit,

[0109] 112 ROM,

[0110] 113 RAM,

[0111] 114 EF (interface),

[0112] 115 Bus,

[0113] 120 Sensor,

[0114] 130 Communication device,

[0115] 140 Storage device,

[0116] 200 Main unit,

[0117] 300 Computing module,

[0118] 301 Image information generation module,

[0119] 302 Labeling module,

[0120] 303 Model building module,

[0121] 304 User authentication module,

[0122] 305 Biological state determination module,

[0123]

[0124]

[0125]

[0126]

[0127]

[0128] 306: Data format conversion module, 310: Control module, 320: Storage module, 321: Basic model, 322: Learning model, 330: Communication module

Claims

Demands

1. Information processing device comprising a computing device (110) characterized in that the computing device (110) is configured to implement: an image information generation module (301) that generates image information showing a graph of a Lorenz curve representing fluctuations of a heartbeat interval based on biological information; a model building module (303) that builds a machine learning model using said image information;and a biological state determination module (305) which inputs said image information showing a graph of a Lorenz curve representing the fluctuations in a user's heartbeat interval, generated by the image information generation module (301), into said learning model and obtains an output value which is the result of determining said biological state of the user.

2. Information processing device according to claim 1, characterized in that it further comprises a communication module (330) which transmits said results of the determination of the biological state to a main unit.

3. Information processing device according to any one of claims 1 or 2, wherein a biological state is the presence or absence of drowsiness, fatigue or lack of concentration.

4. Information processing device according to any one of claims 1 to 3, characterized in that it comprises: a storage module (320) for storing at least one of said learning models built using said biological information obtained by measurement on a user and using said biological information obtained by measurement on individuals belonging to a predetermined category defined according to a combination of types and ranges of physical characteristics.

5. Information processing device according to claim 4, wherein said learning model is stored in the storage module with the user identification information

6. or said user identification information and said predetermined categories; and characterized in that it further comprises a user authentication module which extracts from said storage module said learning model corresponding to said user's identification information or to said category to which said user belongs. A method for determining a biological state performed by an information processing device according to claims 1 to 5, characterized in that a computing device of said information processing device performs: an image information generation step showing a graph of a Lorenz curve representing a fluctuation of a heartbeat interval based on biological information; a step in constructing a machine learning model using said image information; an image information input step, showing a graph of a Lorenz curve representing the fluctuations in the heartbeat interval of a user in said learning model, and obtaining an output value which is the result of determining said biological state of the user.

Images