Information processing device

Abstract

To calculate a degree of concentration of an object person with a higher degree of precision.SOLUTION: An information processing device 10 includes a determination unit 12 for determining whether or not an object person is oriented to a specific direction in which the object person is determined to be in a concentrated state on the basis of moving image data capturing the face of the object person a plurality of times with a lapse of time in between, and a calculation unit 13 for calculating a degree of concentration of the object person on the basis of determination results a plurality of times by the determination unit 12.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus that calculates the concentration of a target person.

Background Art

[0002] As a usage form using a remote communication system, for example, remote classes (distance learning) are known. In a remote class, it is common for listeners to listen to the instructor's explanation while viewing the material content whose screen is shared. In such a remote class, it is not known from the instructor whether the listeners are concentrating on listening to the explanation.

[0003] In Patent Document 1 below, a technique for calculating concentration from the state of the orientation of the face of a target person, specifically, a technique for calculating concentration such that when the target person faces the front, which is the reference direction, the concentration is 100, and the larger the angle deviated from the front, the lower the value, has been proposed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, it cannot be generally said that the greater the angle deviated from the front, the lower the concentration. Therefore, from the viewpoint of calculating the concentration more accurately, there is still room for improvement.

[0006] Therefore, an object of the present disclosure is to calculate the concentration of a target person more accurately.

Means for Solving the Problems

[0007] The information processing apparatus according to the present disclosure includes a determination unit that determines, a plurality of times at intervals of time, whether or not the subject is facing a specific direction determined to be a concentrated state based on moving image data obtained by photographing the face of the subject, and a calculation unit that calculates the degree of concentration of the subject based on the plurality of determination results by the determination unit.

Advantages of the Invention

[0008] According to the present disclosure, the degree of concentration of the subject can be calculated with higher accuracy.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Modes for Carrying Out the Invention

[0010] Hereinafter, various embodiments of the information processing apparatus according to the present disclosure will be described in order with reference to the drawings. First, as a first embodiment, based on the recognition results of both the face orientation and the behavior of the target person based on the moving image data of the target person, it is determined at regular intervals whether the target person is facing a specific orientation determined to be a concentrated state, and the concentration degree of the target person is calculated based on the determination results of multiple times. Next, as a second embodiment, when the target person is not wearing a mask, recognition of conversation and yawning actions (hereinafter referred to as "conversation / yawning recognition") is performed, and the concentration degree of the target person is calculated based on the recognition results. In the following first and second embodiments, an example in which the "specific orientation determined to be a concentrated state" is the "front" of the face of the target person will be described. However, depending on the application, an orientation other than the front may be used as the "specific orientation determined to be a concentrated state", and its variations will be described later.

[0011] [First Embodiment] As shown in FIG. 1, the information processing apparatus 10 in the present embodiment includes, as functional blocks, an acquisition unit 11, a determination unit 12, a calculation unit 13, and an output unit 14. Among these, the acquisition unit 11 is a functional unit that acquires moving image data obtained by photographing the face of the target person. The acquisition source of the moving image data may be a camera (not shown) mounted on the information processing apparatus 10, or an external camera independent of the information processing apparatus 10.

[0012] The determination unit 12 is a functional unit that determines multiple times at intervals whether the subject is facing forward based on the acquired moving image data. In this embodiment, using existing technologies, both the orientation of the subject's face and the subject's actions are recognized from the moving image data, and based on these recognition results, it is determined whether the subject is facing forward. For example, when the determination unit 12 recognizes an action of diverting the face or line of sight from the front as the subject's action, it determines that the subject is not facing forward. That is, if an action of diverting the face from the front or an action of diverting only the line of sight without diverting the face from the front is recognized, it is determined that the subject is not facing forward. Also, for the above-mentioned multiple determinations, the determination unit 12 performs them "multiple times at regular intervals". Hereinafter, as an example, an example will be described in which, for 100 seconds of moving image data, a total of 100 determinations are made every 1 second. That is, the first determination is made based on the moving image data of the first 1 second (elapsed time 0 to 1 second) out of 100 seconds of moving image data, the second determination is made based on the moving image data of the next 1 second (elapsed time 1 to 2 seconds), and the last 100th determination is made based on the moving image data of the last 1 second (elapsed time 99 to 100 seconds).

[0013] The calculation unit 13 is a functional unit that calculates the concentration of the subject based on the determination result by the determination unit 12. Although the details will be described later, when "sleep" is recognized as the subject's action, the calculation unit 13 sets the concentration of the subject to 0.0 (that is, the lowest value in the range of values that the concentration can take in this embodiment (0.0 to 1.0)) regardless of the determination result of whether the subject is facing forward.

[0014] The output unit 14 is a functional unit that outputs the "concentration of the subject" obtained by the calculation by the calculation unit 13. Note that "output" can take various forms such as display output, print output, and data transmission to the outside of the information processing apparatus 10.

[0015] Hereinafter, the processing executed by the information processing apparatus 10 according to the first embodiment will be described with reference to the flowchart of FIG. 2.

[0016] First, the acquisition unit 11 acquires moving image data obtained by photographing the face of the target person (step S1 in FIG. 2) and passes it to the determination unit 12. The determination unit 12 detects the face of the target person from the moving image data by the method of an existing face detection technique (step S2), and determines whether or not the detected face is facing forward (step S3). Here, if the face is not facing forward, the determination unit 12 determines that "the target person is not facing forward" (step S4).

[0017] On the other hand, if the face is facing forward in step S3, the determination unit 12 recognizes the behavior of the target person (for example, sleeping, turning the face away from the front, not turning the face away from the front but only shifting the line of sight, etc.) from the moving image data by the method of an existing behavior recognition technique (step S5), and determines whether or not the target person has turned the face or the line of sight away from the front based on the recognition result (step S6). Here, if it is determined that the target person has turned the face or the line of sight away, the determination unit 12 determines that "the target person is not facing forward" (step S4).

[0018] On the other hand, if it is not determined in step S6 that the target person has turned the face or the line of sight away (that is, if it is determined that neither the face nor the line of sight has been turned away), the determination unit 12 determines that "the target person is facing forward" (step S7). Then, in step S8, the determination unit 12 stores the determination results of steps S4 and S7 and the behavior recognition result of step S5 in a built-in memory (not shown).

[0019] The processes of steps S2 to S8 above are executed once per second for 100 seconds of moving image data, and are executed a total of 100 times. In the next step S9, it is determined whether or not the processing has been completed a predetermined number of times (here, 100 times). If the 100 times of processing have not been completed, a negative determination is made in step S9, and the process returns to step S2 above, and the processes of steps S2 to S8 in the next round are executed.

[0020] In this way, the processes of steps S2 to S8 are executed every second for 100 seconds of moving image data, and are executed a total of 100 times. As a result, 100 sets of determination results and action recognition results are stored by the determination unit 12.

[0021] After the 100 - time process is completed, if a positive determination is made in step S9, the process proceeds to step S10. At this point, the 100 sets of determination results and action recognition results are passed to the calculation unit 13, and the calculation unit 13 calculates the concentration degree of the target person according to the ratio of the number of times determined as "facing forward" in the 100 sets of determination results (step S10). For example, if the number of times determined as "facing forward" among 100 times is 70 times and the number of times determined as "not facing forward" is 30 times, then (70 times / 100 times) = 0.7 is calculated as the concentration degree.

[0022] In the next step S11, the calculation unit 13 checks whether "sleep" is recognized in the above - mentioned action recognition results. If "sleep" is not recognized here (negative determination in step S11), the calculated concentration degree is passed to the output unit 14, and the output unit 14 outputs the calculated concentration degree (step S13). Note that in step S11, considering the possible mis - recognition of actions, for example, if "sleep" is recognized in a predetermined number of times or more among the 100 sets of action recognition results, it may be regarded as "sleep" being recognized.

[0023] On the other hand, if "sleep" is recognized in step S11 (positive determination in step S11), the calculation unit 13 sets the concentration degree to 0.0 regardless of the value calculated in step S10, passes it to the output unit 14, and the output unit 14 outputs the set concentration degree "0.0" (step S13).

[0024] According to the first embodiment described above, based on the recognition results of both the orientation and behavior of the subject's face in the moving image data obtained by photographing the subject's face, it is determined whether the subject is facing a specific orientation (here, "front") determined to be the concentrated state at regular intervals (1 second), and the concentration of the subject is calculated based on the determination results of multiple times (a total of 100 times). As a result, the concentration of the subject can be calculated with higher accuracy.

[0025] In addition, based on both the recognition result of the "orientation of the face" and the recognition result of the "behavior" of the subject, it is determined whether the subject is facing the front. The determination based on the recognition result of the former "orientation of the face" corresponds to a determination using the angle of the direction in which the face is facing, and the determination based on the recognition result of the latter "behavior" corresponds to a determination based on the presence or absence of an action such as turning the face or line of sight away from the front. The two have different recognition logics. Therefore, by using the above two determination results with different recognition logics, it is possible to cover the omission of each other's determinations and further improve the calculation accuracy of the concentration. In addition, in the determination based on the above "behavior" recognition result, even when an action of turning only the line of sight away without turning the face away from the front is recognized in addition to the action of turning the face away from the front, it is determined that the subject is not facing the front, so a more accurate determination can be made.

[0026] Furthermore, when "sleep" is recognized, regardless of the value calculated in step S10, the concentration is set to the lowest value "0.0" within the range of possible values. Therefore, an appropriate value considering the recognition result of "sleep" can be calculated as the concentration.

[0027] Regarding the determination of whether the subject is facing the front, it is not essential to be based on both the recognition result of the "orientation of the face" and the recognition result of the "behavior" of the subject. Either one can be used as the basis. Also, it is not essential to determine whether the subject is facing the front "at regular intervals", and it may be determined multiple times at arbitrary intervals not limited to a fixed time.

[0028] [Second Embodiment] Next, as a second embodiment, an embodiment will be described in which conversation and yawn recognition are performed when the subject is not wearing a mask, and the concentration of the subject is calculated based on the recognition result.

[0029] The functional block configuration of the information processing apparatus 10 in the second embodiment is the same as the configuration of FIG. 1 described above. However, the determination unit 12 determines whether the subject is wearing a mask based on the moving image data obtained by photographing the face of the subject, and when the subject is not wearing a mask, recognizes the conversation action and yawn action of the subject based on the moving image data. It has an additional function. Further, the calculation unit 13 has an additional function of calculating the concentration of the subject based on the recognition results of the conversation action and yawn action of the subject by the determination unit 12. Specifically, when a conversation action is recognized, the calculation unit 13 calculates a value that does not exceed a preset upper limit value (for example, 0.5) for the conversation action as the concentration of the subject, and when a yawn action is recognized, the yawn action A value that does not exceed a preset upper limit value (for example, 0.25) is calculated as the concentration of the subject.

[0030] Hereinafter, the processing executed by the information processing apparatus 10 of the second embodiment will be described with reference to the flowchart of FIG. 3.

[0031] First, the acquisition unit 11 acquires moving image data obtained by photographing the face of the subject (step S21 in FIG. 3) and passes it to the determination unit 12. The determination unit 12 detects the face and mask of the subject from the moving image data by using existing face detection techniques and mask detection techniques (step S22), and determines whether the direction of the detected face is front (step S23). Here, if the face is not facing forward, the determination unit 12 determines that "the subject is not facing forward" (step S24).

[0032] On the one hand, if the face orientation is frontal in step S23, the determination unit 12 determines whether the subject is wearing a mask based on the detection result in step S22 (step S25). If wearing a mask here, it proceeds to step S27 described later. If not wearing a mask, the determination unit 12 performs conversation and yawn recognition (recognition of whether the subject is conversing or yawning) from the moving image data by means of existing techniques related to conversation and yawn recognition (step S26), and then proceeds to step S27.

[0033] In step S27, the determination unit 12 recognizes the subject's actions (for example, sleeping, turning the face away from the front, not turning the face away from the front but only shifting the line of sight, etc.) from the moving image data by means of existing action recognition techniques (step S27), and determines whether the subject has turned the face or the line of sight away from the front based on the recognition result (step S28). Here, if it is determined that the subject has turned the face or the line of sight away, the determination unit 12 determines that "the subject is not facing the front" (step S24).

[0034] On the other hand, if it is not determined in step S28 that the subject has turned the face or the line of sight away (that is, if it is determined that neither the face nor the line of sight has been turned away), the determination unit 12 determines that "the subject is facing the front" (step S29). Then, in step S30, the determination unit 12 saves the determination results of steps S24 and S29 above, the action recognition result of step S27, and the conversation and yawn recognition result of step S26 in a built-in memory (not shown).

[0035] The processing of steps S22 to S30 above is executed once per second for 100 seconds of moving image data, and is executed a total of 100 times. In the next step S31, it is determined whether the processing has been completed a predetermined number of times (here, 100 times). If the 100 - time processing has not been completed, a negative determination is made in step S31, and it returns to step S22 above, and the processing of steps S22 to S30 in the next round is executed.

[0036] In this way, the processes in steps S22 to S30 are executed every second for 100 - second moving - image data, and are executed a total of 100 times. As a result, 100 sets of determination results, action recognition results, and conversation / yawn recognition results are stored by the determination unit 12.

[0037] After the 100 - time process is completed, if it is positively determined in step S31, the process proceeds to step S32. At this point, the 100 sets of determination results, action recognition results, and conversation / yawn recognition results are passed to the calculation unit 13, and the calculation unit 13 calculates the concentration of the subject according to the ratio of the number of times determined as "facing forward" in the 100 sets of determination results (step S32). For example, if the number of times determined as "facing forward" is 70 times and the number of times determined as "not facing forward" is 30 times out of 100 times, the concentration (70 times / 100 times)=0.7 is calculated.

[0038] In the next step S33, the calculation unit 13 checks whether "sleep" is recognized in the above - mentioned action recognition results. If "sleep" is recognized here (positive determination in step S33), regardless of the value calculated in step S32, the calculation unit 13 sets the concentration to 0.0 and passes it to the output unit 14, and the output unit 14 outputs the set concentration "0.0" (step S39). Note that in step S33, considering the possibility of mis - recognition of actions, for example, if "sleep" is recognized in a predetermined number or more of the 100 - time action recognition results, it may be regarded as sleep being recognized.

[0039] On the one hand, if "sleep" is not recognized in step S33 (negative determination in step S33), it is confirmed whether "conversation" is recognized in the above-mentioned conversation and yawn recognition result. If "conversation" is recognized here (positive determination in step S35), the calculation unit 13 sets a value that does not exceed the upper limit value (here 0.5) preset for the conversation action as the concentration of the target person (step S36). For example, if the value calculated in step S32 does not exceed 0.5, the value calculated in step S32 is used as the concentration of the target person. If the value calculated in step S32 exceeds 0.5, 0.5 is used as the concentration of the target person. Then, the calculated concentration is passed to the output unit 14, and the output unit 14 outputs the calculated concentration (step S39). Similar to the above-mentioned recognition of "sleep", in step S35, considering the possibility of misrecognition of actions, for example, when "conversation" is recognized more than the preset reference number of times of conversation recognition among 100 times of conversation and yawn recognition results, it may be regarded that the conversation is recognized.

[0040] On the other hand, if "conversation" is not recognized in step S35 (negative determination in step S35), it is confirmed whether "yawn" is recognized in the above-mentioned conversation and yawn recognition result. If "yawn" is recognized here (positive determination in step S37), the calculation unit 13 sets a value that does not exceed the upper limit value (here 0.25) preset for the yawn action as the concentration of the target person (step S38). For example, if the value calculated in step S32 does not exceed 0.25, the value calculated in step S32 is used as the concentration of the target person. If the value calculated in step S32 exceeds 0.25, 0.25 is used as the concentration of the target person. Then, the calculated concentration is passed to the output unit 14, and the output unit 14 outputs the calculated concentration (step S39). Similar to the above-mentioned recognition of "sleep" and "conversation", in step S37, considering the possibility of misrecognition of actions, for example, when "yawn" is recognized more than the preset reference number of times of yawn recognition among 100 times of conversation and yawn recognition results, it may be regarded that the yawn is recognized.

[0041] According to the second embodiment described above, the same effects as those of the first embodiment described above can be achieved. Furthermore, when the subject is not wearing a mask, the conversation and yawning actions of the subject can be recognized, and in addition to the determination results of the multiple times (100 times) described above, the concentration of the subject can be calculated based on the conversation and yawning recognition results. Specifically, when a conversation action is recognized, a value that does not exceed a preset upper limit value (0.5) for the conversation action is used as the concentration of the subject, and when a yawning action is recognized, a value that does not exceed a preset upper limit value (0.25) for the yawning action is calculated as the concentration of the subject. In this way, by setting a value that does not exceed a preset upper limit value according to the recognized action as the concentration of the subject, a more appropriate value considering the recognition results of conversation and yawning respectively can be calculated as the concentration.

[0042] In the second embodiment, an example is shown in which both conversation and yawning are recognized, and both the conversation recognition result and the yawning recognition result are used as the basis for calculating the concentration. However, it is not essential to use both the conversation recognition result and the yawning recognition result as the basis, and either one can be used as the basis.

[0043] [Variations regarding a specific orientation determined as the concentrated state] In the first and second embodiments described above, the case where the "specific orientation determined as the concentrated state" is the "front" of the subject's face has been described. However, depending on the application, an orientation other than the front may be used as the "specific orientation determined as the concentrated state", and the following cases are assumed.

[0044] As a first case, a case can be cited where it is assumed that the subject is looking down and writing a memo or the like on a notebook on the desk. In this case, the "specific orientation determined as the concentrated state" is set to "front" and "downward", and it is determined that the subject is in a concentrated state when the subject's face is facing forward and when the subject's face is facing downward. On the other hand, when the subject's face is facing either left or right and when the subject's face is facing upward, it is determined that the subject is in a non-concentrated state, and the concentration can be calculated based on these determination results.

[0045] As a second case, there is a case where a display at which a subject looks in a concentrated state and a camera that photographs the subject's face are located in different directions with respect to the subject's face. For example, when the display is located in front of the subject's face and the subject appears to be facing right when the subject looking at the front display is imaged by the camera, it is determined that the state is concentrated only when the subject appears to be facing right in this way, and it is determined that the state is non-concentrated when the subject appears in a state other than facing right, and the concentration may be calculated based on these determination results.

[0046] The gist of the present disclosure resides in the following [1] to [6]. [1] A determination unit that repeatedly determines, at different times, whether or not the subject is facing a specific direction determined to be a concentrated state based on moving image data obtained by photographing the subject's face; A calculation unit that calculates the concentration of the subject based on the determination results of the plurality of times by the determination unit; An information processing apparatus comprising: [2] The information processing apparatus according to [1], wherein the determination unit performs the determination a plurality of times at regular intervals. [3] The information processing apparatus according to [1] or [2], wherein the determination unit recognizes at least one of the direction of the subject's face and the subject's behavior based on the moving image data, and determines whether or not the subject is facing the specific direction based on the recognition result. [4] The information processing apparatus according to [3], wherein when the determination unit recognizes, as the subject's behavior, a behavior of diverting the face or line of sight from the specific direction, the determination unit determines that the subject is not facing the specific direction. [5] The information processing apparatus according to [3] or [4], wherein when the calculation unit recognizes sleep as the subject's behavior by the determination unit, the calculation unit sets the concentration of the subject to the lowest value within the range of possible values. [6] The determination unit determines whether or not the subject is wearing a mask based on the moving image data, and when the subject is not wearing the mask, the determination unit recognizes at least one of the subject's conversation action and yawning action based on the moving image data, The calculation unit calculates the concentration of the target person based on the recognition result of the operation in addition to the determination results of the multiple times. The information processing apparatus according to any one of [1] to [5]. [7] When the conversation operation is recognized by the determination unit and when the yawning operation is recognized, the calculation unit calculates, as the concentration of the target person, a value that does not exceed a preset upper limit value for each case. The information processing apparatus according to [6].

[0047] [Explanation of terms, explanation of hardware configuration (Fig. 4), etc.] Note that the block diagrams used in the above description of the embodiments show blocks of functional units. These functional blocks (constituent parts) are realized by an arbitrary combination of at least one of hardware and software. Also, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one physically or logically combined device, or two or more physically or logically separated devices may be directly or indirectly (for example, using wired, wireless, etc.) connected and realized using these multiple devices. The functional block may be realized by combining software with the above one device or the above multiple devices.

[0048] Functions include, but are not limited to, judgment, decision, determination, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, solution, selection, selection, establishment, comparison, assumption, expectation, regarded as, notification (broadcasting), notification (notifying), communication (communicating), transfer (forwarding), configuration (configuring), reconfiguration (reconfiguring), allocation (allocating, mapping), assignment (assigning), etc. For example, a functional block (constituent part) that functions as transmission is called a transmission unit or a transmitter. As described above, the realization method is not particularly limited.

[0049] For example, an information processing apparatus or the like in an embodiment of the present disclosure may function as a computer that executes the processing of the present disclosure. FIG. 4 is a diagram showing an example of the hardware configuration of an information processing apparatus 10 according to an embodiment of the present disclosure. Physically, the above-described information processing apparatus 10 may be configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

[0050] In the following description, the term "apparatus" can be read as a circuit, a device, a unit, or the like. The hardware configuration of the information processing apparatus 10 may be configured to include one or more of each apparatus shown in the figure, or may be configured without including some of the apparatuses.

[0051] Each function in the information processing apparatus 10 is realized by causing the processor 1001 to load a predetermined software (program) onto hardware such as the processor 1001 and the memory 1002, so that the processor 1001 performs calculations and controls communication by the communication device 1004, or controls at least one of reading and writing data in the memory 1002 and the storage 1003.

[0052] The processor 1001 controls the entire computer by operating an operating system, for example. The processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, a register, and the like.

[0053] Also, the processor 1001 reads a program (program code), software module, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operations described in the above embodiments is used. Although it has been described that various processes are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. Note that the program may be transmitted from a network via a telecommunication line.

[0054] The memory 1002 is a computer-readable recording medium and may be composed of at least one of, for example, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. The memory 1002 may be referred to as a register, cache, main memory (main storage device), etc. The memory 1002 can store a program (program code), software module, etc. executable for implementing the wireless communication method according to an embodiment of the present disclosure.

[0055] Storage 1003 is a computer-readable recording medium, which may be composed of at least one of, for example, an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital versatile disc, a Blu-ray (registered trademark) disc), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 1003 may be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, a server, or other appropriate media including at least one of the memory 1002 and the storage 1003.

[0056] The communication device 1004 is hardware (a transmission / reception device) for performing communication between computers via at least one of a wired network and a wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 1004 may be configured to include, for example, a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc. in order to implement at least one of frequency-division duplexing (FDD: Frequency Division Duplex) and time-division duplexing (TDD: Time Division Duplex).

[0057] The input device 1005 is an input device for receiving external input (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.). The output device 1006 is an output device for performing external output (e.g., a display, a speaker, an LED lamp, etc.). Note that the input device 1005 and the output device 1006 may have an integrated configuration (e.g., a touch panel).

[0058] Also, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus or may be configured using different buses for each device.

[0059] Further, the information processing apparatus 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and part or all of each functional block may be implemented by the hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.

[0060] The notification of information is not limited to the modes / embodiments described in the present disclosure, and other methods may be used. For example, the notification of information may be performed by physical layer signaling (e.g., downlink control information (DCI), uplink control information (UCI)), upper layer signaling (e.g., radio resource control (RRC) signaling, medium access control (MAC) signaling, notification information (master information block (MIB), system information block (SIB))), other signals, or a combination thereof. Further, the RRC signaling may be referred to as an RRC message, and may be, for example, an RRC connection setup message, an RRC connection reconfiguration message, or the like.

[0061] Each aspect / embodiment described in the present disclosure may be applied to at least one of systems using LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), 6th generation mobile communication system (6G), xth generation mobile communication system (xG) (x is, for example, an integer or a decimal), FRA (Future Radio Access), NR (new Radio), New radio access (NX), Future generation radio access (FX), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark), and other suitable systems, as well as next-generation systems extended, modified, created, and defined based on these. Further, multiple systems may be combined (for example, a combination of at least one of LTE and LTE-A and 5G, etc.) and applied.

[0062] The processing procedures, sequences, flowcharts, etc. of each aspect / embodiment described in the present disclosure may be reordered as long as there is no contradiction. For example, for the methods described in the present disclosure, the elements of various steps are presented using an exemplary order and are not limited to the specific order presented.

[0063] The input / output information, etc. may be stored in a specific location (e.g., memory) or may be managed using a management table. The information, etc. to be input / output may be overwritten, updated, or appended. The output information, etc. may be deleted. The input information, etc. may be transmitted to other devices.

[0064] The determination may be made based on a value represented by 1 bit (0 or 1), may be made based on a boolean value (Boolean: true or false), or may be made by comparing numerical values (e.g., comparison with a predetermined value).

[0065] Each aspect / embodiment described in the present disclosure may be used alone, may be used in combination, or may be switched and used during execution. Also, the notification of predetermined information (e.g., notification of "being X") is not limited to being explicitly performed, and may be performed implicitly (e.g., by not performing the notification of the predetermined information).

[0066] As described above in detail about the present disclosure, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure can be implemented as modifications and variations without departing from the spirit and scope of the present disclosure defined by the claims. Therefore, the description of the present disclosure is for the purpose of illustration and has no restrictive meaning for the present disclosure.

[0067] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., whether called software, firmware, middleware, microcode, a hardware description language, or by any other name.

[0068] Also, software, instructions, information, etc. may be transmitted and received via a transmission medium. For example, when software is transmitted from a website, server, or other remote source using at least one of wired technologies (such as coaxial cables, optical fiber cables, twisted pairs, digital subscriber lines (DSL), etc.) and wireless technologies (such as infrared rays, microwaves, etc.), at least one of these wired technologies and wireless technologies is included within the definition of the transmission medium.

[0069] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc., which may be referred to throughout the above description, may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0070] In addition, for the terms described in this disclosure and the terms necessary for understanding this disclosure, they may be replaced with terms having the same or similar meanings. For example, at least one of a channel and a symbol may be a signal (signaling). Also, a signal may be a message. Also, a component carrier (CC) may be referred to as a carrier frequency, a cell, a frequency carrier, etc.

[0071] The terms "system" and "network" used in this disclosure are used interchangeably.

[0072] Also, the information, parameters, etc. described in this disclosure may be represented using absolute values, relative values from a predetermined value, or in terms of corresponding other information. For example, a radio resource may be indicated by an index.

[0073] The names used for the above-mentioned parameters are not limiting in any way. Furthermore, the mathematical formulas and the like using these parameters may be different from those explicitly disclosed in the present disclosure. Since various channels (e.g., PUCCH, PDCCH, etc.) and information elements can be identified by any suitable names, the various names assigned to these various channels and information elements are not limiting in any way.

[0074] The terms "determining" and "deciding" used in the present disclosure may include a wide variety of operations. "Determining" and "deciding" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up (searching, inquiring) (e.g., searching in a table, database, or another data structure), and considering something as having "determined" or "decided" what has been ascertained. Also, "determining" and "deciding" may include considering something as having "determined" or "decided" what has been received (e.g., receiving information), transmitted (e.g., transmitting information), input, output, accessed (e.g., accessing data in a memory). Also, "determining" and "deciding" may include considering something as having "determined" or "decided" what has been resolved, selected, chosen, established, compared, etc. That is, "determining" and "deciding" may include considering something as having "determined" or "decided" some action. Also, "determining (deciding)" may be read as "assuming", "expecting", "considering", etc.

[0075] As used in this disclosure, the term "based on" does not mean "based only on" unless otherwise specified. In other words, the term "based on" means both "based only on" and "based at least on".

[0076] Any reference to an element using designations such as "first", "second", etc. used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Thus, a reference to a first and a second element does not mean that only two elements can be employed, or that the first element must precede the second element in any form.

[0077] In this disclosure, when terms such as "include", "including" and their variants are used, these terms are intended to be inclusive, similar to the term "comprising". Further, the term "or" used in this disclosure is not intended to be an exclusive disjunction.

[0078] In this disclosure, for example, when articles are added by translation, such as a, an and the in English, this disclosure may include that the noun following these articles is in the plural form.

[0079] In this disclosure, the term "A and B are different" may mean that "A and B are different from each other". Note that the term may also mean that "A and B are each different from C". Terms such as "separate", "coupled", etc. may also be interpreted in the same way as "different".

Description of Reference Numerals

[0080] 10... Information processing apparatus, 11... Acquisition unit, 12... Determination unit, 13... Calculation unit, 14... Output unit, 1001... Processor, 1002... Memory, 1003... Storage, 1004... Communication device, 1005... Input device, 1006... Output device, 1007... Bus.

Claims

1. a determination unit that determines, at intervals, whether or not the subject is facing a specific direction that is determined to be a concentration state based on video data of the subject's face; A calculation unit that calculates a concentration level of the subject based on the multiple determination results by the determination unit; An information processing device comprising:

2. The determination unit performs the determination multiple times at regular intervals. The information processing device according to claim 1 .

3. the determination unit recognizes at least one of a facial direction of the subject and a behavior of the subject based on the video data, and determines whether the subject is facing the specific direction based on a recognition result; The information processing device according to claim 1 .

4. When the determination unit recognizes, as the behavior of the subject, a behavior of turning a face or a line of sight away from the specific direction, it determines that the subject is not facing the specific direction. The information processing device according to claim 3 .

5. the calculation unit sets the concentration level of the subject to a minimum value in a range of possible values ​​when the determination unit recognizes sleep as the behavior of the subject. The information processing device according to claim 3 .

6. the determination unit determines whether the subject is wearing a mask based on the video data, and if the subject is not wearing the mask, recognizes at least one of a talking behavior and a yawning behavior of the subject based on the video data; The calculation unit calculates a concentration level of the subject based on a recognition result of the movement in addition to the multiple determination results. The information processing device according to claim 1 .

7. The calculation unit calculates a value not exceeding a preset upper limit value for each of a case where the conversation action and a case where the yawning action are recognized by the determination unit as the concentration degree of the subject. The information processing device according to claim 6.

Images