Information provision device, information provision method, and information provision program

The information providing device addresses the challenge of accurately estimating worker proficiency by using work area and cognitive proficiency estimation units to tailor information presentation, resulting in improved collaboration and work efficiency.

WO2024241406A1PCT designated stage expired Publication Date: 2024-11-28NT T INC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2023/018846
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-19
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Conventional methods for estimating a worker's proficiency in intellectual tasks, such as those involving VR or AR, face challenges in accurately distinguishing relevant from irrelevant user behavior, leading to reduced estimation accuracy.

Method used

An information providing device that includes a work area estimation unit and a cognitive proficiency estimating unit, which analyze user movement and behavior to accurately assess proficiency levels and adapt the presentation of information accordingly, ensuring that information is presented in a format suitable for the worker's skill level.

Benefits of technology

The device effectively estimates worker proficiency with high accuracy, improving collaboration by tailoring information presentation to the individual's skill level, thereby enhancing work efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023018846_28112024_PF_FP_ABST
    Figure JP2023018846_28112024_PF_FP_ABST
Patent Text Reader

Abstract

The information provision device (10) estimates a region where work is to be performed on the basis of information indicating a motion of a user who conducts work. The information provision device (10) estimates the degree of proficiency of the user for the work on the basis of the information indicating the motion of the user and the region estimated by the work region estimation unit. The information provision device (10) presents, to the user, information relating to the work in a form corresponding to the proficiency estimated by a recognition proficiency estimation unit.
Need to check novelty before this filing date? Find Prior Art

Description

Information providing device, information providing method, and information providing program

[0001] The present invention relates to an information providing device, an information providing method, and an information providing program.

[0002] In recent years, advances in technologies such as virtual reality (VR), augmented reality (AR), and remote conferencing have made it possible for workers to collaborate with others who are not present or with AI to perform intellectual tasks (such as playing shogi or programming) (see, for example, non-patent document 1).

[0003] In intellectual work such as the one mentioned above, each worker may have a different level of skill in the work, so it is thought that work efficiency can be improved by exchanging information in a format that is suitable for each worker.

[0004] In this case, it is necessary to estimate the skill level of the worker. For example, Non-Patent Document 2 proposes a method for estimating the skill level of a worker based on the movement of the worker's line of sight.

[0005] Machino et al., “Real-space view-sharing collaboration system using mobile robots equipped with cameras and projectors,” Journal of the Robotics Society of Japan, Vol. 28, No. 6, pp. 746-755, 2010; Saijo et al., “Efforts to address proficiency and conversation satisfaction toward the realization of emotional communication technology,” NTT Technical Journal, Vol. 34, No. 1, pp. 32-35, (2022)

[0006] However, the conventional techniques have a problem in that it is sometimes difficult to accurately estimate the skill level of a worker.

[0007] Non-Patent Document 2 describes estimating the proficiency level based on information about the user's behavior in a work situation (e.g., gaze trajectory). However, the information about the user's behavior may contain information that has little relevance to the work. Such information becomes noise in the estimation of the proficiency level, reducing the estimation accuracy.

[0008] In order to solve the above-mentioned problems and achieve the objectives, the information providing device is characterized by having a work area estimation unit that estimates the area in which the work will be performed based on information indicating the actions of a user performing the work, a cognitive proficiency estimation unit that estimates the user's level of proficiency for the work based on the information indicating the user's actions and the area estimated by the work area estimation unit, and a presentation unit that presents information about the work to the user in a format corresponding to the level of proficiency estimated by the cognitive proficiency estimation unit.

[0009] According to the present invention, the skill level of a worker can be estimated with high accuracy.

[0010] FIG. 1 is a diagram illustrating information presented by an information providing device. FIG. 2 is a diagram illustrating information presented by an information providing device. FIG. 3 is a diagram illustrating information presented by an information providing device. FIG. 4 is a diagram illustrating an example of a configuration of an information providing system according to a first embodiment. FIG. 5 is a diagram illustrating an example of a worker DB. FIG. 6 is a diagram illustrating an example of a work record DB. FIG. 7 is a diagram illustrating an example of a presented information DB. FIG. 8 is a sequence diagram illustrating the processing flow of the information providing system according to the first embodiment. FIG. 9 is a diagram illustrating a method of estimating a work area. FIG. 10 is a flowchart illustrating the processing flow of the information providing device according to the first embodiment. FIG. 11 is a flowchart illustrating the processing flow when a pre-trained model is used for cognitive proficiency estimation. FIG. 12 is a flowchart illustrating the processing flow of learning a model used for cognitive proficiency estimation. FIG. 13 is a diagram illustrating an example of a computer that executes an information provision program.

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An information providing apparatus, an information providing method, and an information providing program according to the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments.

[0012] First Embodiment FIGS. 1, 2, and 3 are diagrams illustrating information provided by an information providing device according to a first embodiment.

[0013] 1, a worker is a user who performs a task, and a collaborator is a user who performs a task together with the worker.

[0014] For example, a worker (operator) performs operations in a task, while a collaborating partner (partner) provides information to the worker via the information providing device 10. The collaborating partner provides instructions for the task or advice regarding the task to the worker.

[0015] For example, the task may be an input task to an information device using a mouse, a keyboard, a touch panel, or a joystick. The task may also be an operation of a controller of an XR (AR, VR, etc.) device, a hand gesture input, or the like.

[0016] Furthermore, the task may be playing a board game such as shogi. In this embodiment, the task is assumed to be playing shogi.

[0017] As shown in Fig. 1, a worker is playing shogi while looking at the board. Here, it is assumed that the worker is looking at the board directly or via a screen 221 displayed on a display device such as VR goggles or a display.

[0018] It is assumed that the worker and the collaborator use terminal devices, such as personal computers, smartphones, tablet terminals, etc. The terminal devices may be integrated with the information providing device.

[0019] The collaborating partner sends a message from a terminal device (terminal device 30 described below) via a network (NW). The network is, for example, the Internet. Note that this embodiment can be realized whether the worker and collaborator are in remote locations or in the same real space (the same work location).

[0020] The message sent by the collaborating partner is provided to the worker's terminal device (terminal device 20 described below) by the information providing device 10. In this case, it can be said that the information providing device 10 functions as an interface for information transmission between the worker and the collaborating partner.

[0021] The information providing device 10 changes the format in which information is presented depending on the worker's level of skill in the work. For example, when the worker has a low level of skill in the work, the information providing device 10 presents information in a format that is intuitively easy to understand.

[0022] Here, messages sent by collaborators include both fixed phrases and non-fixed phrases. Fixed phrases are sentences with a predetermined format. Non-fixed phrases are sentences that can be freely written.

[0023] In the example of Figure 1, the "content" of the message corresponds to the fixed phrase, and the "reason" of the message corresponds to the non-fixed phrase.

[0024] The collaborating partner sends a message saying, "Content: First player 2-6 Pawn. Reason: The Bishop becomes easier to move."

[0025] "White's 26 Pawn" is made up of three parts: "White" indicating whether to play White or Black, "26" indicating the position on the board, and "Pawn" indicating the type of piece. Since the character string candidates for each part are predetermined and the number of combinations is finite, the information providing device 10 can easily interpret the meaning of the fixed phrase.

[0026] The information providing device 10 displays visual information 222a on the screen 221. The information providing device 10 generates the visual information 222a based on the message.

[0027] 1, it is assumed that the worker is a beginner, i.e., the worker has a low level of skill in the work. For this reason, the information providing device 10 adds information to the visual information 222a to promote the worker's understanding.

[0028] As shown in Figure 1, the visual information 222a includes an image representing the "2-6 Pawn" portion. The visual information 222a also includes the following character string, which is an undefined phrase: "Reason: The 'Bishop' becomes easier to move." Furthermore, the visual information 222a includes the following character string as supplementary information regarding the "Pawn": "Supplementary information: The 'Pawn' can only move one step forward."

[0029] This allows a beginner player to understand the intent of the message even if they do not understand the position on the board that "26 Pawn" represents or the movement of the "Pawn" piece.

[0030] 2 shows an example of a screen 221 when the worker is a beginner to intermediate worker, i.e., when the worker's level of skill in the work is medium. However, in this embodiment, the level of skill increases in the order of beginner, beginner, intermediate, and expert. Therefore, the level of skill of a beginner is higher than that of a beginner.

[0031] As shown in FIG. 2, the visual information 222b includes a character string that directly expresses the message.

[0032] This allows beginner to intermediate level workers to know exactly what is in the message sent by the co-worker by looking at the visual information 222b.

[0033] FIG. 3 shows an example of a screen 221 when the worker is an expert, that is, when the worker has a high level of proficiency in the work.

[0034] 3, the visual information 222c highlights one square on the board. The visual information 222c highlights a part of the screen by using brightness, color, or pattern.

[0035] This allows a skilled worker to understand the intent of the message sent by the collaborating partner by looking at the visual information 222c without seeing the message itself.

[0036] The information providing device 10 automatically estimates the skill level of the worker for the work and presents a message in a format according to the estimated skill level, so that the co-worker can send a message without being aware of the skill level of the worker.

[0037] [Configuration of First Embodiment] Fig. 4 is a diagram showing an example of the configuration of an information providing system according to the first embodiment. As shown in Fig. 4, the information providing system 1 includes an information providing device 10, a terminal device 20, and a terminal device 30. The information providing device 10, the terminal device 20, and the terminal device 30 may be separate terminals or may be the same terminal.

[0038] The terminal device 30 accepts a message input by a collaborating partner and transmits the input message to the information providing device 10.

[0039] The information providing device 10 estimates the skill level of the worker. Furthermore, the information providing device 10 transmits information based on the message received from the terminal device 30 to the terminal device 20 in a format according to the estimated skill level.

[0040] The terminal device 20 outputs the information received from the information providing device 10 to the worker.

[0041] In this example, the task is assumed to be a game of shogi. The worker is a shogi player. The co-worker gives the worker advice on shogi. The skill level estimated by the information providing device 10 is called the perceived skill level.

[0042] As shown in FIG. 4 , the information providing device 10 includes a communication unit 11 , a storage unit 12 , and a control unit 13 .

[0043] The communication unit 11 performs data communication with other devices via a network. For example, the communication unit 11 is a network interface card (NIC). When the information providing device 10, the terminal device 20, and the terminal device 30 are the same terminal, the communication unit 11 transfers data within the terminal. Alternatively, each component may be directly connected without going through the communication unit 11.

[0044] The storage unit 12 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), an optical disk, etc. Note that the storage unit 12 may also be a data-rewritable semiconductor memory such as a random access memory (RAM), a flash memory, or a non-volatile static random access memory (NVSRAM).

[0045] The storage unit 12 stores an operating system (OS) and various programs executed by the information providing device 10. The storage unit 12 stores a worker DB 121, a work record DB 122, and a presented information DB 123.

[0046] The worker DB 121 stores information about workers. Fig. 5 is a diagram showing an example of the worker DB. As shown in Fig. 5, the worker DB 121 includes a worker ID, work experience, and a previous estimated cognitive proficiency level.

[0047] The worker ID is an identifier that indicates each row in the worker DB 121. Each row in the worker DB 121 corresponds to a worker.

[0048] The work experience indicates the amount of work experience of the worker corresponding to the worker ID. Here, the work experience is the time (unit: years) that each worker has performed the work. Note that the work experience is not limited to the time performed, but may also include qualifications, grades, etc. that objectively express the quantity and quality of experience, such as grades and ranks.

[0049] The previous estimated cognitive proficiency value is the average value of the cognitive proficiency estimated by the information providing device 10 when the worker performed the task one time before. A worker may perform the same task multiple times. Furthermore, the cognitive proficiency may be estimated multiple times for one task.

[0050] Note that α is a symbol that represents cognitive proficiency. A symbol with a bar directly above α represents the average value of cognitive proficiency. α has an initial value of 0 and varies within the range of 0 to 1.0. For example, the closer α is to 0, the closer a worker is to a beginner (lower level of proficiency), and the closer α is to 1.0, the closer a worker is to an expert (higher level of proficiency).

[0051] For example, FIG. 5 shows that a worker with a worker ID of "1" has one year of work experience and the previous estimated cognitive proficiency value is zero.

[0052] Also, for example, FIG. 5 shows that a worker with a worker ID of "2" has 10 years of work experience and the previous estimated cognitive proficiency value is 0.9.

[0053] The work record DB 122 is a record of work performed by a worker. Fig. 6 is a diagram showing an example of the work record DB. As shown in Fig. 6, the work record DB 122 includes a work ID, a worker ID, a scene, a gaze path, an operation record, an operation time, and an evaluation value.

[0054] The task ID is an identifier that identifies each row in the task record DB 122. Each row in the task record DB 122 corresponds to a task. The worker ID is an identifier that identifies a worker.

[0055] 6 shows only one row as an example, records of multiple tasks are stored in the work record DB 122. Furthermore, the workers whose tasks are stored in the work record DB 122 are not limited to the workers whose cognitive proficiency levels are to be estimated.

[0056] The work record DB 122 may be used as training data for learning a model that calculates cognitive proficiency.

[0057] The scene is information indicating the situation in which the worker is performing the work. In this example, the scene is an image of a shogi board, which is the object of the work.

[0058] The scan path is a trace of the movement of the worker's gaze from an arbitrary time for t' seconds when looking at a scene. In the example of Fig. 6, the scan path is shown in the form of an image, but the scan path may also be shown in the form of time-series data indicating the gaze point coordinates and eye direction collected over t' seconds.

[0059] The operation record is the operation performed by the worker on the scene shown in the scene. The operation time is the time taken to perform the operation shown in the operation record.

[0060] The evaluation value is an evaluation value for the operation performed by the worker, and is calculated based on, for example, the scan path.

[0061] For example, the higher the winning rate in shogi, the higher the evaluation value. The evaluation value may be calculated by an external evaluation tool.

[0062] Note that z' is a symbol representing an evaluation value. z' takes a value ranging from 0 to 1.0. Also, s' is a symbol representing a scene. s' is, for example, an image.

[0063] 6 shows that the task with task ID "1" was performed by a worker with worker ID "3", the task record was "White 2-6 Pawn", and the operation time was 128 seconds. Also, FIG. 6 shows that the evaluation value of the task with task ID "1" is "0.6".

[0064] The presented information DB 123 stores information related to the format of information (presentation format) presented to the worker. Fig. 7 is a diagram showing an example of the presented information DB. As shown in Fig. 7, the presented information DB 123 includes information ID, response recognition proficiency level, message information, presentation format, and presentation position.

[0065] The information providing device 10 generates a presentation format based on a message from a collaborating partner and stores the generated presentation format in the presentation information DB 123. Alternatively, the presentation format may be manually created in advance and stored in the presentation information DB 123.

[0066] The information ID is an identifier that identifies each row of the presentation information DB 123. The corresponding recognition proficiency level is a range of values ​​of the recognition proficiency level that corresponds to each presentation format.

[0067] The message information is a portion of a message sent by a collaborating partner that corresponds to a fixed phrase, and includes, for example, information about work instructions.

[0068] The presentation format is a format generated according to the message information and the recognition proficiency level. The presentation position indicates the position where the information is presented. Here, the presentation position corresponds to a position on a shogi board. The presentation position may be calculated at any time, or may be calculated manually in advance and stored in the presentation information DB 123.

[0069] 7 shows an example of the presentation information DB 123 when the message from the collaborating partner is "Content: First move 2-6 Pawn, Reason: Bishop becomes easier to move" as shown in FIG. 1 etc. The method of generating the presentation format will be described later in detail.

[0070] The control unit 13 controls the entire information providing device 10. The control unit 13 is, for example, an electronic circuit such as a central processing unit (CPU), a micro processing unit (MPU), or a graphics processing unit (GPU), or an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0071] The control unit 13 also has an internal memory for storing programs defining various processing procedures and control data, and executes each process using the internal memory. The control unit 13 also functions as various processing units when various programs are run. For example, the control unit 13 has a cognitive proficiency estimation unit 131, a working area estimation unit 132, and a presentation information generation unit 133.

[0072] The cognitive proficiency estimation unit 131 estimates the user's proficiency for a task based on information indicating the user's motion and the area in which the task is performed. In the following description, the information indicating the user's motion and the area in which the task is performed are referred to as motion information and task area, respectively.

[0073] A user who performs a task is a worker. The task area is, for example, all or part of the scene in the task record DB 122 shown in Fig. 6. The action information is, for example, the scan path in the task record DB 122 shown in Fig. 6.

[0074] For example, the cognitive proficiency estimation unit 131 estimates the user's cognitive proficiency for the task by using the trajectory of the user's gaze during the task as action information.

[0075] In addition, the cognitive proficiency estimation unit 131 can estimate the user's cognitive proficiency for a task by using at least one of the user's actions, the history of operations performed by the user during the task, and the user's biometric information as action information.

[0076] For example, the user's actions are the scan path in the work record DB 122. The cognitive proficiency estimation unit 131 can estimate the proficiency level by using the scan path in the work area, among the scan paths corresponding to the scene, as the user's actions. Also, for example, the operation history is the work record and operation time in the work record DB 122. Also, for example, the user's biological information is the worker's body temperature, heart rate, pupils, and other vital signs.

[0077] The work area estimation unit 132 estimates the work area based on the motion information of the user performing the work. For example, the work area estimation unit 132 estimates the work area based on the trajectory of the user's gaze. However, the area gazed at by the user and the work area may or may not match.

[0078] For example, when a user plays shogi, the user moves one piece on each shogi board. In this case, the user does not work in the entire area of ​​the shogi board. The area where the user actually works on each board is thought to be mainly the position of the piece before and after it is moved, and the area around the piece stand (the place where the piece is placed).

[0079] The work area estimated by the work area estimation unit 132 is passed to the cognitive proficiency estimation unit 131. The cognitive proficiency estimation unit 131 estimates the user's proficiency by regarding the work area as a scene, thereby enabling the proficiency to be estimated with high accuracy.

[0080] The presentation information generation unit 133 presents information about the task to the user via the output unit 22 in a format according to the cognitive proficiency level estimated by the cognitive proficiency estimation unit 131 .

[0081] For example, the presentation information generating unit 133 presents a message to the worker using one of the presentation formats shown in FIG.

[0082] The following describes the configuration of the terminal device 20. As shown in FIG.

[0083] The input unit 21 receives input of data and includes an operation unit 211, a gaze measurement unit 212, and a biological information measurement unit 213.

[0084] The operation unit 211 is a device for an operator to operate, and is, for example, a mouse, a keyboard, a touch panel, a joystick, a controller for an XR device, a sensor for recognizing hand gestures, or the like.

[0085] The gaze measurement unit 212 is a device for measuring the gaze of a worker. For example, the gaze measurement unit 212 is an eye tracker.

[0086] The biological information measuring unit 213 is a device for measuring biological information of the worker. For example, the biological information measuring unit 213 is a wearable device worn by the worker.

[0087] The output unit 22 is a device for outputting information provided by the information providing device 10. For example, the output unit 22 is a display that displays a screen.

[0088] The processing flow of each unit including the cognitive proficiency estimation unit 131 and the presentation information generation unit 133 will be described with reference to Fig. 8. Fig. 8 is a sequence diagram showing the processing flow of the information provision system according to the first embodiment.

[0089] As shown in FIG. 8, first, the input unit 31 of the terminal device 30 accepts a message input from a collaborating partner (step S101).

[0090] The message input to the input unit 31 is transmitted to the presentation information generation unit 133 via the communication unit 11 of the information providing device 10 (step S102).

[0091] Here, the presentation information generation unit 133 requests the cognitive proficiency estimation unit 131 to estimate the cognitive proficiency (step S103).

[0092] The cognitive proficiency estimation unit 131 requests the scene, the worker's operation history, gaze information, and biometric information from the work record DB 122 of the information providing device 10 or the input unit 21 of the terminal device 20 (step S104), and acquires the provided information (step S105).

[0093] The cognitive proficiency estimation unit 131 can acquire operation records as operation history from the work record DB 122. Furthermore, the cognitive proficiency estimation unit 131 can acquire scan paths as gaze information from the work record DB 122. However, the cognitive proficiency estimation unit 131 may also acquire the operation history and gaze information directly from the input unit 21. Furthermore, the cognitive proficiency estimation unit 131 can acquire biometric information from the input unit 21.

[0094] Then, the cognitive proficiency estimation unit 131 analyzes the operation information, gaze information, and biological information (operation analysis, gaze analysis, biological analysis) (steps S106, S107, S108).

[0095] For example, the cognitive proficiency estimation unit 131 converts information acquired through operation analysis, gaze analysis, and biological analysis into a format that can be used in cognitive proficiency estimation.

[0096] Furthermore, the cognitive proficiency estimation unit 131 requests worker information from the worker DB 121 (step S109) and acquires the worker information (step S110).

[0097] Here, the working area estimation unit 132 acquires the scene and the gaze information from the cognitive proficiency estimation unit 131 (step S111). The working area estimation unit 132 may acquire the gaze information converted by the cognitive proficiency estimation unit 131 in step S107.

[0098] The work area estimation unit 132 estimates the work area based on the scene and line of sight information (step S112). The work area estimation unit 132 provides the estimated work area to the cognitive proficiency estimation unit 131 (step S113).

[0099] Next, the cognitive proficiency estimation unit 131 estimates the cognitive proficiency of the worker using the information obtained up to this point (step S114). At this time, the cognitive proficiency estimation unit 131 estimates the cognitive proficiency based on the work area, not the entire scene.

[0100] Thereafter, the presentation information generation unit 133 acquires the cognitive proficiency estimated by the cognitive proficiency estimation unit 131 (step S115), and acquires presentation content and a presentation format corresponding to the message and the cognitive proficiency from the presentation information DB 123 (steps S116 and S117). The presentation content is, for example, message information from the presentation information DB 123.

[0101] The presentation information generator 133 generates presentation information based on the acquired presentation content and presentation format (step S118). Then, the presentation information generator 133 causes the output unit 22 of the terminal device 20 to output the generated presentation information (steps S119 and S120).

[0102] (Example) An example in which a user solves a shogi puzzle in a VR space will be specifically described. First, the work area estimation unit 132 calculates the line of sight trajectory x i (t), the work area estimation unit 132 extracts a part with high importance from the extracted scan path x i Based on the part (t), the part of the scene s' that is of high importance, i.e., the working area, is extracted. Note that the line of sight trajectory obtained in real time is expressed as x i (t), and the scan path stored in the work record DB 122 is denoted by x i It is written as ´(t).

[0103] FIG. 9 is a diagram illustrating a method for estimating a work area. A scan path 500 includes gaze data for the entire scene. An image 510 is an image of the entire shogi board and pieces. Note that the scene may be represented not as two-dimensional information such as an image, but as three-dimensional information such as a spatial model. Similarly, the work area may be represented as a two-dimensional figure or a three-dimensional figure. Image 510 is an example of a work-related area. The work-related area may be rephrased as information about the entire work scene.

[0104] Furthermore, if the scene is a moving image, the working area may be a time width that is part of the playback time of the moving image.

[0105] The scan path 500 is the length of time the user looks at each square when the image 510 is divided into squares. The scan path 500 in Figure 9 shows the length of time (e.g., the number of frames) the user looks at each square.

[0106] The work area estimation unit 132 estimates an area in the image 510 that is surrounded by a figure whose position and shape are determined based on the line of sight 500 as an area where work will be performed.

[0107] First, a rectangle including squares with significant coefficients (non-zero coefficients in this case) and some squares with zero coefficients is determined as a figure in the scan path 500. As shown in Fig. 9, the working area estimation unit 132 extracts figures 501 and 502 from the scan path 500. Both figures 501 and 502 are rectangles.

[0108] Some of the squares with zero coefficients are squares that are known to be important based on prior knowledge. For example, the squares included in the two rows on the bottom side (bottom of the drawing) of figure 501 and the row on the top side of figure 502 in Fig. 9 have zero coefficients, but in this case, there is prior knowledge that the ranges in question are the ranges on the shogi board and the piece table, and so these squares are known to be important based on prior knowledge.

[0109] Then, the work area estimation unit 132 estimates the area surrounded by the graphic 501 and the graphic 502 as a work area 505 .

[0110] Furthermore, the working area estimation unit 132 may determine the shape based on the cells whose values ​​are included in a certain number of the top values ​​or the cells whose values ​​are equal to or greater than a threshold value, rather than on all the cells whose values ​​are significant coefficients.

[0111] Furthermore, the value of each square in Figure 9 is not limited to the length of time each square was looked at, i.e., the gaze time, but may also be the number of times each square was looked at, i.e., the gaze count. The work area estimation unit 132 estimates the work area based on at least one of the gaze count or gaze time, which are information indicating the user's actions. A location where the worker gazes frequently is likely to be an important location, just like a location where the gaze time is long.

[0112] The cognitive proficiency estimation unit 131 iand the line of sight x i The part of (t) corresponding to the working area 505 is input to a model (estimator) to calculate an evaluation value. The model is, for example, a deep neural network. Note that the image 515 in FIG. 9 is a scene s i 5. The image is of the portion corresponding to the work area 505.

[0113] The model has been trained using the work record DB 122 as training data. That is, the model is trained by using all or part of the areas of the scene (the shogi board) s′ of all the workers stored in the work record DB 122 and the scan paths x i The training data is used, with '(t) as the input (explanatory variable) and the evaluation value z' of the operation (each move) performed by each operator as the output (objective variable). For example, the evaluation value in learning increases as the game record (win rate), which is the result of the operation, increases.

[0114] In this way, the cognitive proficiency estimation unit 131 estimates the cognitive proficiency of the worker based on the evaluation value obtained by inputting all or part of a scene area and the gaze trajectory corresponding to the area into the trained model.

[0115] The cognitive proficiency estimation unit 131 calculates the cognitive proficiency α by using the formula (1). i Calculate.

[0116]

[0117] i is a subscript that indicates the number of moves from the start of the shogi game. i is the working area of ​​the board (including the pieces) at the i-th move from the start of the shogi game. i (t) is s i The trajectory of the line of sight movement for t' seconds starting from an arbitrary time t in i (t) may be information representing the position or direction of the line of sight at each time, or information representing the amount of movement of the line of sight trajectory. i (t) may be information representing the location on the board where the gaze has stayed for a certain period of time or more. i (t) may be information that represents the trajectory of the line of sight as a pattern. iis the working space, the trajectory x i In (t), trajectories at positions outside the working area are not used in the calculation.

[0118] z i (s i , x i (t)) is s i and x i This is the evaluation value output from a trained model that uses (t) as input.

[0119] The symbol with a bar just above α indicates the cognitive proficiency level α of a worker who has used the system multiple times. i is the average value of

[0120] b i (t) is s i The stress value is estimated from biological information (for example, skin potential and heart rate) for t' seconds starting from an arbitrary time t in the stress state. The stress value may be calculated using an existing calculation method.

[0121] τn is the time it takes the player to make the nth move. Ti(τ) is the average time it takes the player to make the ith move.

[0122] f(m) is a function of the standard normal distribution with work experience m as a variable. 1 , w 2 , w 3 , w 4 is 0≦α i is a weighting coefficient to make the value ≦1. a is a constant.

[0123] Here, the right-hand side of equation (1) is the product of a row vector and a column vector, and each element of the row vector functions as a weight for each element of the column vector.

[0124] First, the cognitive proficiency estimation unit 131 calculates a first value (element z of the column vector) based on the action at time t. i (s i , x i (t)), the second value based on the history at time t (element 1 / T i (τ)), a third value based on the biological information at a predetermined time (element 1 / b i (t)) is calculated.

[0125] In addition, the cognitive proficiency estimation unit 131 acquires a fourth value (column vector α (bar directly above)) which is the user's cognitive proficiency for the task that has been estimated at a time before time t.

[0126] In addition, the cognitive proficiency estimation unit 131 can use the reciprocal of the measurement value and the normalized measurement value to estimate the cognitive proficiency level, rather than using the measurement value obtained from the terminal device 20 etc. as is.

[0127] Element w of row vector 1 m -a The smaller the work experience m, the larger the value. 2 The smaller the work experience m, the smaller lnm becomes.

[0128] Therefore, in the product of a row vector and a column vector, the less experience a user has with a task, the greater the influence of the first value and the smaller the influence of the fourth value.

[0129] Element w of row vector 3 f(m) becomes larger as the work experience m approaches the mean value of the standard normal distribution. 4 The closer the work experience m is to the mean value of the standard normal distribution, the larger f(m) becomes. Note that the work experience m may be standardized according to the standard normal distribution.

[0130] Therefore, in the product of the row vector and the column vector, the closer the amount of experience the user has with the task is to the predetermined average value, the greater the influence of the second and third values.

[0131] In this way, the cognitive proficiency estimation unit 131 controls the influence of multiplying the row vector by the column vector, and then estimates the user's cognitive proficiency for the task based on the first value, the second value, the third value, and the fourth value.

[0132] The cognitive proficiency estimation unit 131 may estimate the cognitive proficiency using one or two of the first value, the second value, and the third value excluding the fourth value among the elements of the column vector. For example, the cognitive proficiency estimation unit 131 may estimate the cognitive proficiency from only the first value, or may estimate the cognitive proficiency using a column vector having only the first value and the second value as elements.

[0133] The cognitive proficiency estimation unit 131 may estimate the cognitive proficiency only from information acquired during the work.

[0134] For example, the cognitive proficiency estimation unit 131 calculates the cognitive proficiency α based on the amount of gaze movement l(t) per unit time as shown in equation (2). i It should be noted that in formula (2), the average value of the past recognition proficiency is not used, but the biometric information and operation history acquired during the work are used.

[0135]

[0136] The meaning of each symbol in formula (2) is the same as that in formula (1). 5 , w 6 , w 7 is 0≦α i It is a weighting factor to make it ≦1.

[0137] As shown in Figure 7, if the cognitive proficiency estimated by the cognitive proficiency estimation unit 131 is less than a first threshold, the presentation information generation unit 133 displays a character string and an image indicating information about the task in a display area of ​​the device used by the user; if the cognitive proficiency is equal to or greater than the first threshold and less than a second threshold that is greater than the first threshold, the presentation information generation unit 133 displays a character string indicating the information in the display area; and if the cognitive proficiency is equal to or greater than the second threshold, the presentation information generation unit 133 highlights a portion of the display area according to the information. The display area is, for example, the screen 221. Note that the parentheses in the presentation format when the information ID is "3" in Figure 7 are used to indicate which portion is to be highlighted and are not displayed on the screen.

[0138] Here, the first threshold value is, for example, 0.3. The presentation information generation unit 133 refers to the presentation information DB 123, and when the estimated value of the cognitive proficiency level is equal to or greater than 0 and less than 0.3, the presentation information generation unit 133 displays visual information 222 a including an image and a character string as shown in FIG. 1 on the screen 221 output by the output unit 22.

[0139] The second threshold value is, for example, 0.6. When the estimated value of the cognitive proficiency level is equal to or greater than 0.3 and less than 0.6, the presentation information generation unit 133 refers to the presentation information DB 123 and displays visual information 222b including a character string as shown in FIG. 2 on the screen 221 output by the output unit 22.

[0140] In addition, the presentation information generation unit 133 refers to the presentation information DB 123, and if the estimated value of the cognitive proficiency level is 0.6 or higher, displays visual information 222c, which is only highlighted as shown in Figure 3, on the screen 221 output by the output unit 22.

[0141] The presentation information generation unit 133 may generate each piece of visual information as an image and transmit the generated image to the terminal device 20. The terminal device 20 displays the received image on the screen 221 in a superimposed manner.

[0142] In this embodiment, the display modes of visual information are classified into three levels using two thresholds, a first threshold and a second threshold. However, the number of thresholds is not limited to two and may be any positive integer depending on the target task and work. In this case, the display modes of visual information are classified into levels equal to the number of thresholds plus one.

[0143] The terminal device 20 may also be an all-in-one type of AR glasses. In this case, the worker can view the actual shogi board through the display area of ​​the AR glasses. The screen 221 in this embodiment is replaced with the display area of ​​the AR glasses. The presentation information generation unit 133 then uses the AR function to project visual information onto the actual shogi board visible from the display area, superimposing it on the actual shogi board.

[0144] The output unit 22 may be a projector. In this case, the input unit 21 may be a device such as a gaze detection device, a smart watch, a mouse, or a smartphone.

[0145] For example, the output unit 22 projects a screen 221 including a shogi board, shogi pieces, and visual information onto a desk and a screen, allowing the worker to view the projected image without wearing equipment such as AR glasses.

[0146] The terminal device 20 may also be a combination of an input device such as a gaze measurement device, a smart watch, a mouse, or a smartphone, and a display device such as a projector or a display.

[0147] The terminal device 20 may also be a display device such as a projector and a display that includes an input device.

[0148] [Processing of First Embodiment] The flow of processing by the information providing device 10 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the flow of processing by the information providing device according to the first embodiment.

[0149] First, the information providing device 10 acquires the worker's movements, operation history, and biometric information (step S201). The information providing device 10 can acquire each piece of information from the terminal device 20 or the storage unit 12.

[0150] Next, the information providing device 10 estimates and extracts a work area from the worker's movements (step S202). For example, the information providing device 10 extracts the work area 505 in FIG.

[0151] Next, the information providing device 10 extracts a scene related to the extracted worker's action (step S203). For example, the information providing device 10 extracts an image 515 corresponding to the work area 505 from the image 510 in FIG.

[0152] Then, the information providing device 10 estimates the cognitive proficiency level using the extracted actions and situations of the worker (step S204). For example, the information providing device 10 calculates the cognitive proficiency level using equation (1).

[0153] If the calculated cognitive proficiency level is less than the first threshold (Yes in step S205), the information providing device 10 presents information about the task using an image and text (step S206). For example, the information providing device 10 presents the information using the visual information 222a in FIG. 1 .

[0154] If the calculated cognitive proficiency level is not less than the first threshold value (No at step S205), the information providing device 10 proceeds to step S207.

[0155] If the calculated cognitive proficiency level is not less than the second threshold (step S207, No), the information providing device 10 presents information about the task by highlighting it in the display area (step S209). For example, the information providing device 10 presents the information using visual information 222c in Fig. 3. The display area here may be the screen of a display or the display area of ​​AR goggles.

[0156] If the calculated cognitive proficiency level is less than the second threshold (step S207, Yes), the information providing device 10 presents information about the task using text (step S208). For example, the information providing device 10 presents the information using visual information 222b in Fig. 2. Note that the second threshold is greater than the first threshold.

[0157] When a pre-trained model is used for cognitive proficiency estimation, the model accepts input corresponding to a work area rather than the entire scene. For example, the model accepts input of data (e.g., image 515) of a size corresponding to work area 505 rather than the size of image 510 in FIG. 9 . The model is also assumed to have been trained using data of the same size as the input data. Note that the data of a size corresponding to work area 505 is not limited to an image. The data of a size corresponding to work area 505 may be video data, audio data, or a combination of multiple data. In addition, data of work area 505 may be provided as the extracted worker's movements in addition to the above data.

[0158] FIG. 11 is a flowchart showing the flow of processing when a pre-trained model is used for estimating cognitive proficiency.

[0159] First, the information providing device 10 acquires the worker's movements, operation history, and biometric information (step S301). Next, the information providing device 10 extracts the work area and scene in the same position and shape as when the cognitive proficiency estimation model was trained (step S302). The processing from step S303 onward is the same as the processing from step S204 onward in FIG. 10.

[0160] A method for learning the model in this case will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the flow of a process for learning a model used for estimating a cognitive proficiency level.

[0161] First, the information providing device 10 acquires the worker's movements, operation history, and biometric information (step S401).

[0162] Next, the information providing device 10 estimates and extracts a work area from the worker's actions (step S402). The information providing device 10 also extracts scenes related to the extracted worker's actions (step S403). In this case, if work data is used multiple times or by multiple people, the number of dimensions of the data must be unified to match the work area. It is assumed that the learning of an estimation model for work area estimation, or parameters such as thresholds for estimation, and calculation formulas are prepared in advance.

[0163] Here, the information providing device 10 uses the worker's actions extracted in step S402 and the related scenes extracted in step S403 as input data, and the cognitive proficiency level (correct answer label) as training data to train a model through supervised learning (step S404). Note that the information providing device 10 may train the model using a method other than supervised learning. For example, the information providing device 10 trains the model through unsupervised learning. In this case, the model can statistically analyze the collected data and extract areas with a high level of user interaction.

[0164] Effect of First Embodiment As described above, the information providing device 10 has a cognitive proficiency estimation unit 131, a work area estimation unit 132, and a presentation information generation unit 133. The work area estimation unit 132 estimates an area where the work will be performed based on information indicating the movements of the user performing the work. The cognitive proficiency estimation unit 131 estimates the user's proficiency with respect to the work based on the information indicating the user's movements and the area estimated by the work area estimation unit 132. The presentation information generation unit 133 presents information about the work to the user in a format according to the proficiency estimated by the cognitive proficiency estimation unit 131.

[0165] The work area estimation unit 132 can extract information of high importance (e.g., information about a part of a scene corresponding to a work area) from information about the work scene (e.g., information about the entire work scene). As a result, by learning a cognitive proficiency estimation model and performing estimation using the model, limited to important actions and areas, it becomes possible to accurately estimate the proficiency of a worker.

[0166] Furthermore, the work area estimation unit 132 estimates, among the work-related areas, an area surrounded by a figure whose position and shape are determined based on the line of sight, which is information indicating the user's actions, as the area where work is performed. This allows the work area estimation unit 132 to extract not only the point at which the user gazes, but also important areas including that point.

[0167] (Other Examples) The first embodiment is not limited to shogi, but can also be applied to input operations to information devices using a mouse, keyboard, touch panel, and joystick, operation of a controller of an XR device, hand gesture input, and the like.

[0168] Furthermore, the first embodiment can be applied to intellectual tasks involving the operation of equipment, as well as tasks involving the handling of documents and language (for example, responding to an operator at a call center, etc.).

[0169] Furthermore, the information indicating the user's actions while performing an operation is not limited to the trajectory of the line of sight. For example, the information indicating the user's actions while performing an operation may be mouse movement, controller movement of XR equipment, hand movement when inputting hand gestures, joystick movement, cursor movement when operating using a keyboard, the trajectory of head movement, reading response speed and accuracy, etc.

[0170] (Method of Estimating Work Area) The method of estimating the work area by the work area estimation unit 132 is not limited to the method described in FIG. 9. An example of the method of estimating the work area by the work area estimation unit 132 is shown below. Note that an action being taken means that the area has become the target of a user action. For example, an action being taken includes the user looking at a certain location, the user moving to a certain location, the user touching a certain location, etc.

[0171] The location where an action was taken is the range where the worker directed his / her gaze on the work object, the range where the worker touched with his / her hand, the range where the worker touched or operated with a controller, etc. Specifically, the location where an action was taken is the location (square) where the gaze was directed on a specific object in the VR space (for example, the shogi board in the embodiment), the location where the touch was made on a touch panel display of a smartphone or tablet, the location where the cursor was placed by operating a game controller or mouse, etc.

[0172] The work area estimation unit 132 is not limited to a rectangle, and may treat several points where the worker has acted during work as the top, bottom, left, and right edges or vertices, and estimate any polygonal area containing those points as the work area.

[0173] The work area estimation unit 132 may also estimate, as the work area, a range surrounded by a circle or ellipse of any radius, with an area in which an action has been taken for a certain period of time or a certain number of times at its center.

[0174] In addition, the work area estimation unit 132 may select multiple locations where significant input from the worker was made during work (for example, squares with significant coefficient values) and estimate a polygonal, circular, or elliptical area that contains the selected locations as the work area.

[0175] Furthermore, the work area estimation unit 132 may estimate, as the area where work will be performed, an area that is part of the work-related area and does not interfere with objects placed in the work-related area. Conversely, the work area estimation unit 132 may estimate, as the area where work will be performed, an area that is part of the work-related area and interferes with objects placed in the work-related area.

[0176] Here, an object is something other than the worker (for example, a part of the worker's body). For example, when a worker works using a PC, the mouse, keyboard, display, etc. are objects. Also, interfering with an object means that the worker touches the object or interferes with the object's operation.

[0177] For example, when a worker works in a VR space, objects in the VR space are anything other than the worker's own avatar, such as the avatar of the person interacting with them, or CG content such as an operation panel installed in the VR space.

[0178] For example, areas that do not interfere with objects include areas where a robot and a person do not come into contact when working together in a factory, areas where people (avatars) do not come into contact with each other when multiple people are working in real space or virtual space, and areas where a robot does not come into contact with objects that are not touched while working, such as a mug, when doing desk work.

[0179] For a task in which the worker must be careful not to interfere with a specific object, the task area estimation unit 132 estimates an area in which the worker will not interfere with the object as the task area. For example, in a factory or other setting, a worker performs a task while avoiding a robot (object) working with the worker. Also, for example, in a restaurant or other setting, workers serve food or prepare food while taking care not to bump into each other.

[0180] When a task is performed by a task subject while interfering with a specific object, the task area estimation unit 132 estimates the area where the task subject interferes with the object as the task area.

[0181] For example, when playing a game such as shogi, a worker works while interacting with (touching) the shogi board and pieces. When writing on a whiteboard, a worker works while interacting with (touching) the pen, the whiteboard, etc.

[0182] For example, areas that interfere with objects include areas where direct manipulation is performed, such as on a shogi board or piece stand when playing shogi, areas on a workbench or whiteboard that is used jointly by multiple people when working in real or virtual space, areas on a keyboard, display, or notebook that is operated during desk work, etc.

[0183] The work area estimation unit 132 may combine two or more of the work area estimation methods described above. Furthermore, the work area estimation unit 132 may combine multiple work area estimates using different methods into a single work area.

[0184] For example, when performing an operation by touch input on a smartphone or tablet, the work area estimation unit 132 can estimate the work area by combining the frequency of touching, the duration of touching, and the line of sight.

[0185] Furthermore, for example, the work area estimation unit 132 can estimate particularly important work areas based on gaze information during intellectual work that involves infrequent on-screen operations, such as reading a document on a smartphone, tablet, or PC, or participating in a web conference.

[0186] Furthermore, for example, in a work environment where the degree of freedom of the work area is high, such as a VR space or a real space, the work area estimation unit 132 can estimate the work area more precisely by utilizing gaze information and operation information in addition to narrowing down the work area by the object to be worked on.

[0187] Furthermore, for example, the work area estimation unit 132 can obtain movement patterns using a camera viewed from the ceiling, and when estimating the proficiency of staff from the movements of hall staff in a restaurant or the like, estimate the work area from their paths and stopping positions.

[0188] Furthermore, for example, when the work area estimation unit 132 estimates the proficiency level of a staff member from the movement (trajectory) of the fingers and arms of a cashier, it can estimate the work area from information on the position of the object to be operated (e.g., a button) and the path of movement of the fingers.

[0189] (Method of Estimating Work Area Using Machine Learning Model) The work area estimation unit 132 may estimate the work area using a model that has been trained in advance, or may estimate the work area without prior training in accordance with certain criteria.

[0190] The model may be trained to extract a working area that will increase the accuracy of cognitive proficiency estimation. For example, the model is trained after tuning to optimize the "location to be extracted" and "shape to be extracted (including the shape of the figure, the number of vertices, the area, etc.)" so that the accuracy of cognitive proficiency estimation will increase.

[0191] The model may be trained by supervised learning, in which a representative of the workers or the person performing the work labels important locations. Alternatively, the model may be trained by unsupervised learning using collected data. For example, the model is trained to statistically analyze the collected data and extract areas that receive a lot of user interaction.

[0192] (Method of Estimating Work Area by Weighting) Furthermore, when extracting areas related to highly important user actions as work areas, the work area estimation unit 132 may weight the areas as being particularly high or low in importance within the work area. The work area estimation unit 132 may estimate the work area in multiple stages, such as by further narrowing down an area within the work area to which a high weight (or a low weight) is set using weighting.

[0193] The work area estimation unit 132 can estimate the area where work will be performed based on information indicating the user's movements and at least one of the user's biological information and sound information of the environment in which the work will be performed. The work area estimation unit 132 can improve the accuracy of estimating the work area by using information other than the information indicating the user's movements.

[0194] For example, the work area estimation unit 132 uses the user's biological information to weight areas where the user is highly concentrated on the work. The work area estimation unit 132 can use biological information such as the worker's heart rate, brain waves, and gaze information (mainly pupil diameter, blinking, etc.) to measure and weight the concentration level.

[0195] For example, the work area estimation unit 132 assigns weights based on knowledge that beta waves in the brain waves are associated with active states and states of tension, and that pupil diameter and blink information are associated with tension, attention, and interest. When the biological information indicates that the user is active, the work area estimation unit 132 assumes that the user is concentrating on the location where the action was taken, and assigns weights to that location.

[0196] The work area estimation unit 132 may also use sound information, such as environmental sounds, device operation sounds, and user speech, for weighting. For example, when estimating the work area on a shogi board, the work area estimation unit 132 narrows down the important areas by using the sounds of pieces being placed. For example, the work area estimation unit 132 increases the value of a square that is estimated to be the source of a sound, among the squares shown in FIG. 9, by using a weight.

[0197] Furthermore, for example, the work area estimation unit 132 may detect a specific utterance from the wait staff (such as an order utterance) and narrow down the important area near the customer seats. For example, if a customer seat number is included in the utterance from the wait staff, the work area estimation unit 132 may preferentially extract the area near the customer seat number as the work area.

[0198] When prior learning is not required for cognitive proficiency estimation, the cognitive proficiency estimation unit 131 may calculate the cognitive proficiency by increasing the priority of the user's information related to locations with high weights assigned by the working area estimation unit 132. For example, when calculating the proficiency level based on the average gaze time of specific areas across multiple locations on the shogi board, the cognitive proficiency estimation unit 131 adjusts the gaze time so that the gaze time of areas with high importance becomes dominant.

[0199] [System Configuration, etc.] The components of each device shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of the devices can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic. The program may be executed not only by the CPU but also by other processors such as a GPU.

[0200] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0201] [Program] In one embodiment, the information providing device 10 can be implemented by installing an information providing program that executes the above-described information providing process as package software or online software on a desired computer. For example, by executing the above-described information providing program on an information processing device, the information processing device can function as the information providing device 10. The information processing device referred to here includes desktop and notebook personal computers. Other information processing devices also include mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handy-Phone Systems), XR devices such as all-in-one AR glasses and VR goggles, and even slate terminals such as PDAs (Personal Digital Assistants) and tablet terminals.

[0202] The information providing device 10 can also be implemented as an information providing server device that provides services related to the above-mentioned information providing process to a client terminal device used by a user. For example, the information providing server device is implemented as a server device that provides an information providing service that receives information about a task and a user as input and displays information in a format according to the user's level as output. In this case, the information providing server device may be implemented as a web server or as a cloud that provides services related to the above-mentioned information providing process through outsourcing.

[0203] 13 is a diagram showing an example of a computer that executes an information provision program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0204] The memory 1010 includes a read-only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input / output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0205] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the program that defines each process of the information providing device 10 is implemented as a program module 1093 in which computer-executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, the program module 1093 for executing processes similar to those of the functional configuration of the information providing device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0206] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

[0207] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0208] REFERENCE SIGNS LIST 1 Information provision system 10 Information provision device 11 Communication unit 12 Storage unit 13 Control unit 20, 30 Terminal device 21, 31 Input unit 22 Output unit 121 Worker DB 122 Work record DB 123 Presentation information DB 131 Cognitive proficiency estimation unit 132 Work area estimation unit 133 Presentation information generation unit 211 Operation unit 212 Gaze measurement unit 213 Biometric information measurement unit

Claims

1. An information providing device comprising: a work area estimation unit that estimates an area in which the work will be performed based on information indicating the movements of a user performing the work; a cognitive proficiency estimation unit that estimates the user's level of proficiency for the work based on the information indicating the user's movements and the area estimated by the work area estimation unit; and a presentation unit that presents information regarding the work to the user in a format corresponding to the level of proficiency estimated by the cognitive proficiency estimation unit.

2. The information providing device of claim 1, characterized in that the work area estimation unit estimates, among the areas related to the work, an area surrounded by a figure whose position and shape are determined based on the line of sight trajectory, which is information indicating the user's movements, as the area in which the work is to be performed.

3. The information providing device according to claim 1, characterized in that the work area estimation unit estimates an area that is part of the area related to the work and does not interfere with objects placed in the area related to the work as the area in which the work is performed.

4. The information providing device according to claim 1, characterized in that the work area estimation unit estimates an area which is part of the area related to the work and which interferes with objects placed in the area related to the work, as the area in which the work is to be performed.

5. The information providing device of claim 1, characterized in that the work area estimation unit estimates the area in which the work is performed based on information indicating the user's movements and at least one of the user's biometric information and sound information of the environment in which the work is performed.

6. The information providing device according to claim 1, characterized in that the work area estimation unit estimates the area in which the work is performed based on at least one of the number of gazes or the duration of gaze, which are information indicating the user's actions.

7. An information provision method executed by an information provision device, comprising: a work area estimation process for estimating an area in which the work will be performed based on information indicating the movements of a user performing the work; a cognitive proficiency estimation process for estimating the user's level of proficiency for the work based on the information indicating the user's movements and the area estimated by the work area estimation process; and a presentation process for presenting information about the work to the user in a format corresponding to the level of proficiency estimated by the cognitive proficiency estimation process.

8. An information provision program that causes a computer to execute the following steps: a work area estimation step of estimating an area in which the work will be performed based on information indicating the movements of a user performing the work; a cognitive proficiency estimation step of estimating the user's level of proficiency for the work based on the information indicating the user's movements and the area estimated by the work area estimation step; and a presentation step of presenting information about the work to the user in a format corresponding to the level of proficiency estimated by the cognitive proficiency estimation step.

Citation Information

Patent Citations

  • Experience level picture display program and device

    JP2003177854A

  • Image processor, image processing method, and image processing program

    JP2016001467A

  • Display device, display control method, and display system

    JP2020154569A