Information providing apparatus, information providing method, and information providing program

Abstract

An information providing device (10) uses first information (for example, scene image) that is information for identifying a scene in work and second information (for example, trajectory of line of sight) that is information on a user who performs the work to estimate the level of proficiency of the user for the work. The information providing device (10) presents information on the work to the user in a format corresponding to the estimated level of proficiency.

Description

Technical Field

[0001] The present invention relates to an information providing apparatus, an information providing method, and an information providing program.

Background Art

[0002] In recent years, with the development of technologies such as VR (Virtual Reality), AR (Augmented Reality), and teleconferencing, workers have been able to perform intellectual work (such as shogi and programming) jointly with others or AI who are not present at the scene (see, for example, Non-Patent Document 1).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the conventional technology has a problem that it may be difficult to present information related to intellectual work in a form suitable for the user.

[0005] In intellectual work as described above, the proficiency of each worker in the work may vary. For this reason, it is considered that the work efficiency can be improved by exchanging information in a form suitable for the worker.

[0006] On the other hand, in the technology described in Non-Patent Document 1, it is difficult to present in a form suitable for each worker with different proficiencies.

Means for Solving the Problems

[0007] In order to solve the above-described problems and achieve the object, an information providing apparatus includes an estimation unit that estimates the proficiency level of a user with respect to the work based on first information that is information for specifying a scene in the work and second information that is information regarding the user who performs the work, and a presentation unit that presents information regarding the work to the user in a format according to the proficiency level estimated by the estimation unit.

Advantages of the Invention

[0008] According to the present invention, information regarding intellectual work can be presented to a user in a format suitable for the user.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Modes for Carrying Out the Invention

[0010] Hereinafter, embodiments of the information providing apparatus, information providing method, and information providing program according to the present application will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments described below.

[0011] [First Embodiment] FIGS. 1, 2, and 3 are diagrams for explaining the information provided by the information providing apparatus according to the first embodiment.

[0012] The worker in FIG. 1 is a user who performs work. The co-worker is a user who works together with the worker.

[0013] For example, the worker (operator) performs operations in the work. On the other hand, the co-worker (partner) provides information to the worker via the information providing apparatus 10. The co-worker provides instructions for the work or advice regarding the work to the worker.

[0014] For example, the work is an input operation to an information device using a mouse, keyboard, touch panel, and joystick. Also, the work may be an operation of a controller of an XR (AR and VR, etc.) device, hand gesture input, or the like.

[0015] Furthermore, the work may be playing a board game such as shogi. In the present embodiment, it is assumed that the work is to play shogi.

[0016] As shown in FIG. 1, the worker is playing shogi while looking at the board. Here, it is assumed that the worker is looking at the board directly or through the screen 221 displayed on a display device such as a VR goggle or a display.

[0017] It is assumed that the worker and the co-worker each use a terminal device. For example, the terminal device is a personal computer, a smartphone, a tablet-type terminal, or the like. The terminal device may be integrated with the information providing apparatus.

[0018] The collaborative partner sends a message from the terminal device via the network (NW). The network is, for example, the Internet.

[0019] The message sent by the collaborative partner is provided to the operator's terminal device by the information providing device 10. At this time, it can be said that the information providing device 10 functions as an interface in the information transmission between the operator and the collaborative partner.

[0020] The information providing device 10 changes the form when presenting information according to the proficiency of the operator's work. For example, when the proficiency of the operator's work is low, the information providing device 10 presents the information in a form that is easy to intuitively understand.

[0021] Here, the messages sent by the collaborative workers include fixed-form sentences and non-fixed-form sentences. A fixed-form sentence is a sentence whose form is determined in advance. A non-fixed-form sentence is a freely written sentence.

[0022] In the example of FIG. 1, the "content" of the message corresponds to a fixed-form sentence. Also, the "reason" of the message corresponds to a non-fixed-form sentence.

[0023] The collaborative partner sends a message "Content: First move 2 six steps. Reason: 'Flying' becomes easier to move".

[0024] "First move 2 six steps" is composed of three parts: "First move" representing the first or second player, "2 six" representing the position on the board, and "step" representing the type of piece. Since the candidates for each part are determined in advance and the combinations are finite, the information providing device 10 can easily interpret the meaning of the fixed-form sentence.

[0025] The information providing device 10 displays the visual information 222a on the screen 221. The information providing device 10 generates the visual information 222a based on the message.

[0026] In the example of FIG. 1, the operator is a beginner, that is, the proficiency of the operator in the work is low. Therefore, the information providing apparatus 10 adds information for promoting the operator's understanding to the visual information 222a.

[0027] As shown in FIG. 1, the visual information 222a includes an image representing the part of "2 six steps". Further, the visual information 222a includes a character string of "Reason: 'Fly' becomes easier to move" which is an irregular sentence. Furthermore, the visual information 222a includes a character string of "Supplement: 'Step' can only move to the previous square" as supplementary information regarding "step".

[0028] Thus, even if a beginner operator does not understand the position on the board meant by "2 six steps" and the movement of the piece "step", the operator can understand the intention of the message.

[0029] FIG. 2 shows an example of the screen 221 when the operator is a beginner to intermediate level operator, that is, the proficiency of the operator in the work is medium.

[0030] As shown in FIG. 2, the visual information 222b includes a character string representing the message as it is.

[0031] Thus, a beginner to intermediate level operator can accurately know the content of the message sent by the co-worker by looking at the visual information 222b.

[0032] FIG. 3 shows an example of the screen 221 when the operator is a proficient operator, that is, the proficiency of the operator in the work is high.

[0033] As shown in FIG. 3, the visual information 222c emphasizes one square on the board. The visual information 222c emphasizes a part of the screen by brightness, color or pattern.

[0034] As a result, a skilled operator can perceive the intention of the message without seeing the message itself sent by the co-worker by looking at the visual information 222c.

[0035] The information providing device 10 automatically estimates the proficiency level of the operator and presents a message in a format corresponding to the estimated proficiency level. Therefore, the co-worker can send a message without being aware of the operator's proficiency level.

[0036] [Configuration of the First Embodiment] FIG. 4 is a diagram showing a configuration example of the information providing system according to the first embodiment. As shown in FIG. 4, the information providing system 1 includes an information providing device 10, a terminal device 20, and a terminal device 30.

[0037] The terminal device 30 receives an input of a message from the co-worker. The terminal device 30 transmits the input message to the information providing device 10.

[0038] The information providing device 10 estimates the proficiency level of the operator. Further, the information providing device 10 transmits information based on the message received from the terminal device 30 to the terminal device 20 in a format corresponding to the estimated proficiency level.

[0039] The terminal device 20 outputs the information received from the information providing device 10 to the operator.

[0040] Here, it is assumed that the task is shogi. Also, the operator is a shogi player. Also, the co-worker gives advice on shogi to the operator. Also, the proficiency level estimated by the information providing device 10 is called the cognitive proficiency level.

[0041] As shown in FIG. 4, the information providing device 10 includes a communication unit 11, a storage unit 12, and a control unit 13.

[0042] The communication unit 11 performs data communication with other devices via a network. For example, the communication unit 11 is a NIC (Network Interface Card).

[0043] The storage unit 12 is a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an optical disk. Note that the storage unit 12 may be a semiconductor memory capable of rewriting data, such as a RAM (Random Access Memory), a flash memory, or an NVSRAM (Non Volatile Static Random Access Memory).

[0044] The storage unit 12 stores the OS (Operating System) and various programs executed by the information providing device 10. The storage unit 12 stores the operator DB 121, the work record DB 122, and the presentation information DB 123.

[0045] The operator DB 121 is information about the operator. FIG. 5 is a diagram showing an example of the operator DB. As shown in FIG. 5, the operator DB 121 includes an operator ID, work experience, and the previous estimated cognitive proficiency value.

[0046] The operator ID is an identifier indicating each row of the operator DB 121. Each row of the operator DB 121 corresponds to each operator.

[0047] The work experience represents the amount of experience of the operator corresponding to the operator ID with respect to the work. Here, the work experience is the time (unit: year) that each operator has worked.

[0048] The previous estimated cognitive proficiency value is the average value of the cognitive proficiency estimated by the information providing device 10 when the operator performed the work the previous time. The operator may perform the same work multiple times. Also, the cognitive proficiency may be estimated multiple times in one work.

[0049] Note that α is a symbol representing the recognition proficiency. Also, the symbol with a bar above α represents the average value of the recognition proficiency. Further, α has an initial value of 0 and varies in the range from 0 to 1.0.

[0050] For example, Fig. 5 shows that the work experience of the operator with operator ID "1" is 1 year and the previous estimated value of the recognition proficiency is 0.

[0051] Also, for example, Fig. 5 shows that the work experience of the operator with operator ID "2" is 10 years and the previous estimated value of the recognition proficiency is 0.9.

[0052] The work record DB122 is a record of the work of the operator. Fig. 6 is a diagram showing an example of the work record DB. As shown in Fig. 6, the work record DB122 includes a work ID, an operator ID, a scene, a line-of-sight trajectory, an operation record, an operation time, and an evaluation value.

[0053] The work ID is an identifier indicating each row of the work record DB122. Each row of the work record DB122 corresponds to each work. The operator ID is an identifier for identifying the operator.

[0054] In Fig. 6, only one row is shown as an example, but the work record DB122 stores records of a plurality of works. Also, the operator whose work is stored in the work record DB122 is not limited to the operator whose recognition proficiency is to be estimated.

[0055] The work record DB122 may be used as teacher data for training a model for calculating the recognition proficiency.

[0056] The scene is information indicating the scene when the operator performs the work. Here, the scene is an image of the shogi board which is the work target.

[0057] The line-of-sight trajectory is the trajectory of the line-of-sight movement of the operator looking at the scene for t' seconds from an arbitrary time.

[0058] The operation record is the operation performed by the operator for the scene shown in the scene. The operation time is the time taken for the operation shown in the operation record.

[0059] The evaluation value is the evaluation value for the operation performed by the operator. For example, the evaluation value is calculated based on the line-of-sight trajectory.

[0060] For example, the higher the operation that increases the winning rate in shogi, the larger the evaluation value. Also, the evaluation value may be calculated by an external evaluation tool.

[0061] Note that z´ is a symbol representing the evaluation value. Also, z´ takes a value in the range from 0 to 1.0.

[0062] Figure 6 shows that the work with work ID "1" was performed by the operator with operator ID "3", the operation record was "First move 2 six-step", and the operation time was 128 seconds. Also, Figure 6 shows that the evaluation value of the work with work ID "1" is "0.6".

[0063] The presentation information DB123 is information regarding the format (presentation format) of the information presented to the operator. Figure 7 is a diagram showing an example of the presentation information DB. As shown in Figure 7, the presentation information DB123 includes an information ID, corresponding cognitive proficiency, message information, presentation format, and presentation position.

[0064] The information providing device 10 generates a presentation format based on the message from the co-worker and stores it in the presentation information DB123. Also, the presentation format may be created manually in advance and stored in the presentation information DB123.

[0065] The information ID is an identifier indicating each row of the presentation information DB123. The corresponding cognitive proficiency is the range of values of the cognitive proficiency corresponding to each presentation format.

[0066] The message information is the part corresponding to the fixed phrase among the messages sent by the co-worker. For example, the message information includes information regarding work instructions.

[0067] The presentation form is a form generated according to the message information and the level of cognitive proficiency. The presentation position indicates the position where the information is presented. Here, the presentation position corresponds to a position on the shogi board.

[0068] FIG. 7 shows an example of the presentation information DB123 when the message from the co-worker is "Content: First player's 2 six-step move. Reason: The 'flying rook' becomes easier to move" shown in FIG. 1 and the like. Details of the method for generating the presentation form will be described later.

[0069] The control unit 13 controls the entire information providing apparatus 10. The control unit 13 is, for example, an electronic circuit such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0070] Further, the control unit 13 has an internal memory for storing programs and control data that define various processing procedures, and executes each process using the internal memory. Also, the control unit 13 functions as various processing units when various programs operate. For example, the control unit 13 includes an estimation unit 131 and a presentation unit 132.

[0071] The estimation unit 131 estimates the user's level of cognitive proficiency for the work based on first information that is information for specifying the scene in the work and second information that is information about the user who performs the work.

[0072] The user who performs the work is a worker. Also, for example, the first information is the scene of the work record DB122 shown in FIG. 6. Also, for example, the second information is the line-of-sight trajectory of the work record DB122 shown in FIG. 6.

[0073] For example, the estimation unit 131 estimates the user's cognitive proficiency for the work by using the trajectory of the user's line of sight in the work as the second information.

[0074] In addition, the estimation unit 131 can estimate the user's cognitive proficiency for the work by using at least any one of the user's actions, the history of operations performed by the user in the work, and the user's biometric information as the second information.

[0075] For example, the user's action is, for example, the line-of-sight trajectory in the work record DB122. Also, for example, the history of operations is the work record and operation time in the work record DB122. Also, for example, the user's biometric information is vital signs such as the body temperature, heart rate, and pupils of the worker.

[0076] The presentation unit 132 presents information related to the work to the user in a form corresponding to the cognitive proficiency estimated by the estimation unit 131.

[0077] For example, the presentation unit 132 presents a message to the worker using any of the presentation forms shown in FIG. 7.

[0078] The configuration of the terminal device 20 will be described. As shown in FIG. 4, the terminal device 20 includes an input unit 21 and an output unit 22.

[0079] The input unit 21 receives the input of data. The input unit 21 includes an operation unit 211, a line-of-sight measurement unit 212, and a biometric information measurement unit 213.

[0080] The operation unit 211 is a device for the worker to perform operations. For example, the operation unit 211 is a mouse, a keyboard, a touch panel, a joystick, a controller of an XR device, a sensor that recognizes hand gestures, etc.

[0081] The line-of-sight measurement unit 212 is a device for measuring the line of sight of the worker. For example, the line-of-sight measurement unit 212 is an eye tracker.

[0082] The biological information measurement unit 213 is a device for measuring the biological information of an operator. For example, the biological information measurement unit 213 is a wearable device worn by the operator.

[0083] The output unit 22 is a device for outputting the information provided by the information providing apparatus 10. For example, the output unit 22 is a display for displaying a screen.

[0084] Using FIG. 8, the processing flow of each unit including the estimation unit 131 and the presentation unit 132 will be described. FIG. 8 is a sequence diagram showing the processing flow of the information providing system according to the first embodiment.

[0085] As shown in FIG. 8, first, the input unit 31 of the terminal device 30 receives the input of a message from the co-worker (step S101).

[0086] The message input to the input unit 31 is transmitted to the presentation unit 132 via the communication unit 11 of the information providing apparatus 10 (step S102).

[0087] Here, the presentation unit 132 requests the estimation unit 131 to estimate the cognitive proficiency (step S103). The estimation unit 131 requests the operation history, gaze information, and biological information of the operator from the input unit 21 of the terminal device 20 (step S104), and acquires those information (step S105).

[0088] Then, the estimation unit 131 performs the analysis of the operation information, gaze information, and biological information (operation analysis, gaze analysis, biological analysis) (steps S106, S107, S108).

[0089] For example, in the operation analysis, gaze analysis, and biological analysis, the estimation unit 131 converts the acquired information into a form that can be used in the cognitive proficiency estimation.

[0090] Furthermore, the estimation unit 131 requests the worker information from the worker DB 121 (step S109), and acquires the worker information (step S110).

[0091] Subsequently, the estimation unit 131 estimates the cognitive proficiency of the worker using the acquired various types of information (step S111).

[0092] After that, the presentation unit 132 acquires the cognitive proficiency estimated by the estimation unit 131 (step S112), and acquires the presentation content and presentation format according to the message and the cognitive proficiency from the presentation information DB 123 (steps S113, S114). The presentation content is, for example, the message information of the presentation information DB 123.

[0093] The presentation unit 132 generates presentation information based on the acquired presentation content and presentation format (step S115). Then, the presentation unit 132 causes the output unit 22 of the terminal device 20 to output the generated presentation information (steps S116, S117).

[0094] (Example) An example in the case where the work is shogi will be specifically described. The estimation unit 131 inputs the scene s' and the gaze trajectory x(t) into a model (estimator) and calculates an evaluation value. The model is, for example, a deep neural network.

[0095] It is assumed that the model has been learned using the work record DB 122 as teacher data. That is, for the learning of the model, the scenes (shogi boards) s' of all the workers saved in the work record DB 122 and the gaze trajectory x(t) at that time are used as inputs (explanatory variables), and then the evaluation value z' of the operation (one move pointed) performed by each worker is used as an output (objective variable) for learning as teacher data. For example, in the learning, the evaluation value increases as the battle record (winning rate), which is the result of the operation, is higher.

[0096] In this way, the estimation unit 131 estimates the cognitive proficiency of the user based on the evaluation value obtained by inputting the information specifying the operation based on the first information (for example, the scene) and the second information (for example, the gaze trajectory) into the learned model.

[0097] The estimation unit 131 calculates the cognitive proficiency α by equation (1). i to calculate.

[0098]

Number

[0099] i is a subscript indicating the number of moves since the start of the shogi game. s i is the board state at the i-th move since the start of the shogi game. x i (t) is the trajectory of the gaze movement for t' seconds starting from an arbitrary time t in s i . Note that the trajectory x i (t) may be information representing the position or direction of the gaze at each time, or information representing the movement amount of the gaze trajectory. Also, the trajectory x i (t) may be information representing the location on the board where the gaze has stayed for a certain period of time or more. Also, the trajectory x i (t) may be information representing the gaze trajectory as a pattern.

[0100] z i (s i ,x i (t)) is an evaluation value output from a learned model with s i and x i (t) as inputs.

[0101] The symbol with a bar directly above α is the average value of the cognitive proficiency α when the operator who has performed operations using the system multiple times performed the previous operation. i

[0102] b i (t) is a stress value estimated from biological information (e.g., skin potential and heartbeat) for t' seconds starting from an arbitrary time t in s i . The stress value may be calculated by an existing calculation method.

[0103] τn is the time taken by the operator to make a move at the n-th move. Τi(τ) is the average of the time taken by the operator up to the i-th move.

[0104] f(m) is a function of the standard normal distribution with the work experience m as a variable. w1, w2, w3, w4 are such that 0 ≦ α​i It is a weight coefficient such that it is ≤1. a is a constant.

[0105] Here, the right side of formula (1) is the product of a row vector and a column vector. Each element of the row vector functions as a weight for each element of the column vector.

[0106] First, the estimation unit 131 calculates a first value (element z of the column vector i (s i ,x i (t))) based on the operation at time t, a second value (element 1 / T of the column vector i (τ)) based on the history at time t, and a third value (element 1 / b of the column vector i (t)) based on the biological information at a predetermined time.

[0107] Also, the estimation unit 131 obtains a fourth value (α (bar directly above) of the column vector), which is the user's recognition proficiency for the work estimated at times prior to time t.

[0108] Note that the estimation unit 131 does not use the measured values obtained from the terminal device 20 etc. as they are, but can use the reciprocals and normalized measured values of the measured values for estimating the recognition proficiency.

[0109] The element w1m of the row vector -a becomes larger as the work experience m is smaller. Also, the element w2lnm of the row vector becomes smaller as the work experience m is smaller.

[0110] Therefore, in the product of the row vector and the column vector, the less the amount of experience the user has for the work, the greater the influence of the first value and the smaller the influence of the fourth value.

[0111] The element w3f(m) of the row vector becomes larger as the work experience m is closer to the mean value of the standard normal distribution. Also, the element w4f(m) of the row vector becomes larger as the work experience m is closer to the mean value of the standard normal distribution. Note that the work experience m may be normalized according to the standard normal distribution.

[0112] Therefore, in the product of the row vector and the column vector, the closer the amount of experience with respect to the user's operation is to a predetermined average value, the greater the influence of the second value and the third value becomes.

[0113] In this way, the estimation unit 131 estimates the user's cognitive proficiency with respect to the operation based on the first value, the second value, the third value, and the fourth value, after controlling the influence when taking the product of the row vector and the column vector.

[0114] The estimation unit 131 may estimate the cognitive proficiency using one or two of the first value, the second value, and the third value excluding the fourth value among the elements of the column vector. For example, the estimation unit 131 may estimate the cognitive proficiency only from the first value, or may estimate the cognitive proficiency using a column vector having only the first value and the second value as elements.

[0115] Note that the estimation unit 131 may estimate the cognitive proficiency only from the information acquired during the operation.

[0116] For example, as in Equation (2), the estimation unit 131 can calculate the cognitive proficiency α based on the amount of movement of the line of sight l(t) per unit time. In Equation (2), the average value of the past cognitive proficiency is not used, and the biological information and the operation history acquired during the operation are used. i Note that in Equation (2), the meaning of each symbol is the same as that in Equation (1). w5, w6, and w7 are weighting coefficients for 0 ≤ α ≤ 1.

[0117]

Equation

[0118] (2) The meaning of each symbol in Equation (2) is the same as that in Equation (1). w5, w6, and w7 are weighting coefficients for 0 ≤ α i ≤ 1.

[0119] When the cognitive proficiency level estimated by the estimation unit 131 is less than the first threshold, the prompting unit 132 causes a character string and an image indicating information related to the work to be displayed in the display area of the device used by the user. When the cognitive proficiency level is equal to or greater than the first threshold and less than a second threshold greater than the first threshold, the prompting unit 132 causes a character string indicating information to be displayed in the display area. When the cognitive proficiency level is equal to or greater than the second threshold, the prompting unit 132 highlights a part of the display area according to the information. Note that the display area is, for example, the screen 221.

[0120] Here, the first threshold is, for example, 0.3. When the estimated value of the cognitive proficiency level is 0 or more and less than 0.3, the prompting unit 132 refers to the prompting information DB 123 and causes the visual information 222a including the image and the character string as shown in FIG. 1 to be displayed on the screen 221 output by the output unit 22.

[0121] Also, the second threshold is, for example, 0.6. When the estimated value of the cognitive proficiency level is 0.3 or more and less than 0.6, the prompting unit 132 refers to the prompting information DB 123 and causes the visual information 222b including the character string as shown in FIG. 2 to be displayed on the screen 221 output by the output unit 22.

[0122] Also, when the estimated value of the cognitive proficiency level is 0.6 or more, the prompting unit 132 refers to the prompting information DB 123 and causes the visual information 222c with only highlighting as shown in FIG. 3 to be displayed on the screen 221 output by the output unit 22.

[0123] Note that the prompting unit 132 may generate each visual information as an image and transmit the generated image to the terminal device 20. The terminal device 20 superimposes and displays the received image on the screen 221.

[0124] In this embodiment, the display modes of the visual information are classified into three stages by two thresholds, i.e., the first threshold and the second threshold. On the other hand, the number of thresholds is not limited to 2, and may be any positive integer according to the target task and work. In that case, the display modes of the visual information are classified into the number of stages obtained by adding 1 to the number of thresholds.

[0125] Further, the terminal device 20 may be an all-in-one type of AR glasses. In that case, the operator can see the actual shogi board through the display area of the AR glasses. Also, the screen 221 in the present embodiment is replaced by the display area of the AR glasses. Then, the presentation unit 132 projects visual information onto the actual shogi board visible from the display area by the AR function and superimposes and displays it.

[0126] The output unit 22 may be a projector. In that case, the input unit 21 may be a device such as an eye gaze measurement device, a smartwatch, a mouse, or a smartphone.

[0127] For example, the output unit 22 projects a screen 221 including a shogi board, shogi pieces, and visual information onto a desk and a screen. At that time, the operator can see the projected image without wearing a device such as AR glasses.

[0128] Also, the terminal device 20 may be a combination of an input device such as an eye gaze measurement device, a smartwatch, a mouse, or a smartphone and a display device such as a projector or a display.

[0129] Also, the terminal device 20 may be a projector equipped with an input device and a display device such as a display.

[0130] [Processing of the First Embodiment] Using FIG. 9, the processing flow of the information providing device 10 will be described. FIG. 9 is a flowchart showing the processing flow of the information providing device according to the first embodiment.

[0131] As shown in FIG. 9, first, the information providing device 10 acquires the actions, operation history, and biometric information of the operator (step S201). The information providing device 10 can acquire each piece of information from the terminal device 20 or the storage unit 12.

[0132] Next, the information providing apparatus 10 calculates the operator's proficiency level for the work from the acquired information (step S202). For example, the information providing apparatus 10 calculates the proficiency level according to equation (1).

[0133] When the calculated proficiency level is less than the first threshold value (step S203, Yes), the information providing apparatus 10 presents the information related to the work by an image and a character string (step S204). For example, the information providing apparatus 10 presents the information by the visual information 222a in FIG. 1.

[0134] When the calculated proficiency level is not less than the first threshold value (step S203, No), the information providing apparatus 10 proceeds to step S205.

[0135] When the calculated proficiency level is not less than the second threshold value (step S205, No), the information providing apparatus 10 presents the information related to the work by highlighting on the display area (step S206). For example, the information providing apparatus 10 presents the information by the visual information 222c in FIG. 3. Also, the display area here may be the screen of the display or the display area of the AR goggles.

[0136] When the calculated proficiency level is less than the second threshold value (step S205, Yes), the information providing apparatus 10 presents the information related to the work by a character string (step S207). For example, the information providing apparatus 10 presents the information by the visual information 222b in FIG. 2. Note that the second threshold value is larger than the first threshold value.

[0137] [Effect of the First Embodiment] The estimation unit 131 estimates the user's proficiency level for the work based on the first information which is the information for specifying the scene in the work and the second information which is the information about the user who performs the work. The presentation unit 132 presents the information related to the work to the user in a format according to the proficiency level estimated by the estimation unit 131.

[0138] The information providing device 10 can automatically estimate the proficiency of a user (worker) performing a task in real time. As a result, according to the first embodiment, information regarding intellectual work can be presented to the user in a suitable format.

[0139] Furthermore, even when the worker first performs a task for which proficiency is to be estimated by the information providing device 10, or when the user's proficiency has improved in a task that the information providing device 10 does not target for proficiency estimation, the information providing device 10 can estimate the proficiency from the information during the task acquired in real time.

[0140] The estimation unit 131 estimates the proficiency of the user with respect to the task by using the trajectory of the user's line of sight in the task as the second information. Thereby, information necessary for estimating the proficiency can be collected without disturbing the worker's task.

[0141] The estimation unit 131 estimates the proficiency of the user with respect to the task by using at least any one of the user's actions, the history of operations by the user in the task, and the user's biometric information as the second information. In this way, by using the information collected from multiple viewpoints, the accuracy of proficiency estimation can be improved.

[0142] The estimation unit 131 calculates a first value based on the action at a predetermined time, a second value based on the history at the predetermined time, and a third value based on the biometric information at the predetermined time, and acquires a fourth value that is the proficiency of the user with respect to the task estimated at a second time in the past from the predetermined time. The influence of the first value increases as the amount of experience of the user with respect to the task decreases, and the influence of the fourth value decreases. The proficiency of the user with respect to the task is estimated based on the first value, the second value, the third value, and the fourth value such that the influence of the second value and the third value increases as the amount of experience of the user with respect to the task approaches a predetermined average value.

[0143] Thereby, since it is possible to flexibly adjust which element to emphasize according to the amount of experience of the worker, the accuracy of proficiency estimation can be improved.

[0144] When the proficiency level estimated by the estimation unit 131 is less than the first threshold value, the prompting unit 132 causes a character string and an image indicating information related to the work to be displayed in the display area of the device used by the user. When the proficiency level is equal to or greater than the first threshold value and less than a second threshold value greater than the first threshold value, the prompting unit 132 causes the character string indicating the information to be displayed in the display area. When the proficiency level is equal to or greater than the second threshold value, the prompting unit 132 highlights a part of the display area according to the information.

[0145] In this way, beginners can intuitively understand the information through the image. On the other hand, for skilled users, unnecessary information can be avoided so as not to interfere with the work.

[0146] The estimation unit 131 estimates the proficiency level of the user based on an evaluation value obtained by inputting information for specifying an operation based on the first information and the second information into a learned model. In this way, based on the tendencies of a large number of workers, the proficiency level can be accurately estimated.

[0147] (Other embodiments) The first embodiment is not limited to shogi, and is also applicable to input operations to information devices using a mouse, keyboard, touch panel, and joystick, operations of the controller of XR devices, hand gesture inputs, and the like.

[0148] Furthermore, the first embodiment is applicable not only to intellectual work involving device operations but also to work involving handling documents and languages (for example, operator responses in a call center, etc.).

[0149] Also, the second information is not limited to the trajectory of the line of sight. For example, the second information may be mouse movement, movement of the controller of XR equipment, movement of the hand in hand gesture input, movement of a joystick, cursor movement in an operation using a keyboard, trajectory of head movement, reading response speed and accuracy, and the like.

[0150] [System configuration, etc.] Moreover, each component of each illustrated device is functionally conceptual and does not necessarily have to be physically configured as shown in the figure. That is, the specific forms of distribution and integration of each device are not limited to those shown in the figure, and all or part of them can be functionally or physically distributed or integrated in any unit according to various loads, usage situations, etc. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic. Note that the program may be executed not only by the CPU but also by other processors such as GPUs.

[0151] Also, among the various processes described in the embodiments, all or part of the processes described as being automatically performed can be manually performed, or all or part of the processes described as being manually performed can be automatically performed by a known method. In addition, regarding the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings, they can be arbitrarily changed unless otherwise specified.

[0152] [Program] As one embodiment, the information providing apparatus 10 can be implemented by installing an information providing program that executes the above-described information providing process as package software or online software on a desired computer. For example, by causing the information processing apparatus to execute the above-described information providing program, the information processing apparatus can be made to function as the information providing apparatus 10. The information processing apparatus mentioned here includes desktop or notebook personal computers. In addition, the information processing apparatus also includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone System), xR devices such as all-in-one type AR glasses and VR goggles, and further includes slate terminals such as PDAs (Personal Digital Assistants) and tablet type terminals within its scope.

[0153] Further, the information providing device 10 can also be implemented as an information providing server device that uses the terminal device used by the user as a client and provides a service related to the above information providing process to the client. For example, the information providing server device is implemented as a server device that provides an information providing service that takes information related to work and the user as input and outputs information displayed in a format according to the proficiency level. In this case, the information providing server device may be implemented as a Web server, or may be implemented as a cloud that provides a service related to the above information providing process by outsourcing.

[0154] FIG. 10 is a diagram showing an example of a computer that executes an information providing program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0155] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System), for example. The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100, for example. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0156] The hard disk drive 1090 stores, for example, an OS 1091, application programs 1092, program modules 1093, and program data 1094. That is, the programs that define the respective processes of the information providing apparatus 10 are implemented as program modules 1093 in which computer-executable code is described. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, program modules 1093 for executing processes similar to the functional configuration in the information providing apparatus 10 are stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0157] Also, the setting data used in the processes of the above-described embodiments is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program modules 1093 and the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processes of the above-described embodiments.

[0158] Note that the program modules 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, and may be stored, for example, in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program modules 1093 and the program data 1094 may be stored in another computer connected via a network (such as a LAN (Local Area Network) or a WAN (Wide Area Network)). Then, the program modules 1093 and the program data 1094 may be read by the CPU 1020 from the other computer via the network interface 1070.

Description of Reference Numerals

[0159] 1 Information providing system 10 Information providing device 11 Communication unit 12 Memory unit 13 Control unit 20, 30 Terminal devices 21, 31 Input units 22 Output unit 121 Operator DB 122 Work record DB 123 Prompt information DB 211 Operation unit 212 Gaze measurement unit 213 Biometric information measurement unit

Claims

1. An estimation unit that estimates the proficiency of the user for the operation based on first information that is information for identifying a scene in the operation and second information that is information about the user who performs the operation; A presentation unit that presents the information about the operation to the user in a format according to the proficiency estimated by the estimation unit; comprising The estimation unit calculates a first value based on the user's actions at a predetermined time, a second value based on the operation history by the user at the predetermined time, and a third value based on the biometric information of the user at the predetermined time, acquires a fourth value that is the proficiency of the user for the operation estimated at a time prior to the predetermined time, the influence of the first value increases as the amount of experience of the user for the operation decreases, and the influence of the fourth value decreases, the influence of the second value and the third value increases as the amount of experience of the user for the operation approaches a predetermined average value, estimates the proficiency of the user for the operation based on the first value, the second value, the third value, and the fourth value, which are the second information An information providing apparatus characterized by the above.

2. An estimation unit that estimates the proficiency of the user for the operation based on first information that is information for identifying a scene in the operation and second information that is information about the user who performs the operation; When the proficiency estimated by the estimation unit is less than a first threshold, a character string and an image indicating the information about the operation are displayed in a display area of the device used by the user. When the proficiency is equal to or greater than the first threshold and less than a second threshold greater than the first threshold, the character string indicating the information is displayed in the display area. When the proficiency is equal to or greater than the second threshold, a presentation unit that highlights a part of the display area according to the information; An information providing apparatus characterized by having the above.

3. The information providing apparatus according to claim 1 or 2, wherein the estimation unit estimates the proficiency of the user for the operation using the second information including the locus of the user's gaze in the operation.

4. The information providing apparatus according to claim 1 or 2, wherein the estimation unit estimates the proficiency of the user based on an evaluation value obtained by inputting the first information and the second information into a learned model.

5. An information providing method executed by an information providing device, comprising: an estimating step of estimating the proficiency of the user with respect to the work based on first information that is information for specifying a scene in the work and second information that is information regarding the user who performs the work; a presenting step of presenting information regarding the work to the user in a format corresponding to the proficiency estimated in the estimating step; and the estimating step calculates a first value based on the user's actions at a predetermined time, a second value based on the history of operations by the user at the predetermined time, and a third value based on the user's biological information at the predetermined time, acquires a fourth value that is the proficiency of the user with respect to the work estimated at a time prior to the predetermined time, wherein the influence of the first value increases as the amount of experience of the user with respect to the work decreases, and the influence of the fourth value decreases, and the influence of the second value and the third value increases as the amount of experience of the user with respect to the work approaches a predetermined average value, estimating the proficiency of the user with respect to the work based on the first value, the second value, the third value, and the fourth value, which are the second information; characterized by the above. **Claim 6**: An information providing method executed by an information providing device, comprising: an estimating step of estimating the proficiency of the user with respect to the work based on first information that is information for specifying a scene in the work and second information that is information regarding the user who performs the work; a presenting step of, when the proficiency estimated in the estimating step is less than a first threshold, displaying a character string and an image indicating the information regarding the work in a display area of a device used by the user, and when the proficiency is equal to or greater than the first threshold and less than a second threshold greater than the first threshold, displaying the character string indicating the information in the display area, and when the proficiency is equal to or greater than the second threshold, highlighting a part of the display area according to the information; characterized by including the above. **Claim 7** An information providing program for causing a computer to function as the information providing device according to claim 1 or 2.

Images