Information control device, information control program, and information control method

Abstract

To provide an information control device and others, which allow unnecessary information to be removed or displayed in a less visible manner with respect to information needed by an operator.SOLUTION: An information control device is provided, comprising an acquisition unit for acquiring multiple pieces of display information constituting a part of an XR space and information on work done by a user, and an output unit configured to output the display information in a display mode other than a normal mode when a degree of relevance between the display information and the work satisfies given conditions.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information control device that controls information to be presented. [Background technology]

[0002] XR (Extended Reality) technology is becoming more widespread. XR technology is a general term for VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), etc. Regarding XR, research is being conducted on technology to add useful information, and various services are being considered.

[0003] When performing work support using AR technology, if images of all devices present in the actual work environment and AR content are displayed on the screen at the same time, the cognitive load on the worker may increase and work efficiency may decrease. In light of this, a work support device has been proposed that presents information to support the worker's work in accordance with the work environment (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-181456 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Patent Document 1 is intended to assist workers in easily reaching devices that are the subject of work in a work space such as a server room, and is not intended to assist with work itself.

[0006] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an information control device and the like that uses information necessary for a worker as a standard, and deletes or reduces the visibility of unnecessary information when displayed. [Means for solving the problem]

[0007] An information control device according to one embodiment of the present application is characterized in that it includes an acquisition unit that acquires a plurality of pieces of display information constituting a part of an XR space and information on an operation performed by a user, and an output unit that outputs the display information in a display mode different from normal when a degree of association between the display information and the operation satisfies a predetermined condition. Effect of the Invention

[0008] According to one aspect of the present invention, it is possible to delete information that is not necessary for the worker or to display it with reduced visibility. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of the configuration of a work support system. [Diagram 2] 2 is a block diagram showing an example of a hardware configuration of an information control device. FIG. [Diagram 3] FIG. 2 is a block diagram showing an example of a hardware configuration of a display device. [Figure 4] FIG. 11 is an explanatory diagram showing an example of a screen DB. [Diagram 5] FIG. 2 is an explanatory diagram illustrating an example of a part DB. [Figure 6] FIG. 11 is an explanatory diagram illustrating an example of a classification master DB. [Figure 7] FIG. 11 is an explanatory diagram showing an example of a classification DB. [Figure 8] FIG. 11 is an explanatory diagram showing an example of a setting DB. [Figure 9] FIG. 11 is an explanatory diagram showing an example of a work procedure DB. [Figure 10] 13 is a flowchart showing an example of a procedure for a display process. [Figure 11] FIG. 2 is an explanatory diagram showing an example of a display screen. [Figure 12] FIG. 2 is an explanatory diagram showing an example of a display screen. [Figure 13] FIG. 11 is an explanatory diagram showing an example of a display setting screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment will be described with reference to the drawings. FIG. 1 is an explanatory diagram showing an example of the configuration of a work support system. The work support system 100 includes an information control device 1 and a display device 2. The information control device 1 and the display device 2 are communicatively connected by short-range wireless communication. The information control device 1 and the display device 2 do not necessarily need to be separate entities, and may be an integrated device in which the configuration and functions of the information control device 1 are incorporated into the display device 2.

[0011] The information control device 1 is configured with a tablet computer, a PC (Personal Computer), a smartphone, etc. The information control device 1 may be configured with a server computer, a workstation, etc. The information control device 1 may also be configured with a multi-computer consisting of multiple computers, a virtual machine virtually constructed by software, or a quantum computer. Furthermore, the functions of the information control device 1 may be realized by a cloud service.

[0012] The display device 2 is a display device that is worn on the face, head, or the like. The display device 2 is configured as a head-mounted display, VR goggles, AR goggles, MR goggles, SR goggles, smart glasses, or the like. The display device 2 displays an image from the user's viewpoint in the XR space. The display device 2 may be configured as a smartphone fixed to a housing for goggles. The display device 2 may include an operation device used to operate the information control device 1.

[0013] 2 is a block diagram showing an example of a hardware configuration of an information control device 1. The information control device 1 includes a control unit 11, a main memory unit 12, an auxiliary memory unit 13, a communication unit 14, and a reading unit 15. The control unit 11, the main memory unit 12, the auxiliary memory unit 13, the communication unit 14, and the reading unit 15 are connected by a bus B.

[0014] The control unit 11 has one or more arithmetic processing devices such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), etc. The control unit 11 reads out and executes a control program 1P (program, program product) stored in the auxiliary storage unit 13, thereby performing various information processing, control processing, etc. related to the information control device 1, and realizing functional units such as an acquisition unit, an output unit, etc.

[0015] The main memory unit 12 is a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, etc. The main memory unit 12 mainly temporarily stores data necessary for the control unit 11 to execute arithmetic processing.

[0016] The auxiliary storage unit 13 is a hard disk or a solid state drive (SSD) or the like, and stores the control program 1P and various DBs (databases) necessary for the control unit 11 to execute processing. The auxiliary storage unit 13 stores a screen DB 131, a part DB 132, a classification master DB 133, a classification DB 134, a setting DB 135, and a work procedure DB 136. The auxiliary storage unit 13 may be an external storage device separate from the information control device 1. The various DBs and the like stored in the auxiliary storage unit 13 may be stored in a database server or cloud storage different from the information control device 1.

[0017] The communication unit 14 simultaneously performs communication using a plurality of different communication standards. For example, the communication unit 14 communicates with the display device 2 by short-range wireless communication such as Bluetooth (registered trademark) or ZigBee (registered trademark). The communication unit 14 may communicate with the display device 2 via a Universal Serial Bus (USB) cable. On the other hand, the communication unit 14 communicates with the display device 2 via a wireless LAN router. In addition, the control unit 11 may use the communication unit 14 to download the control program 1P from another computer via a network N or the like, and store it in the auxiliary storage unit 13.

[0018] The reading unit 15 reads the portable storage medium 1a including a CD (Compact Disc)-ROM and a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 15 and store it in the auxiliary storage unit 13. The control unit 11 may also read the control program 1P from the semiconductor memory 1b.

[0019] 3 is a block diagram showing an example of the hardware configuration of the display device 2. The display device 2 includes a control unit 21, a main memory unit 22, an auxiliary memory unit 23, a communication unit 24, an input unit 25, an output unit 26, and a posture detection unit 27. Each component is connected by a bus B.

[0020] The control unit 21 has one or more arithmetic processing devices such as a CPU, an MPU, a GPU, etc. The control unit 21 provides various functions by reading and executing a control program 2P (program, program product) stored in the auxiliary storage unit 23.

[0021] The main memory unit 22 is an SRAM, a DRAM, a flash memory, etc. The main memory unit 22 mainly temporarily stores data necessary for the control unit 21 to execute arithmetic processing.

[0022] The auxiliary storage unit 23 is a hard disk or an SSD, etc., and stores various data necessary for the control unit 21 to execute processes. The various DBs and the like stored in the auxiliary storage unit 23 may be stored in a database server or cloud storage.

[0023] The communication unit 24 communicates with the information control device 1 by short-range wireless communication, or wired serial communication (e.g., USB communication) or parallel communication. The communication unit 24 also communicates with an external computer via a network N. The control unit 21 may also use the communication unit 24 to download the control program 2P from another computer via the network N or the like, and store it in the auxiliary storage unit 23.

[0024] The input unit 25 is a keyboard or a mouse. Alternatively, a controller that is held in the hand may be used as the input unit 25. The controller is equipped with an operation switch, a small joystick, and the like. The output unit 26 includes a liquid crystal display panel, and the like. The output unit 26 displays the XR image and the like output by the information control device 1. Alternatively, the input unit 25 and the output unit 26 may be integrated to configure a touch panel display. Note that the XR image and the like output by the information control device 1 may be displayed on a so-called normal PC monitor or liquid crystal display that is mainly used by connecting to a PC.

[0025] The attitude detection unit 27 detects the attitude of the display device 2 and notifies the control unit 21 of the detection result. The attitude detection unit 27 is configured using an acceleration sensor, a gyro sensor, a magnetic sensor, or the like. The attitude detection unit 27 detects the gravitational acceleration when the display device 2 is stationary, for example, using an acceleration sensor that detects acceleration in three axial directions. This allows the control unit 21 to calculate the degree to which the display device 2 is tilted with respect to the vertical direction from the detection result of the acceleration sensor (the direction of the gravitational acceleration, etc.). Furthermore, when the display device 2 is moving, the acceleration sensor detects the acceleration according to the movement, so that the attitude detection unit 27 can calculate the moving speed, moving direction, position, etc. of the display device 2. Furthermore, for example, when the attitude detection unit 27 has a gyro sensor, the tilt, position, etc. of the display device 2 can be calculated by calculating the time integral value (i.e., the rotation angle) of the angular velocity detected by the gyro sensor. However, the method of detecting the attitude of the display device 2 is not limited to this, and various other methods may be used. The tilt, position, direction, etc. of the display device 2 obtained from the detection result of the attitude detection unit 27 are called the attitude.

[0026] Next, the database used by the work support system 100 will be described. FIG. 4 is an explanatory diagram showing an example of the screen DB. The screen DB 131 stores the design of the screen to be displayed on the display device 2, particularly the work screen. The screen DB 131 includes a screen ID column, a title column, a part ID column, and a position column. The screen ID column stores a screen ID that uniquely identifies a screen. The title column stores the title of the screen. The part ID column stores part IDs that identify parts (display information) that make up the screen. Parts are also called controls, and are, for example, text boxes, pull-down menus, check boxes, etc. It is also possible to create a nested structure of multiple screens by specifying a screen as a part. The position column stores the coordinates of the position where the part is to be placed on the screen.

[0027] FIG. 5 is an explanatory diagram showing an example of a part DB. The part DB 132 stores information about parts. The part DB 132 includes a part ID column, a type column, and a property column. The part ID column stores a part ID that uniquely identifies a part. The type column stores the type of the part. As described above, the types are text boxes, pull-down menus, check boxes, etc. The property column stores attributes related to the parts. The attributes differ depending on the type of part. For example, for a text box, the attributes include the size, the content of the text to be displayed at the time of initial display, and a setting as to whether or not the user is permitted to edit the text. For a pull-down menu, the attributes include the content to be displayed as the menu, the information specifying the menu to be displayed at the time of initial display, and the value to be returned when each menu is selected. For a check box, the attributes include the value of the label to be displayed together with the check box, and a setting as to whether or not to check the initial display.

[0028] FIG. 6 is an explanatory diagram showing an example of a category master DB. The category master DB 133 stores category definitions. A category is assigned to each element based on information possessed by the element to be displayed on the display device 2. The elements are, for example, screens and parts. The information control device 1 controls how each element is displayed on the display device 2 based on the category. The category master DB 133 includes a category ID column, a major category column, and a category column. The category ID column stores a category ID that uniquely identifies a category. The major category column stores major categories of categories. The category column stores categories. A major category is a group of multiple categories. By defining the major categories, it becomes possible to control the display of multiple categories collectively.

[0029] 7 is an explanatory diagram showing an example of a classification DB. The classification DB 134 stores classifications of parts. The classification DB 134 includes a part ID column and a classification ID column. The part ID column stores part IDs. The classification ID column stores classification IDs of classifications corresponding to parts.

[0030] FIG. 8 is an explanatory diagram showing an example of the setting DB. The setting DB 135 stores display settings for each user. The setting DB 135 includes a user ID column, a noise removal column, an irrelevant item column, and a non-relevant item column. The user ID column stores a user ID that identifies a user. The noise removal column includes a background column, a person column, and a document column. The background column stores display settings for the background. The person column stores display settings for the person. In the case of VR display, the person includes an avatar. The document column stores display settings for the document. The irrelevant item column includes a history column, a link column, and a popup column. The history column stores display settings for the work history. The link column stores display settings for the hyperlink. The popup column stores display settings for the popup window. The non-relevant item column includes a reference column and a past non-reference column. The reference column stores display settings for reference information. The past non-reference column stores display settings for reference information that has not been referenced in the past. The display settings are 1 for display and 0 for non-display. The display settings may include settings other than display and non-display, such as displaying with low contrast, displaying in a reduced size, etc.

[0031] FIG. 9 is an explanatory diagram showing an example of a work procedure DB. The work procedure DB 136 stores information about work procedures. A work procedure is a grouping of multiple tasks that are performed in a series. The work procedure DB 136 includes a work ID column, a number column, a content column, and a screen ID column. The work ID column stores a work ID that identifies a work procedure. The number column stores the sequence number of the task. The content column stores the task content. The screen ID column stores the screen ID of the screen to be displayed during the task.

[0032] Next, the information processing performed by the work support system 100 will be described. FIG. 10 is a flow chart showing an example of the procedure of the display processing. The control unit 11 of the information control device 1 acquires the display setting from the setting DB 135 (step S1). The control unit 11 acquires the information of the work to be performed by the user from the work procedure DB 136 (step S2). The control unit 11 searches the screen DB 131 using the screen ID included in the information of the work as a search key, and acquires the screen information (step S3). The control unit 11 selects a part included in the screen information (step S4). The control unit 11 determines the classification of the selected part (step S5). In this embodiment, the control unit 11 determines the classification of the part from the classification DB 134. The control unit 11 searches the classification DB 134 using the part ID of the selected part as a search key, and determines the classification of the part. The control unit 11 compares the acquired classification with the display setting and determines whether or not to change the display mode of the part (step S6). When the control unit 11 determines that the display mode of the part is not to be changed (NO in step S6), the process proceeds to step S8. When the control unit 11 determines that the display mode of the part is to be changed (YES in step S6), it changes the display mode of the part (step S7). The control unit 11 determines whether or not there is an unprocessed part (step S8). When the control unit 11 determines that there is an unprocessed part (YES in step S8), it returns the process to step S4 and processes the unprocessed part. When the control unit 11 determines that there is no unprocessed part (NO in step S8), it outputs the display screen to the display device 2 (step S9) and ends the process. Note that changing the display mode of the part means, for example, displaying the text lightly, displaying the text characters in a smaller size, or not displaying the text. On the other hand, the normal display mode is a mode in which all parts included in the screen are displayed. However, when a display control different from the display control in this embodiment is performed, there may be parts that are not displayed. The display screen output in step S9 is displayed on the display device 2, and the user recognizes the XR space by looking at it. Also, when the display screen includes a part whose display mode has been changed, it can be said that the part whose display mode has been changed is output.Therefore, outputting a display screen including a part whose display mode has been changed is an example of "outputting display information in a display mode different from normal."

[0033] In the above description, when the classification of a part matches a classification specified in advance, it is determined that the part is unrelated to the work content and the display mode of the part is changed, but this is not limited to the above. As described below, it may be determined whether or not to change the display mode of the part depending on the degree of association such as the degree of match or similarity. For example, when the content of the part is text, language analysis is performed. The control unit 11 divides the text into words by morphological analysis. It is determined whether each divided word is included in the explanation of the work content. If the ratio of matching words (degree of match) is equal to or greater than a predetermined threshold, the control unit 11 changes the display mode of the part, and if the degree of match is less than the threshold, the control unit 11 does not change the display mode.

[0034] The display mode may be changed depending on the type of XR space to be output. For example, when displaying in a VR space, if the degree of relevance is low as described above, the display may be output in a mode that does not distract the user from the original work.

[0035] On the other hand, when displayed in an AR space, if the information to be displayed superimposed has a low degree of relevance to the work, it may be output in a manner that does not distract the user from the original work. Also, if the information superimposed in the real world has a low degree of relevance to the original work, the information may be output in a manner that blocks the information (for example, by masking). For example, when working, everything other than the computer screen is masked. When listening to a lecture, information other than the lecture presentation and the speaker is masked. When monitoring, information unrelated to the monitored subject is masked. Masking is expected to allow the user to concentrate more and become more immersed in work, listening to a lecture, or monitoring.

[0036] The control unit 11 also determines whether the word is not included in the explanatory text, but a word with a similar meaning is included in the explanatory text. The control unit 11 determines the similarity between the text and the explanatory text indicating the work content from the degree of match and the ratio of similar words. If the similarity is equal to or greater than a predetermined threshold, the control unit 11 changes the display mode of the part, and if the similarity is less than the threshold, the control unit 11 does not change the display mode.

[0037] The calculation of the similarity may utilize a natural language model such as BERT (Bidirectional Encoder Representations from Transformers). In addition, when the contents of the screen include text but the information on the screen does not include text data, the control unit 11 acquires the contents of the screen as an image. The control unit 11 performs character recognition from the acquired image using a known character recognition technique, and acquires the displayed text.

[0038] 11 and 12 are explanatory diagrams showing examples of display screens. Figs. 11 and 12 are screens when working on batch 0002 in the flow diagram. Fig. 11 is the normal display screen, displaying all parts (elements) to be displayed. Fig. 12 displays a narrowed-down display of parts. In Fig. 12, batches that are not the subject of work in the flow diagram do not display names and are displayed with dotted frames. Also, no detailed information is displayed.

[0039] FIG. 13 is an explanatory diagram showing an example of a display setting screen. In the example shown in FIG. 13, noise removal, irrelevant item removal, and non-work-related item removal are defined as major categories for which display control is possible. In addition, background, person, and document subcategories are defined for noise removal. History, link, and BOP subcategories are defined for irrelevant item removal. Reference and non-past reference subcategories are defined for non-work-related item removal. A slide switch is displayed for each subcategory. When the knob kn is moved to the right, removal is turned on, and the parts corresponding to the subcategory are not displayed on the screen. When the knob kn is moved to the left, removal is turned off, and the parts corresponding to the subcategory are displayed on the screen. PT is a pointer. The display device 2 controls the position of the pointer PT by eye tracking. The position of the pointer PT may be controlled by the posture of the display device 2. Input is possible by stopping the pointer PT at a predetermined position for a predetermined time or more. For example, when the pointer PT is stopped on the knob kn, ON is switched to OFF, and OFF is switched to ON. By using eye tracking, the user can set the display even when he / she is busy with a task. The display setting screen shown in Fig. 13 may be displayed on top of the display setting screen shown in Fig. 11 or 12 (overlay display).

[0040] In the description of the display setting screen, only one of the two options, "delete ON" or "delete OFF", can be selected as the display control, but this is not limited to this. The threshold value that determines whether to display normally or change the display mode may be changed by the user using the knob kn. For example, in the process of determining that a document is noise, a numerical value indicating the degree of relevance between the contents of the document and the contents of the work is calculated, and it is determined whether the numerical value is equal to or greater than the threshold value or less than the threshold value. By making the threshold value changeable by the user, it becomes possible to display documents that are determined as noise in the initial setting in the normal display mode without determining them as noise. The same applies to other items that are determined using a threshold value. For example, in the setting that documents that have not been referenced in the past are not displayed, documents that have not been referenced within the past month but have been referenced within the past three months are displayed by setting the past range by the user.

[0041] In this embodiment, since information that is not necessary for the work is not displayed, the worker can easily identify the work target, which is expected to improve the efficiency of the work.

[0042] The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims. [Explanation of symbols]

[0043] 100 Work Support System 1 Information and Control Equipment 11 Control section 12 Main memory 13 Auxiliary storage 131 Screen DB 132 Parts DB 133 Classification Master DB 134 Classification DB 135 Configuration DB 136 Work procedure DB 14 Communications Department 15 Reading unit 1P control program 1a Removable storage media 1b Semiconductor memory 2 Display device 21 Control section 22 Main memory 23 Auxiliary storage 24 Communications Department 25 Input section 26 Output section 27 Attitude detection unit 2P control program B Bus N Network

Claims

1. An acquisition unit that acquires a plurality of pieces of display information constituting a part of the XR space and information on an operation performed by a user; an output unit that outputs the display information in a display mode different from normal display mode when the degree of relevance between the display information and the work satisfies a predetermined condition; Equipped with The information control device is characterized in that the degree of association is determined based on a similarity or a degree of agreement between a word included in the display information and a word associated with the task.

2. When the degree of relevance of the display information is determined to be noise, an irrelevant item, or irrelevant to the task, the output unit outputs the display information in a display mode different from normal.

2. The information control device according to claim 1 .

3. The display mode different from the normal mode is a display with low contrast.

3. The information control device according to claim 1 or 2.

4. The display mode different from the normal mode is no display output.

3. The information control device according to claim 1 or 2.

5. Acquire a plurality of pieces of display information constituting the XR space and information on the work performed by the user; When a degree of relevance between the display information and the task satisfies a predetermined condition, the display information is output to the XR space in a display mode different from a normal display mode, The degree of association is calculated based on a similarity or a match between a word included in the display information and a word associated with the task. An information control program that causes a computer to execute a process.

6. Acquire a plurality of pieces of display information constituting the XR space and information on the work performed by the user; When a degree of relevance between the display information and the task satisfies a predetermined condition, the display information is output to the XR space in a display mode different from a normal display mode, The degree of association is calculated based on a similarity or a match between a word included in the display information and a word associated with the task. An information control method, characterized in that the processing is executed by a computer.

Images