Output control system, output control method, and program

Abstract

This output control system comprises: a specifying unit; and a first output control unit. The specifying unit specifies the position of a tablet by recognizing the code displayed on a tablet in a real space by using an imaging device. The first output control unit displays a shape representing an input area of the tablet of which the position is specified in a virtual space on the head-mounted display.

Description

[Technical Field]

[0001] The present disclosure relates to an output control system and the like. [Background technology]

[0002] There is technology that allows characters, images, etc. to be displayed in virtual space.

[0003] Patent Document 1 discloses a technology in which a tracker is attached to an electronic pen or digitizer, the tracker monitors the reception of infrared laser light with a sensor, and when the sensor detects the reception of infrared laser light, turns on a light-emitting unit consisting of an LED (Light Emitting Diode). Patent Document 1 also discloses a technology in which a light-emitting tracking device detects the light emitted by the light-emitting unit of the tracker, thereby detecting the position of the electronic pen to which the tracker is attached within a space area subject to motion detection. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-046458 Summary of the Invention [Problem to be solved by the invention]

[0005] HMD (Head Mount ed Di s A user wearing a tablet (play) cannot see the tablet. As a result, they may not be able to recognize the tablet's location. Therefore, the tablet's location may be identified to help the user recognize its location. For example, when identifying the tablet's location using infrared rays, the accuracy of identifying the tablet's location may be low.

[0006] An example of an object of the present disclosure is to provide an output control system or the like that improves the accuracy of identifying the position of a tablet. [Means for solving the problem]

[0007] An output control system in one aspect of the present disclosure includes an identification means for identifying the position of the tablet by recognizing a code displayed on the tablet in real space using an imaging device, and a first output control means for displaying a shape representing the input area of ​​the tablet whose position has been identified in a virtual space on a head-mounted display.

[0008] An output control method in one aspect of the present disclosure identifies the position of a tablet by recognizing a code displayed on the tablet in real space using an imaging device, and displays a shape representing the input area of ​​the tablet whose position has been identified in a virtual space on a head-mounted display.

[0009] A program in one aspect of the present disclosure causes a computer to identify the position of a tablet by recognizing a code displayed on the tablet in real space using an imaging device, and to display a shape representing the input area of ​​the tablet whose position has been identified in a virtual space on a head-mounted display.

[0010] The program may be stored in a non-transitory computer-readable recording medium. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to improve the accuracy of identifying the position of a tablet. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a configuration example of an output control system according to a first embodiment; [Figure 2] 4 is a flowchart illustrating an example of an operation of the output control system according to the first embodiment. [Figure 3] FIG. 2 is an explanatory diagram showing an example of connections between the output control system and each device. [Figure 4] FIG. 10 is a block diagram showing a configuration example of an output control system according to a second embodiment. [Figure 5] FIG. 10 is an explanatory diagram showing an example of displaying one chord. [Figure 6] FIG. 1 is an explanatory diagram showing a first example of displaying a plurality of codes. [Figure 7] FIG. 10 is an explanatory diagram showing a second example of displaying a plurality of codes. [Figure 8] FIG. 10 is an explanatory diagram showing an example of recognizing a chord. [Figure 9] FIG. 10 is an explanatory diagram showing an example of displaying writing on an HMD. [Figure 10] FIG. 10 is an explanatory diagram showing an example of displaying an electronic pen on an HMD. [Figure 11] FIG. 10 is an explanatory diagram showing an example of code switching. [Figure 12] 10 is an explanatory diagram showing an example in which a code is associated with identification information of an HMD. FIG. [Figure 13] FIG. 10 is an explanatory diagram regarding output control of an HMD when a tablet is provided in each area. [Figure 14] 10 is a flowchart illustrating an example of an operation of the output control system according to the second embodiment. [Figure 15] FIG. 2 is an explanatory diagram illustrating an example of the hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, with reference to the drawings, embodiments of an output control system, an output control method, a program, and a non-transitory recording medium for recording a program according to the present disclosure will be described in detail. The disclosed technology is not limited to these embodiments.

[0014] (Embodiment 1) First, a basic function of an output control system will be described in the first embodiment. Fig. 1 is a block diagram showing an example of the configuration of an output control system according to the first embodiment.

[0015] The output control system 10 includes an identification unit 101 and a first output control unit 102. The identification unit 101 has a function for identifying the position of the tablet. The first output control unit 102 has a function for controlling the display of the head-mounted display. Hereinafter, the head-mounted display will be referred to as an HMD (Head Mount Display). ed Di s It is abbreviated as "play".

[0016] Here, a tablet is a device that allows pointing operations using a pointing device. Examples of pointing devices include pen-type devices. While the type and shape of the pointing device are not particularly limited, the following description will use a pen-type device as an example of a pointing device. A pen-type device is also referred to as an electronic pen. For example, a tablet is a device that reads the position pointed by the electronic pen using a sensor built into the tablet and sends at least one of information about the pointed position and information about the movement to the output control system 10. The output control system 10, for example, acquires at least one of information about the position and information about the movement as writing on the tablet. Alternatively, for example, a tablet is a device that sends characters, numbers, etc. identified by at least one of information about the position and information about the movement to the output control system 10. The output control system 10 may acquire characters, numbers, etc. identified by at least one of information about the position and information about the movement as writing on the tablet.

[0017] The identification unit 101 identifies the position of the tablet by recognizing a code displayed on the tablet in real space using an imaging device. The code is represented by, for example, at least one of numbers and letters. The code may also be represented by a one-dimensional code or a two-dimensional code. If the code is a two-dimensional code, the position can be identified. The number of codes is not particularly limited and may be one or more.

[0018] The code may also be displayed on the tablet by being attached as a sticker. If the tablet is a liquid crystal tablet, the tablet may output the code. For example, in the case of a liquid crystal tablet, the code may be dynamically changed. Changing the code is not limited to changing the information represented by the code, but also includes changing the size of the code, the number of codes, the display position of the code, etc.

[0019] The HMD may also include an imaging device. Specifically, for example, the identification unit 101 recognizes the code using an imaging device provided in the head-mounted display.

[0020] Then, the identification unit 101 identifies the position of the tablet based on the recognized code. Specifically, for example, the identification unit 101 identifies the position of the recognized code as the position of the tablet.

[0021] The first output control unit 102 displays a shape representing the input area of ​​the tablet whose position has been identified in the virtual space of the HMD. This shape may be referred to as an input shape.

[0022] Furthermore, when there is writing on the tablet, for example, the first output control unit 102 displays the writing on the tablet. Specifically, for example, the first output control unit 102 displays the writing on the tablet within an input shape. For example, when the input shape is a rectangle, the first output control unit 102 displays the writing within the rectangle. The shape representing the input area may be identified via an imaging device. Furthermore, when a two-dimensional code is used as the code, the two-dimensional code may include information on the shape representing the input area. Note that the color of the input shape and the color of the writing may be specified by the user and are not particularly limited.

[0023] 2 is a flowchart showing an example of an operation of the output control system 10 according to the first embodiment. The identification unit 101 identifies the position of the tablet by recognizing a code displayed on the tablet in real space using an imaging device (step S101). The first output control unit 102 writes on the tablet whose position has been identified, and also displays a shape representing the input area of ​​the tablet in the virtual space on the head-mounted display (step S102).

[0024] As mentioned above, a user may use an HMD to attend university lectures or corporate training sessions. If the user tries to take notes on paper while wearing the HMD, they cannot see what is in front of them. This prevents the user from checking their notes. Furthermore, the controller that controls each operation of the HMD does not allow for fine writing, as is the case with notebooks. Therefore, a user may consider using a tablet. For example, if notes written on a tablet are displayed on the HMD, the user can check the notes while taking them. However, because the user is wearing the HMD, the user cannot recognize the tablet in real space. However, when the position of an electronic pen or tablet is identified using infrared light, the accuracy of identifying the tablet's position may be low. In particular, when a user wearing an HMD writes on a tablet while moving, the accuracy of identifying the tablet's position using infrared light may be low. An example of a user writing on a tablet while moving is when a user wearing an HMD takes notes using a tablet while moving around the facility, such as a school, store, factory, or warehouse. In a school, the user may be a student, teacher, or the like. In the case of a store, the users are store clerks, customers, etc. In the case of a factory or warehouse, the users are workers, employees, etc.

[0025] Therefore, in the first embodiment, the output control system 10 identifies the position of the tablet by recognizing the code displayed on the tablet using an imaging device. Then, the output control system 10 writes on the tablet and displays a shape representing the input area of ​​the tablet in the virtual space of the HMD. This allows the output control system 10 to improve the accuracy of identifying the position of the tablet.

[0026] Furthermore, generally, an imaging device is pre-installed in many HMDs. However, some HMDs do not have infrared communication capabilities. For this reason, the output control system 10 can improve versatility by recognizing the code using an imaging device installed in the HMD. Furthermore, if the HMD does not have infrared communication capabilities, there is no need to purchase a separate device with infrared communication capabilities, which can reduce costs.

[0027] (Embodiment 2) Next, a second embodiment will be described in detail with reference to the drawings. In the second embodiment, an example of identifying the position of a tablet by recognizing a code will be described in more detail. Below, explanations of content that overlaps with the above explanation will be omitted to the extent that the explanation of the second embodiment is not unclear.

[0028] 3 is an explanatory diagram showing an example of connections between the output control system and each device. The output control system 20 is connected to an HMD 21, for example, via a communication network NT. The output control system 20 is also connected to a tablet 22, for example, via the communication network NT. The communication network NT connecting the HMD 21 and the communication network NT connecting the tablet 22 may be the same or different, and are not particularly limited.

[0029] As described above, the tablet 22 reads the position pointed by the electronic pen 23 using a sensor built into the tablet 22. Then, the tablet 22 transmits at least one of information on the pointed position and information on the movement as writing to the output control system 20 via the communication network NT. Alternatively, the tablet 22 transmits letters, numbers, etc. specified by this information as writing to the output control system 20 via the communication network NT.

[0030] The type of the HMD 21 is not particularly limited, but for example, the HMD 21 includes an imaging device. The HMD 21 is worn by a user. For example, the user may be walking, standing still, or sitting while wearing the HMD 21.

[0031] 4 is a block diagram showing an example of a configuration of an output control system 20 according to the second embodiment. The output control system 20 includes an identifying unit 201, a first output control unit 202, a writing acquisition unit 203, a second output control unit 204, an identification unit 205, and an identification information acquisition unit 206. The output control system 20 further includes the writing acquisition unit 203, the second output control unit 204, the identification unit 205, and the identification information acquisition unit 206 in addition to the components of the output control system 20 according to the first embodiment. The identifying unit 201 has the function of the identifying unit 101 according to the first embodiment as a basic function. Furthermore, the first output control unit 202 has the function of the first output control unit 102 according to the first embodiment as a basic function.

[0032] The output control system 20 also includes, for example, a code DB 2001. For example, the code DB 2001 stores, for each tablet 22, identification information for identifying the tablet 22 and a code in association with each other. This allows the tablets 22 to be associated with the codes. If multiple codes are used for each tablet 22, the code DB 2001 stores, for each tablet 22, the multiple codes in association with identification information for identifying the tablet 22. For example, if the codes are two-dimensional codes, the code DB 2001 may store, for each tablet 22, identification information for identifying the tablet 22, the code, and various information included in the code in association with each other. For example, if the codes are two-dimensional codes, the codes may include information on the size of the tablet 22, information on the size of the code, and identification information for identifying the HMD 21 previously associated with the tablet 22, and the information included in the codes is not particularly limited.

[0033] Here, the number of codes displayed on the tablet 22 is not particularly limited, and may be one or more. Furthermore, the size of the code is not particularly limited. As described in the first embodiment, the code may be displayed on the tablet 22 by being attached as a sticker. If the tablet 22 is a liquid crystal tablet, the tablet 22 may output the code. Note that an example in which the second output control unit 204 controls the display on the tablet 22 when the tablet 22 is a liquid crystal tablet will be described later.

[0034] <Code display example> Here, examples of code display will be described with reference to FIGS.

[0035] Fig. 5 is an explanatory diagram showing an example of the display of one code. As shown in Fig. 5, one code may be displayed on the tablet 22. For example, in Fig. 5, one code may be displayed so as to cover the entire tablet 22. By displaying a larger code on the tablet 22, the code becomes easier to recognize even when the user is far away from the tablet 22. The size of the code may also be treated as the size of the tablet 22.

[0036] Fig. 6 is an explanatory diagram showing a display example 1 of a plurality of codes. As shown in Fig. 6, the number of the plurality of codes is four, and the plurality of codes are displayed at the four corners of the tablet 22. Furthermore, the number of codes is not limited to the example of Fig. 6, and the plurality of codes may be displayed dispersedly at different positions. For example, if the number of codes is three, three-point positioning can be used to more accurately identify the position of the tablet 22.

[0037] Fig. 7 is an explanatory diagram showing a display example 2 of a plurality of codes. As shown in Fig. 7, the plurality of codes may be displayed so as to fill the entire tablet 22. In the example of Fig. 7, the number of the plurality of codes is 15, but the number of the plurality of codes is not particularly limited.

[0038] Returning to the description of FIG. 4 , the identification unit 201 identifies the position of the tablet 22 by recognizing the code displayed on the tablet 22 in real space using an imaging device. The identification unit 201 may continuously identify the position of the tablet 22. That is, the identification unit 201 may track the tablet 22. Alternatively, the identification unit 201 may identify the position of the tablet a predetermined number of times at specific timings. Alternatively, the identification unit 201 may perform a combination of these. For example, the identification unit 201 may identify the position of the tablet only once when the user starts using the tablet. Note that the start of use may be a timing specified by the user, or may be when the HMD 21 is started to be used. For example, the identification unit 201 may identify the position of the tablet once when use is started, and then identify the position of the tablet at predetermined time intervals. That is, the identification unit 201 may track the tablet. Furthermore, for example, if the HMD 21 is equipped with a sensor capable of detecting the movement of the user, the identification unit 201 may newly identify the position of the tablet 22 when the user moves away from the position where the identification unit 201 recognized the code. In this way, the timing at which the identification unit 201 identifies the position of the tablet is not particularly limited.

[0039] 8 is an explanatory diagram showing an example of code recognition. The identification unit 201 recognizes the code displayed on the tablet 22 by the imaging device of the HMD 21.

[0040] Then, the identification unit 201 uses the recognized code to identify the position of the tablet 22. Specifically, the identification unit 201 may identify the position of the recognized code as the position of the tablet 22.

[0041] <Example of distance determination> Furthermore, the identification unit 201 may identify the distance from the imaging device to the code based on the captured code. Specifically, for example, if the size of the code is determined in advance, the identification unit 201 identifies the distance from the imaging device to the code based on the ratio between the recognized size of the code and the actual size of the code.

[0042] <Example of determining tilt> Furthermore, the identification unit 201 may use the recognized code to identify the tilt of the tablet 22. Specifically, for example, if the vertical and horizontal sizes of the code are predetermined, the identification unit 201 can identify the tilt of the tablet 22 based on the ratio between the vertical and horizontal sizes of the recognized code and the vertical and horizontal sizes of the actual code.

[0043] Furthermore, the greater the number of codes, the more accurately the inclination can be determined. When the tablet 22 is rectangular and left and right are defined, the determination unit 201 can determine the inclination of the left side of the tablet 22 by comparing the aspect ratio of the code displayed on the left side of the tablet 22 with the aspect ratio of the recognized code. The determination unit 201 can then determine the inclination of the right side of the tablet 22 by comparing the aspect ratio of the code displayed on the right side of the tablet 22 with the aspect ratio of the recognized code on the right side. Note that when multiple codes are arranged at different positions, the codes may be displayed in a dispersed manner not limited to left and right, but may also be displayed in a dispersed manner up and down, or both up and down and left and right.

[0044] <Example of specifying the shape of the input area> The identification unit 201 may identify the shape of the input area of ​​the tablet 22 by using an imaging device. The shape of the input area may be predetermined. As described in the first embodiment, for example, when a two-dimensional code is used, the two-dimensional code may include information representing the shape of the input area, and the identification unit 201 may recognize the two-dimensional code to obtain the information representing the shape of the input area included in the two-dimensional code.

[0045] Alternatively, the length and width of the code and the length and width of the input area of ​​the tablet 22 are predetermined, and the ratio between the length and width of the code and the length and width of the input area of ​​the tablet 22 is also predetermined. besideSpecify the size, and based on the horizontal size of the recognized code and the horizontal ratio, specify the vertical size of the tablet 22. Thereby, the specifying unit 201 can specify the shape of the input area of the tablet 22.

[0046] Alternatively, the specifying unit 201 may use the imaging device of the HMD 21 to specify the shape of the input area of the tablet 22 where the code is recognized. Here, for example, the specifying unit 201 may specify the shape of the tablet 22 as the shape of the input area.

[0047] Next, the writing acquisition unit 203 acquires the writing to the tablet 22 whose position has been specified. As described above, the writing acquisition unit 203 acquires, for example, at least one of the position information and the movement information as the writing to the tablet 22.

[0048] <Display to HMD 21> The first output control unit 202 causes the HMD 21 to display the shape representing the input area of the tablet 22 whose position has been specified. Further, the first output control unit 202 causes the HMD 21 to display the writing within the input shape. When there is no writing, the first output control unit 202 may display the input shape.

[0049] In addition, the first output control unit 202 may display a picture. The shape of the picture only needs to be the input shape, and the type of the picture is not particularly limited such as a blackboard or a notebook. The picture may be specified by the user. Also, for example, pictures may be prepared in advance according to the usage scenarios of the virtual space. Then, the first output control unit 202 uses the shape of the picture corresponding to the user's usage scenario as the input shape and causes the HMD 21 to display the picture. The usage scenario may be specified by the user or automatically detected. For example, when the user is studying or receiving training, the tablet 22 is represented by a picture of a notebook, so that a situation where the user is studying while taking notes using a notebook in the real space or a situation of receiving training can be reproduced on the virtual space.

[0050] Next, an example of displaying an input shape based on the aspect ratio of the input area of ​​the tablet 22 will be described. For example, the first output control unit 202 may display the input area in the virtual space at a predetermined size based on the aspect ratio of the input area of ​​the tablet 22. More specifically, for example, the first output control unit 202 displays the input area in the virtual space at a predetermined size so that the input area has the same aspect ratio as the aspect ratio of the input area of ​​the tablet 22. Note that the same ratio also includes "approximately the same." For example, "approximately the same" is not particularly limited as long as the aspect ratio of the input area of ​​the tablet 22 and the aspect ratio of the input area in the virtual space can be recognized as the same by the user. The predetermined size is not particularly limited and may be the size of the real space, a size enlarged from the size of the real space, a size reduced from the size of the real space, or a size specified by the user. For example, the predetermined size may be switchable by the user during display on the HMD 21.

[0051] Next, an example of displaying an input shape according to the distance to the tablet 22 will be described. When the distance to the tablet 22 is specified, for example, the output control system 20 displays the input shape in the virtual space according to the specified distance. Specifically, for example, the output control system 20 displays the input shape in the virtual space so that the shorter the specified distance, the shorter the distance from the user to the input shape. For example, the output control system 20 displays the input shape in the virtual space so that the longer the specified distance, the longer the distance from the user to the input shape. For example, when a user is moving in real space, the distance from the user to the input shape in the virtual space changes according to the specified distance, allowing the user to feel as if they are moving in the virtual space.

[0052] Next, Tablet 22 TiltAn example will be described in which an input shape is displayed based on the above. When the tilt of the tablet 22 is specified, for example, the first output control unit 202 displays the input shape in the virtual space so that it has the same tilt as the specified tilt. This allows the user to check the orientation of the tablet 22.

[0053] Furthermore, for example, the first output control unit 202 may display a shape representing the input area of ​​the tablet 22 and writing on the tablet 22 at a position in the virtual space corresponding to the position of the tablet 22. For example, the user cannot recognize the position of the tablet 22 in real space because the user is wearing the HMD 21. For this reason, the first output control unit 202 may highlight the position in the virtual space corresponding to the position of the tablet 22. This makes it easier for the user to recognize the position of the tablet 22 in real space.

[0054] FIG. 9 is an explanatory diagram showing an example of displaying writing on the HMD 21. For example, in FIG. 9, the HMD 21 displays a document in a virtual space. The HMD 21 also displays a shape representing an input area of ​​the tablet 22 in the virtual space. The HMD 21 also displays writing in the shape representing the input area in the virtual space. In FIG. 9, "Page 3 will be on the test" is displayed in the shape representing the input area in the virtual space.

[0055] <Tracking of Electronic Pen 23> Returning to the description of Fig. 4, the identification unit 201 may further track the electronic pen 23 capable of operating the tablet 22. Specifically, for example, the identification unit 201 may identify the position of the electronic pen 23 through infrared communication by the HMD 21. Alternatively, for example, the identification unit 201 may identify the position of the electronic pen 23 through image recognition by an imaging device of the HMD 21.

[0056] The first output control unit 202 further displays, in the virtual space, the identified electronic pen 23 or a mark indicating the contact point of the electronic pen 23. Here, displaying the electronic pen 23 means, for example, displaying an image of the electronic pen 23, a model of the electronic pen 23, or the like. The contact point of the electronic pen 23 is the point at which the electronic pen 23 contacts the tablet 22.

[0057] Furthermore, if the electronic pen 23 moves too quickly, it may be difficult for the user to see the electronic pen 23 or the mark representing the electronic pen 23 displayed on the HMD 21. Therefore, the first output control unit 202 displays the electronic pen 23 or the mark representing the electronic pen 23 at predetermined intervals. The predetermined intervals may be any interval that is easy for the user to see, may be determined by the user, and are not particularly limited.

[0058] Fig. 10 is an explanatory diagram showing an example of displaying the electronic pen 23 on the HMD 21. In Fig. 10, the HMD 21 further displays the electronic pen 23 in addition to Fig. 9. This visualizes the movement of the electronic pen 23, allowing the user to confirm the operation by the user.

[0059] <Example of code display on LCD tablet> As described above, a liquid crystal tablet may be used as the tablet 22. When the tablet 22 is a liquid crystal tablet, the second output control unit 204 controls the display of the tablet 22. For example, when the tablet 22 is a liquid crystal tablet, the second output control unit 204 may display code on the tablet 22. Generally, writings on the tablet 22 are displayed on liquid crystal tablets, but when code is to be displayed on the tablet 22, the second output control unit 204 displays the code on the tablet 22 but does not display the writings on the tablet 22. Alternatively, the second output control unit 204 may display the code on the tablet 22 and display the writings on the tablet 22 on the tablet 22 so that the code is not obscured.

[0060] For example, the second output control unit 204 may display the code in a predetermined size, a predetermined number, and at a predetermined display position.

[0061] Furthermore, for example, the second output control unit 204 may display the code so that the size, number, and display position correspond to the distance from the HMD 21 to the tablet 22.

[0062] <Example of code modification using an LCD tablet> Furthermore, when a liquid crystal tablet is used as the tablet 22, the second output control unit 204 may dynamically change the code. As an example of changing the code, the second output control unit 204 changes at least one of the size of the code, the number of codes, and the display position of the code.

[0063] Here, the timing at which the second output control unit 204 changes the code is not particularly limited, but for example, the second output control unit 204 may change the code when the code is not recognized for a predetermined time. Here, an example of changing the number of codes and the code size when the code is not recognized for a predetermined time will be described with reference to FIG. 11.

[0064] Fig. 11 is an explanatory diagram showing an example of code switching. In Fig. 11, the second output control unit 204 displays four codes at the four corners of the tablet 22. For example, a user wearing the HMD 21 may be located at a predetermined location in real space, away from the tablet 22. The predetermined space may be outdoors or indoors. For example, the predetermined space may be a store, a school, or the like, and is not particularly limited.

[0065] Because the user is located far away from the tablet 22, the identification unit 201 cannot recognize the code using the imaging device of the HMD 21. Therefore, for example, when the identification unit 201 cannot recognize the code for a predetermined time, the second output control unit 204 reduces the number of codes to one and increases the size of the code, thereby displaying one code so that it covers the entire tablet 22. This makes it easier for the user to recognize the code even when the user is located far away from the tablet 22.

[0066] As described above, the greater the number of codes, the more accurate the identification of the tilt of the tablet 22. Therefore, as another example of the timing for changing the code, the second output control unit 204 may change the code when the distance from the HMD 21 to the tablet 22 becomes shorter than the predetermined distance from a distance equal to or greater than a predetermined distance. This case is, for example, when the user approaches the tablet 22. More specifically, the second output control unit 204 may change the display of the tablet 22 from the code shown in FIG. 5 to the code shown in FIG. 7 or the code shown in FIG. 6. For example, in FIG. 11, when the identification unit 201 recognizes a code, the identification unit 201 identifies the distance from the HMD 21 to the tablet 22. Although not shown, the second output control unit 204 may restore the number and size of the codes when the distance becomes within the predetermined distance.

[0067] Furthermore, if the distance from the HMD 21 to the tablet 22 is long, the imaging device provided in the HMD 21 has difficulty recognizing the code. Therefore, as another timing for changing the code, the second output control unit 204 may change the code when the distance from the HMD 21 to the tablet 22 changes from a case where the distance is shorter than a predetermined distance to a case where the distance is equal to or greater than the predetermined distance. That is, this case would be, for example, when the user moves away from the tablet 22. Specifically, for example, the second output control unit 204 may change the code so that the number of codes is reduced and the size of the codes is increased. More specifically, for example, the second output control unit 204 may change the display of the tablet 22 from the codes shown in FIG. 7 or the codes shown in FIG. 6 to the codes shown in FIG. 5.

[0068] Furthermore, for example, the user wearing the HMD 21 may move. Therefore, as another timing for changing the code, if the HMD 21 is equipped with a sensor capable of detecting the user's movement, the second output control unit 204 may change the code when it is detected that the user has moved away from the position where the identification unit 201 recognized the code. For example, if the distance from the HMD 21 to the tablet 22 is short and movement of the user is detected, the second output control unit 204 may change the code so as to increase its size. Note that the second output control unit 204 may reduce the number of codes in order to increase the size of the code.

[0069] 11, if the code is not recognized for a predetermined time, the code may be obscured by a hand or other object, making it impossible for the image capture device to recognize the code. For example, if the code is recognized at least once and then cannot be recognized for a predetermined time or longer, the second output control unit 204 may change the code. In such a case, the second output control unit 204 may display a smaller code, for example.

[0070] Furthermore, if the code cannot be recognized for a predetermined time or longer after being recognized at least once, the identification unit 205 may identify whether a part of the user, such as a hand, has been captured by the imaging device. For example, if a part of the user has been captured, the second output control unit 204 may change the code.

[0071] Alternatively, the identification unit 205 may identify whether a part of the user has been photographed by the imaging device. If it is identified that a part of the user has been photographed, the second output control unit 204 may change the code.

[0072] Alternatively, instead of being limited to the example of changing the code when it is identified that part of the user has been photographed or when the code is not recognized for a predetermined period of time, the first output control unit 202 may notify the user that the code is being obscured by part of the user.

[0073] Furthermore, if the code cannot be recognized for a long time, it is possible that the user has lost sight of the tablet 22. Therefore, if the code cannot be recognized for a second predetermined time or longer, the first output control unit 202 may switch the display of the virtual space on the HMD 21 to the display of an image captured by the imaging device. This makes it easier for the user to find the tablet 22.

[0074] Furthermore, if the user turns away, the imaging device may not be able to capture an image of the code on the tablet 22. Therefore, a sensor provided in the HMD 21 may detect that the user is facing away from the tablet 22, or may detect a change in the direction of the head. Then, the first output control unit 202 may notify the user that the tablet 22 is behind them, depending on the detection results.

[0075] <Example of preventing peeping> There may be multiple tablets 22. For example, if there are multiple students in a classroom, there will be a tablet 22 and HMD 21 corresponding to each of the multiple students.

[0076] In such a case, in order to peek at what another student has written on their tablet 22, the student may try to get the imaging device of the HMD 21 they are wearing to recognize the code displayed on the other student's tablet 22.

[0077] Therefore, the code DB 2001 may store the code, the identification information of the tablet 22, and the identification information of the HMD 21 in association with each other. This allows the tablet 22 and the code to be associated with the HMD 21. The identification information acquisition unit 206 acquires the identification information of the HMD 21 that includes an imaging device worn by the user. When the identification unit 201 recognizes a code, it compares the identification information of the HMD 21 that is associated with the recognized code in the code DB 2001 with the acquired identification information.

[0078] Then, when the identification information of the HMD21 associated with the recognized code matches the acquired identification information of the HMD21, the first output control unit 202 writes on the tablet 22 and displays a shape representing the input area of ​​the tablet 22 in the virtual space of the HMD21.

[0079] On the other hand, if the identification information of the HMD21 associated with the recognized code does not match the acquired identification information, the first output control unit 202 does not allow writing on the tablet 22 or displaying a shape representing the input area of ​​the tablet 22 in the virtual space of the HMD21.

[0080] 12 is an explanatory diagram showing an example in which a code is associated with identification information of the HMD 21. In FIG.

[0081] User A wears the HMD 21-1 and uses the electronic pen 23-1. The HMD 21-1 is associated with the tablet 22-1. Specifically, for example, the code DB 2001 stores the code displayed on the tablet 22-1, the identification information of the tablet 22-1, and the identification information of the HMD 21-1 in association with each other. Therefore, writing on the tablet 22-1 is displayed on the HMD 21-1.

[0082] User B wears the HMD 21-2 and uses the electronic pen 23-2. The HMD 21-2 is associated with the tablet 22-2. Specifically, for example, the code DB 2001 stores the code displayed on the tablet 22-2, the identification information of the tablet 22-2, and the identification information of the HMD 21-2 in association with each other. Therefore, writing on the tablet 22-2 is displayed on the HMD 21-2.

[0083] Here, assume that user B peers at the code displayed on tablet 22-1 via HMD 21-2. At this time, the identification unit 201 recognizes the code displayed on tablet 22-1 using the imaging device of HMD 21-2. The identification information acquisition unit 206 acquires the identification information of HMD 21-2. The identification unit 201 compares the identification information of HMD 21-1 associated with the recognized code with the identification information of HMD 21-2. Since these pieces of identification information do not match, the first output control unit 202 does not cause HMD 21-2 to display the writing on tablet 22-1. As a result, user B cannot see the writing on tablet 22-1 by user A. At this time, the first output control unit 202 may or may not display the input shape of tablet 22-1. When the input shape is displayed, for example, user B can recognize that a tablet is nearby. Furthermore, if the input shape is not displayed, for example, user B will not be aware that there is a tablet next to him.

[0084] If the code is a two-dimensional code, the two-dimensional code may include identification information of the HMD 21. Therefore, the identification unit 201 can obtain the identification information of the HMD 21 corresponding to the tablet 22-1 by recognizing the code displayed on the tablet 22-1 using the imaging device of the HMD 21-2. The identification unit 201 then compares the identification information of the HMD 21 included in the two-dimensional code with the identification information of the HMD 21 equipped with the imaging device. Depending on the comparison result, the first output control unit 202 controls the HMD 21 equipped with the imaging device to display writing on the tablet 22 on which the code is displayed.

[0085] This makes it possible to prevent peeping, for example, in education, it makes it possible to prevent cheating. Note that instead of the identification information of the HMD 21, the identification information of the user may be used.

[0086] Furthermore, although an example has been given in which one HMD 21 is associated with the tablet 22, the number may be multiple and is not particularly limited. That is, the code DB 2001 may store, in association with each other, identification information of the tablet 22, a code displayed on the tablet 22, and identification information of each of multiple HMDs 21. For example, each student's HMD 21 may be associated with one tablet 22, and the teacher's HMD 21 may be associated with each of the tablets 22 of multiple students. This prevents students from seeing writings on the tablets 22 of other students, but allows the teacher to see writings on each student's tablet 22. Furthermore, for example, in a situation such as a meeting where one tablet 22 is shared by multiple users, the tablet 22 may be associated with multiple HMDs 21.

[0087] <Example of output control for writing to multiple tablets 22> The above-mentioned example is not limiting, and there may be multiple tablets 22. For example, in a warehouse or factory, tablets 22 may be provided for each company, or tablets 22 may be provided for each area. Also, as mentioned above, there are cases where a teacher can view the tablets 22 assigned to each of multiple students. In this way, there are cases where a user can use multiple tablets 22.

[0088] First, it is considered that it would be inconvenient to use if all writings on each of the multiple tablets 22 were displayed on the user's HMD 21. Therefore, when the identification unit 201 recognizes a new code, the first output control unit 202 displays the writings on the tablets 22 associated with the newly recognized code on the HMD 21. When the identification unit 201 recognizes a new code, the first output control unit 202 does not display on the HMD 21 the writings on the tablets 22 associated with previously recognized codes.

[0089] Fig. 13 is an explanatory diagram regarding output control of the HMD 21 when there is a tablet 22 for each area. In Fig. 13, for example, a facility has area X and area Y. In Fig. 13, tablets 22 are provided for each area. For example, area X has tablet 22-1, and area Y has tablet 22-2.

[0090] For example, initially, the user is in area X. Then, the identification unit 201 recognizes the code displayed on the tablet 22-1, for example, via the imaging device of the HMD 21 worn by the user. In such a case, the first output control unit 202 causes the HMD 21 to display the writing on the tablet 22-1 associated with the recognized code. Meanwhile, the user moves from area X to area Y, and the identification unit 201 recognizes the code displayed on the tablet 22-2 in area Y, for example, via the imaging device of the HMD 21 worn by the user. In such a case, the first output control unit 202 causes the HMD 21 to display the writing on the tablet 22-2 associated with the recognized code. Meanwhile, the first output control unit 202 does not cause the HMD 21 to display the writing on the tablet 22-1 associated with the recognized code.

[0091] As a result, the writing on the tablet 22 associated with the newly recognized code is displayed, and the writing on the other tablets 22 is no longer displayed, making it easier for the user to see the writing.

[0092] Next, another display example when there are multiple tablets 22 will be described. For example, the tablets 22 to be displayed may be determined for each time period. For example, the code DB 2001 may store, for each HMD 21, the identification information of the HMD 21, a code for each time period, and the identification information of the tablet 22, in association with each other. For example, the identification unit 201 may check whether the recognized code is a code associated with the identification information of the HMD 21 and the time period. If the recognized code matches the code associated with the identification information of the HMD 21 and the time period, the first output control unit 202 causes the HMD 21 to display, on the tablet 22, the writing on the tablet 22 associated with the code and a shape representing the input area of ​​the tablet 22. If the recognized code does not match the code associated with the identification information of the HMD 21 and the time period, the first output control unit 202 does not cause the HMD 21 to display, on the tablet 22, the writing on the tablet 22 associated with the code and a shape representing the input area of ​​the tablet 22. In this way, users can be separated by time period.

[0093] To give an example using Figure 13, for example, the code displayed on tablet 22-1 is associated with the identification information of HMD21 and the first time zone, and the code displayed on tablet 22-1 is associated with the identification information of HMD21 and the second time zone.

[0094] The identification unit 201 recognizes the code displayed on the tablet 22-1 by using the imaging device of the HMD 21. If it is the first time period, the identification unit 201 checks whether the code displayed on the tablet 22-1 is associated with the identification information of the HMD 21 and the first time period. Because the code displayed on the tablet 22-1 is associated with the identification information of the HMD 21 and the first time period, the identification unit 201 causes the HMD 21 to display the writing and shape on the tablet 22-1 associated with the recognized code.

[0095] Meanwhile, the identification unit 201 recognizes the code displayed on the tablet 22-2 by the imaging device of the HMD 21. Then, if it is the first time period, the identification unit 201 checks whether the code displayed on the tablet 22-2 is associated with the identification information of the HMD 21 and the first time period. Because the code displayed on the tablet 22-2 is not associated with the identification information of the HMD 21 and the first time period, the writing and shape on the tablet 22-2 associated with the recognized code are not displayed on the HMD 21.

[0096] Furthermore, if it is the second time zone, the identification unit 201 verifies whether the code displayed on the tablet 22-1 is associated with the identification information of the HMD 21 and the second time zone. If the code displayed on the tablet 22-1 is not associated with the identification information of the HMD 21 or the second time zone, the writing and shape on the tablet 22-1 associated with the recognized code are not displayed on the HMD 21. On the other hand, if it is the second time zone, the identification unit 201 verifies whether the code displayed on the tablet 22-2 is associated with the identification information of the HMD 21 and the second time zone. Since the code displayed on the tablet 22-2 is associated with the identification information of the HMD 21 and the second time zone, the writing and shape on the tablet 22-2 associated with the recognized code are displayed on the HMD 21.

[0097] Alternatively, as another display example when there are multiple tablets 22, the first output control unit 202 displays a shape representing the input area of ​​the tablet 22 and writing on the tablet 22 for each of a predetermined number of tablets 22 among the multiple tablets 22. In this display example, for example, the first output control unit 202 selects a specified number of tablets 22 from the multiple tablets 22. The selection method may be the order in which they were recognized, or may be determined by time period, and is not particularly limited. For example, selecting the predetermined number of tablets 22 in the order in which they were recognized may mean selecting the predetermined number of tablets 22 that were recognized most recently, or the predetermined number of tablets 22 that were recognized first. Then, the first output control unit 202 displays a shape representing the input area of ​​the tablet 22 and writing on the tablet 22 for each of the selected predetermined number of tablets 22.

[0098] 14 is a flowchart showing an example of an operation of the output control system 20 according to the second embodiment. Here, as an example of the operation, an example will be described in which a code displayed on the tablet 22 is recognized at the start of use and then the code is recognized again at a predetermined timing. Also, as an example of the operation, an example will be described in which the tablet 22 is a liquid crystal tablet and the code is changed if the code cannot be recognized for a predetermined time or longer.

[0099] The identification unit 201 determines whether the code was recognized in a time shorter than a predetermined time (step S201). Next, if the code was recognized in a time shorter than the predetermined time (step S201: Yes), the identification unit 201 identifies the position of the tablet 22 based on the recognized code (step S202). Note that in step S202, the identification unit 201 may identify at least one of the distance from the HMD 21 to the tablet 22 and the inclination of the tablet 22. Note that in step S201, the identification unit 201 identifies a shape representing an input area of ​​the tablet 22.

[0100] The writing acquisition unit 203 acquires the writing on the tablet 22 associated with the recognized code (step S203). Then, the first output control unit 202 displays a shape representing the input area of ​​the tablet 22 together with the acquired writing (step S204).

[0101] Next, the identification unit 201 determines whether it is time to recognize the code (step S205). If it is not time to recognize the code (step S205: No), the process returns to step S203. As a result, when a new writing is acquired, the new writing is displayed on the HMD 21.

[0102] On the other hand, if it is time to recognize the code (step S205: Yes), the process returns to step S201, whereby the identification unit 201 recognizes the code again.

[0103] Furthermore, if the code is not recognized for a predetermined time or longer (step S201: No), the second output control unit 204 changes the code (step S206) and returns to step S201. For example, if the code is not recognized for a predetermined time or longer, the second output control unit 204 may change the code shown in Fig. 5 from the code shown in Fig. 6 or 7.

[0104] In this way, these processes are repeated as needed.

[0105] As described above, in the second embodiment, the code displayed on the tablet 22 may be a two-dimensional code, which allows various information to be included in the code.

[0106] Furthermore, the output control system 20 identifies the tilt of the tablet 22 and causes the HMD 21 to display the input shape of the identified tilt.

[0107] Furthermore, the output control system 20 specifies the distance from the imaging device to the tablet 22 and displays an input shape in the virtual space according to the specified distance. For example, the output control system 20 displays the input shape in the virtual space so that the shorter the specified distance in the virtual space, the shorter the distance from the user to the input shape. For example, the output control system 20 displays the input shape in the virtual space so that the longer the specified distance in the virtual space, the longer the distance from the user to the input shape. For example, when the user is moving in the real space, the distance from the user to the input shape in the virtual space changes according to the specified distance, thereby giving the user the feeling that they are moving in the virtual space.

[0108] Furthermore, multiple codes may be displayed at different positions on the tablet 22. This can improve the accuracy of specifying the position, distance, and tilt.

[0109] Also, tablets To 22 If the tablet 22 is a liquid crystal tablet, the output control system 20 changes at least one of the number of codes displayed on the tablet 22, the size of the codes, and the display position of the codes.

[0110] Also, if the code is not recognized for a predetermined period of time, the output control system 20 changes the code displayed on the tablet 22 so as to reduce the number of codes and increase the size of the codes.

[0111] Furthermore, the output control system 20 changes at least one of the number of codes and the size of the codes displayed on the liquid crystal tablet according to the distance from the imaging device to the tablet 22.

[0112] Furthermore, the output control system 20 identifies the position of the electronic pen 23 that can operate the tablet 22, and further displays in the virtual space a shape representing the identified electronic pen 23 or a mark indicating the contact point of the electronic pen 23. This visualizes the movement of the electronic pen 23 in the virtual space. Note that the contact point of the electronic pen 23 is the point of contact with the tablet 22. As a method for identifying the position of the electronic pen 23, the output control system 20 identifies the position of the electronic pen 23 by infrared communication via the tablet 22. As another method for identifying the position of the electronic pen 23, the output control system 20 identifies the position of the electronic pen 23 by image recognition using an imaging device.

[0113] Furthermore, the output control system 20 displays the tablet 22 and writings on the tablet 22 at a position in the virtual space corresponding to the position of the tablet 22. Therefore, the user can take notes while recognizing the tablet 22 in the real space in the virtual space.

[0114] Furthermore, the output control system 20 displays the input shape in a predetermined size in the virtual space based on the aspect ratio of the input area. This allows the tablet 22 to be displayed in the virtual space at a size suited to the user while maintaining the aspect ratio. Therefore, the user can take notes while recognizing the tablet 22 in the real space in the virtual space.

[0115] Furthermore, the output control system 20 displays a picture corresponding to the usage scene in the virtual space. The shape of this picture is the input shape of the tablet 22. It is possible to reproduce the usage scene in the real space when the user takes notes.

[0116] This concludes the description of embodiment 2. Embodiment 2 may be modified and used. Modifications will be described below.

[0117] <Variation 1> In the second embodiment, an example of a method for specifying the position of the tablet 22 by a code has been described. In the first modification, at least one of the example of a method for specifying the position of the tablet 22 using infrared rays or ultrasonic waves and the method for specifying the position of the tablet 22 by image recognition using an imaging device may be used in combination with the example of a method for specifying the position of the tablet 22 by a code. In this way, the position of the tablet 22 is specified by combining a plurality of specifying methods, thereby achieving further improvement in accuracy. Alternatively, a plurality of specifying methods may be repeated.

[0118] When infrared rays or ultrasonic waves are used, for example, the HMD 21 is provided with a device for emitting infrared rays or ultrasonic waves, and the tablet 22 is provided with a sensor such as an infrared sensor or an ultrasonic sensor. The identification unit 201 may identify the position of the tablet 22 depending on the reception status of the infrared rays or ultrasonic waves emitted from the HMD 21 at the tablet 22. The same applies when the identification unit 201 tracks the electronic pen 23 using infrared rays or ultrasonic waves.

[0119] Furthermore, when image recognition using an imaging device is used, the identification unit 201 captures an image of the tablet 22 using the imaging device of the HMD 21. Then, the identification unit 201 can identify the position of the tablet 22 by detecting four corners or three points of the captured image of the tablet 22. Alternatively, the identification unit 201 can identify the position of the tablet 22 by detecting the frame of the input area of ​​the captured image of the tablet 22. Note that the frame is a line. Note that in the case of the electronic pen 23, for example, the identification unit 201 may track part or all of the electronic pen 23, such as the contact point of the electronic pen 23 or the other end of the electronic pen 23, by image recognition.

[0120] This concludes the description of each embodiment and each modified example. Note that each embodiment and the modified examples described in each embodiment may be used in combination. Also, for example, in each embodiment, the output control system may be configured to include each functional unit and part of the information.

[0121] Furthermore, the above-described embodiments and modifications are not limited to the examples described above, and various modifications are possible. Furthermore, the configuration of the output control system 20 in each embodiment is not particularly limited. The output control system 20 may be realized by a single device, such as a single terminal device or a single server. Alternatively, the output control system 20 may be realized by different devices for different functions or data. Furthermore, the output control system 20 may be configured by multiple servers.

[0122] Furthermore, between the output control system 20 and other devices described in each embodiment and variant, the subject of transmission and reception may be the information itself or an identifier representing the information, and is not particularly limited.

[0123] In each embodiment and modification, each table or piece of information may include part of the information described above. Also, each table or piece of information may include information other than the information described above. Each table or piece of information may be divided into multiple tables or multiple pieces of information in more detail. In this way, there are no particular limitations on how each table or piece of information is realized.

[0124] Furthermore, the screens displayed on the tablet 22 and the HMD 21 are merely examples and are not particularly limited. Buttons, lists, check boxes, information display fields, input fields, and the like (not shown) may be added to each screen. Furthermore, the background color of the screen may be changed.

[0125] Also, for example, in each embodiment, the process of generating information on the screen to be displayed on the tablet 22 may be performed by the second output control unit 204 of the output control system 20, or may be performed by the tablet 22.

[0126] Also, for example, in each embodiment, the process of generating screen information to be displayed on the HMD 21 may be performed by the first output control unit 202 of the output control system 20 or by the HMD 21. Specifically, for example, the first output control unit 202 may transmit writing and input shapes and display-related information such as their display positions to the HMD 21, and the HMD 21 may generate screen information based on this information and display the screen. Alternatively, the first output control unit 202 may generate screen information to be displayed on the HMD 21 and transmit the screen information to the HMD 21, and the HMD 21 may display the screen based on the received screen information.

[0127] (Example of computer hardware configuration) Next, an example of a hardware configuration in which each device, such as the output control system 20, HMD 21, and tablet 22, described in each embodiment, is implemented by a computer will be described. Fig. 15 is an explanatory diagram showing an example of a hardware configuration of a computer. For example, some or all of each device can be implemented using any combination of a computer 80 and a program as shown in Fig. 15.

[0128] The computer 80 includes, for example, a processor 801, a read-only memory (ROM) 802, a random access memory (RAM) 803, and a storage device 804. The computer 80 also includes a communication interface 805 and an input / output interface 806. The components are connected to each other, for example, via a bus 807. The number of each component is not particularly limited, and there may be one or more of each component.

[0129] The processor 801 controls the entire computer 80. Examples of the processor 801 include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit). The computer 80 has a storage unit including a ROM 802, a RAM 803, and a storage device 804. Examples of the storage device 804 include a semiconductor memory such as a flash memory, a hard disk drive (HDD), and a solid state drive (SSD). For example, the storage device 804 stores an operating system (OS) program, application programs, and programs according to the embodiments. Alternatively, the ROM 802 stores application programs and programs according to the embodiments. The RAM 803 is used as a work area for the processor 801.

[0130] The processor 801 also loads programs stored in the storage device 804, ROM 802, etc. The processor 801 then executes each process coded in the program. The processor 801 may also download various programs via the communication network NT. The processor 801 also functions as a part or all of the computer 80. The processor 801 may then execute the processes or instructions in the illustrated flowchart based on the program.

[0131] The communication interface 805 is connected to a communication network NT such as a LAN (Local Area Network) or WAN (Wide Area Network) via a wireless or wired communication line. The communication network NT may be composed of multiple communication networks NT. As a result, the computer 80 is connected to external devices and external computers 80 via the communication networks NT. The communication interface 805 serves as an interface between the communication network NT and the inside of the computer 80. The communication interface 805 also controls the input and output of data from external devices and external computers 80.

[0132] Furthermore, the input / output interface 806 is connected to at least one of an input device, an output device, and an input / output device. The connection method may be wireless or wired. Examples of input devices include a keyboard, a mouse, and a microphone. Examples of output devices include a display device, a lighting device, and a speaker that outputs audio. Examples of input / output devices include a touch panel display. Note that the input device, output device, and input / output device may be built into the computer 80 or may be external.

[0133] The hardware configuration of computer 80 is an example. Computer 80 may have some of the components shown in FIG. 15. Computer 80 may have components other than those shown in FIG. 15. For example, computer 80 may have a drive device or the like. Then, processor 801 may read programs and data stored in a recording medium attached to the drive device or the like into RAM 803. Examples of non-transitory tangible recording media include optical disks, flexible disks, magneto-optical disks, and USB (Universal Serial Bus) memories. As described above, computer 80 may have input devices such as a keyboard and a mouse. Computer 80 may have an output device such as a display. Computer 80 may also have an input device, an output device, and an input / output device.

[0134] The computer 80 may have various sensors (not shown). The type of the sensor is not particularly limited. The sensor may be, for example, an acceleration sensor, a gyro sensor, a magnetic sensor, a GPS (Global Positioning System) sensor, or the like. ) Se Examples of sensors include fingerprint sensors, LiDAR (Light, Detection and Ranging), proximity sensors, and ambient light sensors.

[0135] This concludes the description of the hardware configuration of each device. There are various variations in the implementation of each device. For example, the output control system 20 may be implemented by any combination of different computers and programs for each component. Furthermore, multiple components included in each device may be implemented by any combination of a single computer and program.

[0136] Furthermore, some or all of the components of each device may be realized by circuits for specific applications. Furthermore, some or all of the components of each device may be realized by general-purpose circuits including a processor such as an FPGA (Field Programmable Gate Array). Furthermore, some or all of the components of each device may be realized by a combination of circuits for specific applications and general-purpose circuits. Furthermore, these circuits may be a single integrated circuit. Alternatively, these circuits may be divided into multiple integrated circuits. Furthermore, the multiple integrated circuits may be configured by being connected via a bus or the like.

[0137] Furthermore, when some or all of the components of each device are realized by a plurality of computers, circuits, etc., the plurality of computers, circuits, etc. may be centrally located or distributed.

[0138] The output control method described in each embodiment is realized by execution by the output control system 20. Furthermore, for example, the output control method is realized by execution of a prepared program by a computer such as a server or a terminal device. The program described in each embodiment is recorded on a computer-readable recording medium such as an HDD, SSD, flexible disk, optical disk, flexible disk, magneto-optical disk, or USB memory. The program is then executed by being read from the recording medium by the computer. Furthermore, the program may be distributed via a communication network NT.

[0139] The functions of each of the components of the output control system 20 in each of the embodiments described above may be realized by hardware, such as a computer, or may be realized by a computer or firmware under program control.

[0140] Although the present disclosure has been described above with reference to various embodiments, the present disclosure is not limited to the above embodiments. The configuration and details of each of the present disclosures may include embodiments to which various modifications that would be apparent to those skilled in the art are applied within the scope of the present disclosure. The present disclosure may also include embodiments in which the details described herein are appropriately combined or substituted as necessary. For example, details described using a particular embodiment may also be applied to other embodiments to the extent that no contradiction occurs. For example, although multiple operations are described in sequence in the form of a flowchart, the order of the descriptions does not limit the order in which the multiple operations are performed. Therefore, when implementing each embodiment, the order of the multiple operations may be changed as long as it does not interfere with the content.

[0141] Some or all of the above-described embodiments can be described as follows: However, some or all of the above-described embodiments are not limited to the following.

[0142] (Appendix 1) an identification means for identifying the position of the tablet by recognizing a code displayed on the tablet in real space using an imaging device; a first output control means for displaying a shape representing the input area of ​​the tablet whose position has been identified in a virtual space on a head-mounted display; An output control system comprising: (Appendix 2) the first output control means displays writing on the tablet within the shape representing the input area of ​​the tablet; 10. The output control system of claim 1. (Appendix 3) The head-mounted display includes the imaging device. 3. The output control system of claim 1 or 2. (Appendix 4) The code is a two-dimensional code. 4. The output control system of claim 1. (Appendix 5) the specifying means specifies the tilt of the tablet, the first output control means displays the specified shape of the gradient; 5. The output control system of any one of appendixes 1 to 4. (Appendix 6) the specifying means specifies a distance from the imaging device to the tablet; the first output control means displays the shape corresponding to the specified distance in the virtual space; 6. The output control system of any one of appendixes 1 to 5. (Appendix 7) The code is a plurality of codes displayed at different positions on the tablet. 7. The output control system of any one of appendixes 1 to 6. (Appendix 8) the tablet is a liquid crystal tablet, a second output control means for changing at least one of the number of the codes displayed on the liquid crystal tablet, the size of the codes, and the display position of the codes; 8. An output control system according to any one of claims 1 to 7, comprising: (Appendix 9) the second output control means, when the code is not recognized by the identification means for a predetermined time or longer, changes the code displayed on the liquid crystal tablet so as to reduce the number of the codes and increase the size of the codes. 10. The output control system of claim 8. (Appendix 10) the specifying means specifies a distance from the imaging device to the tablet; the second output control means changes at least one of the number of the codes and the size of the codes displayed on the liquid crystal tablet according to the distance. 10. The output control system of claim 8 or 9. (Appendix 11) the specifying means specifies a position of a pointing device capable of operating the tablet; the first output control means further displays, in the virtual space, a mark indicating the identified pointing device or a contact point of the pointing device; 11. The output control system of any one of appendixes 1 to 10. (Appendix 12) the specifying means specifies the position of the pointing device by infrared communication using the head-mounted display. 12. The output control system of claim 11. (Appendix 13) the specifying means specifies the position of the pointing device by image recognition using the imaging device; 13. The output control system of claim 12. (Appendix 14) the first output control means displays the shape and the writing on the tablet at a position in the virtual space corresponding to the position of the tablet in the real space; 14. The output control system of any one of claims 1 to 13. (Appendix 15) the first output control means displays the input area in the virtual space at a predetermined size based on the aspect ratio of the input area; 15. The output control system of any one of appendixes 1 to 14. (Appendix 16) the first output control means displays a picture according to a usage scene of the virtual space, the pictorial shape is the shape representing the input area of ​​the tablet; 16. The output control system of any one of appendixes 1 to 15. (Appendix 17) Identifying the position of the tablet by recognizing a code displayed on the tablet in real space with an imaging device; a shape representing the input area of ​​the tablet whose position has been identified is displayed in a virtual space on a head-mounted display; Output control method. (Appendix 18) On the computer, Identifying the position of the tablet by recognizing a code displayed on the tablet in real space with an imaging device; a shape representing the input area of ​​the tablet whose position has been identified is displayed in a virtual space on a head-mounted display; A non-transitory recording medium readable by the computer that records a program for executing processing. (Appendix 19) On the computer, Identifying the position of the tablet by recognizing a code displayed on the tablet in real space with an imaging device; a shape representing the input area of ​​the tablet whose position has been identified is displayed in a virtual space on a head-mounted display; A program that executes a process. [Explanation of symbols]

[0143] 10,20 Output control system 21 HMD 22, 22-1, 22-2 tablets 23, 23-1, 23-2 Electronic pen 101,201 Specific part 102, 202 First output control section 203 write Acquisition department 204 Second output control section 205 Identification Unit 206 Identification information acquisition unit 2001 Code DB 80 Computer 801 processor 802 ROM 803 RAM 804 Storage device 805 Communication Interface 806 Input / Output Interface 807 Bus NT Communications Network

Claims

1. an identification means for identifying the position of the tablet by recognizing a code displayed on the tablet in real space using an imaging device; a first output control means for displaying a shape representing the input area of ​​the tablet whose position has been identified in a virtual space on a head-mounted display; a second output control means for changing at least one of the number of the codes displayed on the tablet, the size of the codes, and the display position of the codes; Equipped with The tablet is a liquid crystal tablet. Output control system.

2. the first output control means displays writing on the tablet within the shape representing the input area of ​​the tablet; The output control system according to claim 1 .

3. the specifying means specifies the tilt of the tablet, the first output control means displays the specified shape of the gradient; 3. The output control system according to claim 1 or 2.

4. the specifying means specifies a distance from the imaging device to the tablet; the first output control means displays the shape corresponding to the specified distance in the virtual space; 4. The output control system according to claim 1.

5. the second output control means changes the code displayed on the tablet so as to reduce the number of codes and increase the size of the codes when the code is not recognized by the identification means for a predetermined time or longer.

5. The output control system according to claim 1.

6. the specifying means specifies a distance from the imaging device to the tablet; the second output control means changes at least one of the number of the codes and the size of the codes displayed on the tablet according to the distance.

6. The output control system according to claim 1.

7. the specifying means specifies a position of a pointing device capable of operating the tablet; the first output control means further displays, in the virtual space, a mark indicating the identified pointing device or a contact point of the pointing device; The output control system according to any one of claims 1 to 6.

8. A computer comprising: Identifying the position of the tablet by recognizing a code displayed on the tablet in real space with an imaging device; displaying a shape representing the input area of ​​the tablet whose position has been identified in a virtual space on a head-mounted display; changing at least one of the number of the codes displayed on the tablet, the size of the codes, and the display positions of the codes; Execute the process, The tablet is a liquid crystal tablet. Output control method.

9. On the computer, Identifying the position of the tablet by recognizing a code displayed on the tablet in real space with an imaging device; displaying a shape representing the input area of ​​the tablet whose position has been identified in a virtual space on a head-mounted display; changing at least one of the number of the codes displayed on the tablet, the size of the codes, and the display positions of the codes; Execute the process, The tablet is a liquid crystal tablet. program.

Images