Display system, display control device, and display control program

Abstract

To enable a user to confirm a conversation record later in a display system for displaying a spoken dialog after conversion into text.SOLUTION: A display system according to the present invention comprises a display unit, a voice acquisition unit, a display control device, an operation unit, and a server. The display unit, when used, is located between a plurality of users. The voice acquisition unit is located at a position at which voices of one or more of the plurality of users are acquirable. The display control device receives input of a voice signal from the voice acquisition unit and causes the display unit to display text information converted from the voice signal. The operation unit accepts operation on the display control device. The display control device transmits, to the server, text information displayed on the display unit before a second operation is accepted after accepting a first operation, on the basis of the first and the second operation on the operation unit, as well as acquires connection information for connecting to the server and causes the display unit to display the connection information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a display system, a display control device, and a display control program. [Background technology]

[0002] Conventionally, in situations where users such as a counter clerk and a visitor are conversing across a partition, a technology has been proposed in which the voice spoken by at least one person is converted into text information and displayed on a transparent display that is visible to the other speaker (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-124615 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional techniques have the problem that the content of the conversation between the counter staff and the visitor cannot be recorded and checked later.

[0005] Therefore, the purpose of the present disclosure, which has been made with this in mind, is to enable a user to check a conversation record in a display system that converts a spoken dialogue into text and displays it. [Means for solving the problem]

[0006] In one embodiment, (1) a display system includes a display unit that is positioned between multiple users when in use, an audio acquisition unit that is positioned so as to acquire the audio of one or more of the multiple users, a display control device that receives an audio signal from the audio acquisition unit and causes the display unit to display text information converted from the audio signal, an operation unit that accepts operations on the display control device, and a server, wherein the display control device transmits the text information that has been displayed on the display unit from when the first operation is accepted until when the second operation is accepted based on a first operation and a second operation on the operation unit, and acquires connection information for connecting to the server and displays the connection information on the display unit.

[0007] (2) In the display system of (1) above, the display control device can display a notification regarding the handling of personal information based on the first operation on the operation unit, and can transmit the character information to the server based on the third operation on the operation unit in response to the notification and the second operation.

[0008] (3) In the display system of (1) or (2) above, the connection information may include a two-dimensional code and a password for connecting to the server.

[0009] (4) In any of the display systems (1) to (3) above, the display control device does not need to cause the audio acquisition unit to acquire audio while the connection information is being displayed on the display unit.

[0010] (5) In any of the display systems (1) to (4) above, the server can use the connection information to display the character information on a user terminal connected to the server, and when a summary instruction is received from the user terminal, generate or obtain summary information that summarizes the character information and display the summary information on the user terminal.

[0011] (6) In the display system of (5) above, the server determines whether or not there is summary information that has been generated or acquired in the past, and if there is summary information that has been generated or acquired in the past, when it receives the summary instruction from the user terminal, it does not generate or acquire the summary information, but can display the summary information that has been generated or acquired in the past on the user terminal.

[0012] In one embodiment, (7) a display control device includes a display unit that is placed between multiple users when in use, a voice acquisition unit that is placed in a position where it can acquire the voice of one or more of the multiple users, a communication unit that is configured to be able to communicate with a server, an input unit that receives input from an operation unit, and a control unit that acquires a voice signal from the voice acquisition unit via the communication unit and causes the display unit to display character information into which the voice signal is converted, and the control unit transmits the character information that has been displayed on the display unit from the time the first operation is accepted until the time the second operation is accepted to the server based on a first operation and a second operation on the operation unit, and acquires connection information for connecting to the server and causes the connection information to be displayed on the display unit.

[0013] In one embodiment, (8) the display control program is a control program for a display control device that includes a display unit that is placed between multiple users when in use, a voice acquisition unit that is placed in a position where it can acquire the voice of one or more of the multiple users, a communication unit that is configured to be able to communicate with a server, and an input unit that receives input from an operation unit, and causes a control unit of the display control device to execute processing including acquiring a voice signal from the voice acquisition unit via the communication unit, displaying character information into which the voice signal is converted on the display unit, and transmitting to the server, based on a first operation and a second operation on the operation unit, the character information displayed on the display unit from when the first operation is accepted until when the second operation is accepted, and acquiring connection information for connecting to the server and displaying the connection information on the display unit. [Effects of the Invention]

[0014] According to an embodiment of the present disclosure, in a display system that converts a spoken dialogue into text and displays it, a user can later check the conversation record. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating an example of a usage scene of a display system according to an embodiment. [Figure 2] FIG. 2 is a top view of the display system of FIG. 1. [Figure 3] FIG. 2 is a block diagram showing a schematic configuration of the display system of FIG. [Figure 4] FIG. 10 is a diagram showing an example of a display on a transparent screen as seen from the first user's side. [Figure 5] FIG. 10 is a diagram showing an example of a display on the transparent screen as seen from the side of a second user. [Figure 6] 10 is a flowchart illustrating an example of processing executed by a control unit of the display control device. [Figure 7] 8 is a flowchart showing a voice display process executed in FIG. 7. [Figure 8] FIG. 10 is a diagram showing an example of connection information for checking conversation records displayed to a second user. [Figure 9] 10 is a flowchart of a process executed by the first server when the second user checks the content of the conversation. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. The drawings used in the following description are schematic. The dimensions and ratios in the drawings do not necessarily correspond to the actual dimensions and ratios.

[0017] (Display system configuration) As shown in FIGS. 1 and 2, a display system 1 according to one embodiment is a system that supports dialogue between a first user U1 and a second user U2. As an example, the first user U1 is an employee of a government, local government, or public institution. The second user U2 is, for example, an ordinary citizen visiting the government, local government, or public institution. This system is particularly effective when the second user U2 is hearing impaired, for example. The situations in which the display system 1 is used are not limited to those described above. The display system 1 can also be used at the counters of financial institutions, medical institutions, public transportation facilities, etc., at the business offices of private businesses, and in office conference rooms, etc.

[0018] An acrylic plate, a vinyl curtain, or the like is installed between the first user U1 and the second user U2. In recent years, acrylic plates or vinyl curtains have been installed as a measure against droplets. In the display system 1, the acrylic plate, the vinyl curtain, or the like is used as a base material 10, and a transparent screen 5 is placed on this base material 10. The projector 30 projects the content of the speech of the first user U1 and / or the second user U2 onto this transparent screen 5. This allows the first user U1 and / or the second user U2, who are conversing across the transparent screen 5, to visually confirm the content of the speech even if they cannot hear what the other person is saying.

[0019] The transparent screen 5 can be placed anywhere on the base material 10. The transparent screen 5 may be placed in a position that does not prevent the first user U1 and the second user U2 from seeing each other's faces. For example, the transparent screen 5 may be placed in a position where the first user U1 and the second user U2 lower their lines of sight downward from the horizontal direction.

[0020] As shown in FIGS. 1 to 3 , the display system 1 includes a transparent screen 5, a display control device 20, an operation unit 25, a projector 30, a first microphone 40a, a second microphone 40b, and a first server 61, which is a cloud server located in a remote location and performs voice recognition processing. The transparent screen 5 and the projector 30 constitute a display unit. Hereinafter, the transparent screen 5 and the projector 30 may be collectively referred to as the display unit. The first microphone 40a and the second microphone 40b constitute a voice acquisition unit. The operation unit 25, the first microphone 40a, and the second microphone 40b are each connected to the display control device 20 via a wired or wireless connection so as to be able to communicate with each other. The display control device 20 and the projector 30 are connected via a wired or wireless connection so as to be able to communicate with each other. The display control device 20 is configured to be able to communicate with the first server 61 via a network 60. The display system 1 may further include a second server 62, which is a cloud server having a text summarization function. The display control device 20 and the first server 61 are configured to be able to communicate with a second server 62 via a network 60.

[0021] The transparent screen 5 can be a film-, sheet-, or plate-like member for projector projection that can be attached to the base material 10. The transparent screen 5 diffuses a portion of the light incident from the projector 30 to the incident side and the exit side. The first user U1 and the second user U2 can recognize the image projected from the projector 30 when the light diffused by the transparent screen 5 enters their field of vision. The shape of the transparent screen 5 can be, for example, rectangular, but is not limited to this. The transparent screen 5 can have various shapes.

[0022] The first microphone 40a is arranged on the first user U1 side of the transparent screen 5. For example, the first microphone 40a may be arranged on a desk in front of the first user U1. The first microphone 40a mainly converts the voice uttered by the first user U1 into a voice signal and outputs it to the display control device 20. The second microphone 40b is arranged on the second user U2 side of the transparent screen 5. For example, the second microphone 40b may be arranged on a desk in front of the second user U2. The second microphone 40b mainly converts the voice uttered by the second user U2 into a voice signal and outputs it to the display control device 20. The number of microphones constituting the voice acquisition unit is not limited to two. The display system 1 may be configured to have one microphone arranged on the first user U1 side in order to display only the speech of the first user U1 as text. The display system 1 may also be configured to have three or more microphones arranged across the transparent screen 5.

[0023] The display control device 20 transmits the audio signals acquired from the first microphone 40a and the second microphone 40b to the first server 61. In order to distinguish between the audio of the first user U1 and the audio of the second user U2, the display control device 20 may transmit the audio signal from the first microphone 40a and the audio signal from the second microphone 40b separately to the first server 61.

[0024] The display control device 20 acquires character information obtained by converting a voice signal through voice recognition processing in the first server 61. The display control device 20 can process the acquired character information. The display control device 20 can acquire or generate image information based on the character information. The display control device 20 transmits an image signal based on the acquired character information to the projector 30, which projects and displays a character string corresponding to the character information on the transparent screen 5.

[0025] The display control device 20 can also receive operations from the first user U1 via the operation unit 25 and execute processing in accordance with the operations from the first user U1. Furthermore, the display control device 20 can compile character information acquired as a result of voice recognition and transmit it to the first server 61, acquire connection information for the second user U2 to connect to the first server, and display it on the projector 30.

[0026] For example, a general-purpose information device such as a mobile phone (smartphone) or a personal computer, or a dedicated information device, can be used as the display control device 20. The display control device 20 includes a communication unit 21, a control unit 22, a storage unit 23, and an input unit 24.

[0027] The communication unit 21 communicates with devices external to the display control device 20 via wireless or wired communication means. The external devices include the operation unit 25, the first microphone 40a, the second microphone 40b, the projector 30, and the first server 61. The communication unit 21 may support communication using various communication methods, such as a serial communication standard, a wired LAN (local area network) standard, a wireless LAN standard such as Wi-Fi, and mobile communication standards such as 4G (4th Generation) and 5G (5th Generation).

[0028] The control unit 22 includes one or more processors. Processors include general-purpose processors that load specific programs to execute specific functions, and dedicated processors specialized for specific processing. Dedicated processors include application-specific integrated circuits (ASICs). Processors include programmable logic devices (PLDs). PLDs include field-programmable gate arrays (FPGAs). The control unit 22 may be either a system-on-a-chip (SoC) or a system in a package (SiP) in which one or more processors work together.

[0029] The control unit 22 controls the entire display control device 20 and executes various processes as the display control device 20. The control unit 22 is configured to transmit, for example, audio signals acquired from the first microphone 40a and the second microphone 40b to the first server 61 via the communication unit 21. The control unit 22 is configured to acquire, via the communication unit 21, character information converted from the audio signals by the first server 61. The control unit 22 causes the projector 30 to display a character string based on the acquired character information.

[0030] The control unit 22 executes a display process to display the character information acquired from the first server 61 on the projector 30. The control unit 22 can control the display position and display mode of the character string on the transparent screen 5.

[0031] The control unit 22 can cause the projector 30 to project characters included in the character information with the characters flipped left to right. In the present application, a state in which the displayed characters appear as "normal characters" to the first user U1 is considered to be a normal projection state. In this case, the characters viewed by the second user U2 are "mirror characters." "Normal characters" are characters displayed in a normal manner. "Mirror characters" are characters in which the left and right sides of normal characters are flipped. Characters projected from the projector 30 with the characters flipped left to right appear as "mirror characters" to the first user U1 and as "normal characters" to the second user U2.

[0032] The control unit 22 may be configured to highlight a predetermined word to be emphasized in the display process when the text information includes the predetermined word to be emphasized. Hereinafter, the word to be emphasized will be referred to as an emphasized word. When the text information includes an emphasized word, the control unit 22 can change the color of the character string corresponding to the word to, for example, a conspicuous color. Keywords that are important in a conversation are stored in advance in the storage unit 23 as emphasized words.

[0033] When the text information includes a word to be illustrated, the control unit 22 can display a still image or a moving image related to the illustrated object through an illustration process. Hereinafter, the word to be illustrated will be referred to as an illustrated word. An illustrated word is a specific word associated with a still image and / or a moving image for explanation. Data of the still image and the moving image corresponding to the illustrated word is stored in the memory unit 23 as illustration display data.

[0034] The storage unit 23 may include at least one of a semiconductor storage device, a magnetic storage device, and an optical storage device. The semiconductor storage device may include a volatile memory such as a dynamic random access memory (DRAM) and a static random access memory (SRAM), and a nonvolatile memory such as a read only memory (ROM) and a flash memory. The semiconductor storage device includes a solid state drive (SSD) that uses a flash memory. The magnetic storage device includes a magnetic tape, a floppy disk (registered trademark), a hard disk, etc. The optical storage device includes, for example, a compact disc (CD), a digital versatile disc (DVD), and a Blu-ray (registered trademark).

[0035] The storage unit 23 is configured to store programs executed by the control unit 22, information necessary for the processing executed by the control unit 22, and information obtained as a result of the execution by the control unit 22. The storage unit 23 may store emphasis words used when emphasizing text information received from the first server 61, as well as graphic words and graphic display data used when displaying graphic information based on the text information. If permission is given by the second user, the storage unit 23 may store text information generated by voice recognition processing from voice information sequentially received from the first server 61 as a conversation record.

[0036] The input unit 24 accepts operations of the first user U1 on the display control device 20. The input unit 24 includes, for example, a touch panel, a keyboard, a mouse, and a pen input device. The input unit 24 may be built into the display control device 20. The input unit 24 may also be an external device attached to the display control device 20.

[0037] The operation unit 25 is externally attached to the display control device 20 and receives operations from the first user U1. A signal from the operation unit 25 is input to the display control device 20 via the input unit 24. The operation unit 25 can be considered to be part of the input unit 24. As an example, the operation unit 25 may include one or more switches Sw1 to Sw3. The switches Sw1 to Sw3 are, for example, push button switches. The first user U1 can instruct the display control device 20 to perform various operations by operating the switches Sw1 to Sw3.

[0038] The projector 30 is disposed on the side of the first user U1, and projects images for character strings and illustrations onto the transparent screen 5 based on image signals received from the display control device 20.

[0039] The first server 61 is a server that can be accessed via a network 60 such as the Internet. The first server 61 of this embodiment is a voice recognition server that provides voice recognition services. When the first server 61 receives a voice signal, it performs voice recognition processing on the received voice signal. The first server 61 transmits text information obtained as a result of the voice recognition to the sender of the voice signal. The first server 61 can further acquire the text information converted by the voice recognition from the display control device 20 as a conversation record and display the acquired conversation record on the user terminal 50 of the second user. The first server 50 can also transmit the conversation record to the second server 62 to acquire a summarized conversation record and transmit it to the user terminal 50. Note that the voice recognition processing of the first server 61 and the processing of acquiring and transmitting the conversation record may each be performed by separate servers.

[0040] The second server 62 is a cloud server available via the network 60. The second server 62 provides a function for summarizing text using technologies such as natural language processing and AI (Artificial Intelligence).

[0041] (Example of display system) Next, an example of an image displayed on the transparent screen 5 by the projector 30 based on an image signal from the display control device 20 will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is an example of a view of the transparent screen 5 seen from the first user U1 side. Fig. 5 is an example of a view of the transparent screen 5 seen from the second user U2 side.

[0042] 4 and 5, the transparent screen 5 is divided into a first display area A1, a second display area A2, and a third display area A3. The first display area A1 is an area for displaying text information generated by speech recognition of speech uttered by the first user U1 and the second user U2 while scrolling sequentially. In the first display area A1, the text information may be projected in a horizontally inverted manner so that it appears normal when viewed from the second user U2.

[0043] The second display area A2 is an area where still images and / or moving images are displayed when a graphical display is performed. The second display area A2 is also called a graphical display area. The second display area A2 is displayed so as to be oriented correctly when viewed from the second user U2.

[0044] The third display area A3 is an area for displaying the most recent speech content. The third display area A3 is used by the first user U1 to confirm that the speech he or she uttered has been correctly recognized. Therefore, in the third display area A3, text information is displayed in correct characters as seen by the first user U1.

[0045] The diagrams shown in Figures 4 and 5 are merely examples of the arrangement of the first display area A1, the second display area A2, and the third display area A3. The positions, sizes, shapes, etc. of the first display area A1, the second display area A2, and the third display area A3 may be set as appropriate. Furthermore, it is not essential to provide the second display area A2 and the third display area A3. For example, the second display area A2 may be provided only when there is an illustration to be displayed.

[0046] (Display control method by display control device) A display control method executed by the control unit 22 of the display control device 20 will be described below with reference to the flow chart of FIG. 6. The display control device 20 may be configured to read and implement a display control program recorded on a non-transitory computer-readable medium to perform the processing performed by the control unit 22 described below. Non-transitory computer-readable media include, but are not limited to, magnetic storage media, optical storage media, magneto-optical storage media, and semiconductor storage media. Magnetic storage media include magnetic disks, hard disks, and magnetic tapes. Optical storage media include optical discs such as CDs (Compact Discs), DVDs, and Blu-ray (registered trademark) Discs. Semiconductor storage media include ROMs (Read Only Memory), EEPROMs (Electrically Erasable Programmable Read-Only Memory), flash memories, etc.

[0047] First, when the first user U1 starts the display system 1, the control unit 22 starts an audio display process of the audio acquired from the first microphone 40a and the second microphone 40b via the communication unit 21 (step S101). The display system 1 may be started by turning on / off a switch provided in the display control device 20, or by turning on / off the first microphone 40a and the second microphone 40b, etc.

[0048] The audio display process is executed according to the flowchart of Fig. 7. The control unit 22 continues to execute the audio display process of Fig. 7 until an instruction to stop or end the process is received. The processes from step S102 onwards in the flowchart of Fig. 6 may be executed in parallel with the audio display process of Fig. 7 while the audio display process of Fig. 7 is being executed. The audio display process of Fig. 7 will be described below.

[0049] The control unit 22 acquires an audio signal from the first microphone 40a and / or the second microphone 40b via the communication unit 21 (step S201).

[0050] The control unit 22 transmits the audio signal acquired from the first microphone 40a and / or the second microphone 40b to the first server 61 via the communication unit 21 (step S202).

[0051] The control unit 22 acquires, via the communication unit 21, character information converted from the voice signal by the voice recognition process in the first server 61 (step S203).

[0052] If necessary, the control unit 22 performs highlighting processing, illustration processing, etc. on the character information acquired in step S203. The control unit 22 transmits an image signal for display to the projector 30 via the communication unit 21, and displays the character information as a string of characters on the transparent screen 5 (step S204).

[0053] If the control unit 22 is set to record the conversation between the first user U1 and the second user U2 (step S205: Yes), the control unit 22 stores character information corresponding to the character string displayed on the transparent screen 5 in the storage unit 23 (step S206). If the control unit 22 is not set to store the conversation (step S205: No), the control unit 22 does not store the character information in the storage unit 23.

[0054] The control unit 22 repeats the processes of steps S201 to S206 unless an instruction to stop or end is received (step S207: No).

[0055] Returning to the explanation of the flowchart in Fig. 6, after the audio display process is started, the control unit 22 waits for an operation to start recording a conversation (step S102). When the first user U1 wishes to keep a conversation record, the first user U1 performs a first operation on the operation unit 25. The first operation is executed by operating the switch Sw1 of the operation unit 25, for example.

[0056] When the first user U1 performs an operation to start conversation recording (step S102: Yes), the control unit 22 causes the projector 30 to display a conversation recording permission confirmation screen on the transparent screen 5 (step S103). The conversation recording permission confirmation screen includes, for example, content that explains how personal information is handled and requests the second user U2 to give permission to save the conversation record.

[0057] If the second user U2 agrees to the saving of the conversation record, the first user U1 performs a third operation on the operation unit 25, which is an operation to permit the conversation record. The third operation is executed, for example, by operating the switch Sw3 of the operation unit 25. When the operation to permit the conversation record is received (step S104: Yes), the control unit 22 starts saving the conversation record (step S105). When the saving of the conversation record is started in step S105, the state becomes conversation recording in step S205 of the flowchart in FIG. 7 (step S205: Yes), and the character information acquired from the first server 61 thereafter is recorded in the storage unit 23 (step S206).

[0058] If there is no operation to start conversation recording in step S102 (step S102: No), and if there is no operation to permit conversation recording in step S104 (step S104: No), the control unit 22 continues to wait for an operation to start conversation recording.

[0059] If permission to record the conversation has been obtained in advance from the second user U2, steps S103 and S104 may be omitted. In this case, the conversation recording in step S105 is started when the first user U1 performs an operation to start the conversation recording (step S102: Yes).

[0060] When conversation recording starts in step S105, the control unit 22 displays the fact that the conversation is being recorded in part of the image projected onto the transparent screen 5 by the projector 30 (step S106). This allows the second user U2 to constantly confirm that the conversation is being recorded during the conversation.

[0061] The control unit 22 waits for an instruction from the first user U1 to end the conversation recording (step S107). When the first user U1 wishes to end the saving of the conversation recording, the first user U1 performs a second operation on the operation unit 25. The second operation is executed, for example, by operating the switch Sw2 of the operation unit 25. The control unit 22 continues to display the message indicating that the conversation is being recorded in step S106 unless the first user U1 performs the second operation (step S107: No).

[0062] The first user U1 may perform the second operation when the conversation with the second user U2 ends or ends. When the first user U1 performs the second operation, the control unit 22 determines that an instruction to end the conversation recording has been issued (step S107: Yes), and ends the conversation record, i.e., the process of storing the character information in the storage unit 23 (step S108).

[0063] After completing the conversation recording, the control unit 22 transmits all conversation records stored in the storage unit 23 to the first server 61 (step S109). The control unit 22 transmits the conversation records recorded between the third operation and the second operation of the first user U1 to the first server 61. If the operations of steps S103 and S104 are not performed, the control unit 22 may transmit the conversation records recorded between the first operation and the second operation of the first user U1 to the first server 61. Once the conversation records are transmitted to the first server 61, the first user U1 can check the contents of the conversation records transmitted to the first server 61 by, for example, accessing the website of the first server 61 using any information terminal. The first user U1 may use the display control device 20 as an information terminal for checking the information of the first server 61.

[0064] After step S109, the control unit 22 waits for an instruction to present the conversation record to the second user U2 (step S110). The first user U1 can operate the information terminal to select whether or not to present the conversation record to the second user U2 for the second user U2's confirmation. When the first user U1 instructs the second user U2 via the information terminal to present the conversation content to the second user U2, the first server 61 transmits, to the display control device 20, information indicating that the conversation record will be presented to the second user U2 and connection information for connecting to the first server 61.

[0065] When the control unit 22 receives an instruction to present the conversation record to the second user U2 (step S110: Yes), the control unit 22 causes the projector 30 to display on the transparent screen 5 a message indicating that the conversation record can be viewed. The control unit 22 also causes the projector 30 to display on the transparent screen 5 a two-dimensional code and one-time password, which are connection information for connecting to the first server 61, along with this message (step S111). The two-dimensional code is, for example, a QR code (registered trademark). As an example, the control unit 22 causes the projector 30 to display on the transparent screen 5 an image as shown in FIG. 8. For example, the two-dimensional code corresponds to the address of a website for viewing the conversation record provided by the first server 61, and the one-time password is the password for accessing this website. The one-time password may be a number or a character string of any number of digits. The one-time password may include two or more of numbers, letters, and symbols. The second user U2 can view the conversation record by reading this connection information into any user terminal 50.

[0066] While displaying the connection information, the control unit 22 does not need to acquire audio using the first microphone 40a and the second microphone 40b and / or transmit audio signals to the first server 61. This is to avoid confusion in the display content caused by displaying the connection information and displaying based on audio signals simultaneously.

[0067] In step S110, when the control unit 22 receives an instruction indicating that the conversation record should not be presented to the second user U2 (step S110: No), the control unit 22 does not display the connection information.

[0068] The first user U1 may thereafter select whether to continue displaying the connection information on the transparent screen 5 (step S112). If the first user U1 selects to continue displaying the connection information on the transparent screen 5 (step S112: Yes), the connection information continues to be displayed on the transparent screen 5 until the dialogue between the first user U1 and the second user U2 ends (step S113). If the first user U1 selects not to continue displaying the connection information (step S112: No), the control unit 22 may erase the connection information displayed on the transparent screen 5 by the projector 30 after a predetermined time has elapsed.

[0069] When the first user U1 switches off the display control device 20 to instruct the control unit 22 to end the processing, the control unit 22 ends all processing including the audio display processing.

[0070] (Check the conversation record) Next, the process executed by the first server 61 when the second user U2 checks the conversation record using the connection information will be described with reference to the flowchart in Fig. 9. The flowchart in Fig. 9 is executed by the control unit of the first server 61. The control unit of the first server 61 is configured to include one or more processors.

[0071] First, the second user U2 accesses a dedicated site for displaying conversation records on the first server 61 by reading a two-dimensional code using the user terminal 50. The user terminal 50 is, for example, a smartphone or a portable information terminal owned by the second user U2. When the two-dimensional code is read using the user terminal 50, a login screen is displayed on the display of the user terminal 50 (step S301).

[0072] The second user U2 enters the one-time password acquired in step S111 into the displayed login screen. If the correct password is entered (step S302: Yes), the first server 61 proceeds to the next step, S303. If the entered password is incorrect (step S302: No), the first server 61 displays an error message and returns to displaying the login screen (step S301).

[0073] In step S303, the first server 61 accepts the log-in. The first server 61 displays the entire text of the conversation record on the display of the user terminal 50 (step S304).

[0074] The entire conversation record is long and takes time to read, and it may be difficult to grasp the gist of the conversation. Therefore, if a summary of the conversation record can be presented, it will contribute to improving convenience for the second user U2. The first server 61 waits for an instruction to summarize the conversation record from the second user U2 (step S305). The first server 61 can display a button or the like on the user terminal 50 to instruct the second user U2 to summarize the conversation record.

[0075] When the first server 61 receives an instruction to summarize a conversation record from the user terminal 50 (S305: Yes), it checks whether the first server 61 has summary information that previously summarized the conversation record (step S306). If there is no past summary information (step S306: No), the first server 61 summarizes the entire conversation record (step S307).

[0076] The first server 61 may transmit the entire text of the conversation record to the second server 62 to summarize the conversation record. The second server 62 generates a summary sentence from the entire text of the conversation record using technology such as generation AI. The summary sentence is also called summary information. The first server 61 obtains the summary information of the conversation record from the second server 62. Note that the first server 61 may itself have a text summarization function and generate summary information from the entire text of the conversation record. Once summarized, the summary information is stored in the first server 61 for a certain period of time.

[0077] In step S306, if there is previously acquired or generated summary information (step S306: Yes), the first server 61 uses the previous summary information without performing a new summarization process. This makes it possible to effectively utilize the summary information that has already been performed and to prevent waste caused by performing duplicate summarization processes.

[0078] The first server 61 causes the user terminal 50 to display the newly acquired or generated summary information or the past summary information (step S308).

[0079] On the other hand, if there is no instruction to summarize the conversation record in step S305 (step S305: No), the first server 61 proceeds to step S309 without displaying the summary information.

[0080] In step S309, the first server 61 waits for a download instruction from the user terminal 50 (step S309). At this time, the first server 61 may cause the user terminal 50 to display a button for instructing the user terminal 50 to download.

[0081] When the first server 61 receives a download instruction from the user terminal 50 (step S309: Yes), it generates a file by converting the conversation record and / or summary information into a PDF (Portable Document Format) (step S310). The format of the download file is not limited to PDF and may be any format. The second user U2 may be able to select, from the user terminal 50, whether to download the conversation record or the summary information.

[0082] The first server 61 transmits the file generated in step S310 to the user terminal 50 (step S311). In this way, the second user U2 can keep a record of the content of the conversation with the first user U1 and check it later.

[0083] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art would easily be able to make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included within the scope of the present disclosure. For example, the functions included in each component can be rearranged so as not to cause logical inconsistencies, and multiple components can be combined or divided into one.

[0084] In this disclosure, the terms "first" and "second" are identifiers for distinguishing the configuration. In this disclosure, the configurations distinguished by terms such as "first" and "second" can have their numbers swapped. For example, the first microphone 40a can swap the identifiers "first" and "second" with the second microphone 40b. The identifier swapping is performed simultaneously. The configurations remain distinguished even after the identifier swapping. Identifiers may be deleted. A configuration from which an identifier has been deleted is distinguished by a symbol. The identifiers "first" and "second" in this disclosure should not be used solely to interpret the order of the configurations or to justify the existence of an identifier with a smaller number.

[0085] The components and functional blocks included in each embodiment of the present disclosure may be arranged in the same hardware or different hardware as appropriate. The first microphone 40a in the embodiment of the present disclosure may be built into the display control device 20. The display control device 20 and the projector 30 may be integrally configured with the same hardware. Some or all of the communication unit 21, control unit 22, storage unit 23, and input unit 24 of the display control device 20 in the embodiment of the present disclosure may be included in the projector 30.

[0086] The display unit of the present disclosure is not limited to a configuration including the transparent screen 5 and the projector 30. The display unit may include, for example, a transparent organic light-emitting diode (OLED) display in which light-emitting elements are arranged within a transparent substrate.

[0087] In the above embodiment, the speech recognition process and the summarization process are performed by a cloud server located at a remote location, but the present invention is not limited to this. The speech recognition process and the summarization process may be performed by separate servers rather than the same server located at a remote location. At least one of the speech recognition process and the summarization process may be performed by a server located near the display control device 20 or by the display control device 20 itself. [Explanation of symbols]

[0088] 1 Display System 5 Transparent screen (display) 10 Base material 20 Display control device 21 Communications Department 22 Control Unit 23 Memory section 24 Input section 25 Control section 30 Projector (display unit) 40a First microphone (voice acquisition unit) 40b Second microphone (voice acquisition unit) 50 User Terminals 60 Network 61 First Server (Server) 62 Second Server (Server) U1 First User U2 Second user A1 1st display area A2 2nd display area A3 3rd display area

Claims

1. a display unit disposed between a plurality of users when in use; a voice acquisition unit disposed at a position capable of acquiring voices of at least one of the plurality of users; a display control device that receives an input of a voice signal from the voice acquisition unit and displays character information converted from the voice signal on the display unit; an operation unit that accepts an operation on the display control device; Server and A display system comprising: The display control device transmits the character information displayed on the display unit from when the first operation is accepted until when the second operation is accepted to the server based on a first operation and a second operation on the operation unit, and obtains connection information for connecting to the server and displays the connection information on the display unit.

2. The display system of claim 1, wherein the display control device displays a notification regarding the handling of personal information based on the first operation on the operation unit, and transmits the character information to the server based on a third operation on the operation unit in response to the notification and the second operation.

3. The display system according to claim 1 , wherein the connection information includes a two-dimensional code and a password for connecting to the server.

4. The display system according to claim 1 , wherein the display control device does not cause the audio acquisition unit to acquire audio while the connection information is being displayed on the display unit.

5. The display system described in claim 1, wherein the server displays the character information on a user terminal connected to the server using the connection information, and when a summary instruction is received from the user terminal, generates or obtains summary information that summarizes the character information and displays the summary information on the user terminal.

6. The display system described in claim 5, wherein the server determines whether or not summary information has been generated or acquired in the past, and if the summary information has been generated or acquired in the past, when the server receives the summary instruction from the user terminal, it does not generate or acquire the summary information, but instead displays the summary information that has been generated or acquired in the past on the user terminal.

7. a display unit disposed between a plurality of users when in use, a voice acquisition unit disposed in a position capable of acquiring voices of at least one of the plurality of users, and a communication unit configured to be able to communicate with a server; an input unit that receives input from an operation unit; a control unit that acquires a voice signal from the voice acquisition unit via the communication unit and displays character information converted from the voice signal on the display unit; The control unit transmits the character information displayed on the display unit from when the first operation is accepted until when the second operation is accepted to the server based on a first operation and a second operation on the operation unit, and obtains connection information for connecting to the server and displays the connection information on the display unit.

8. A control program for a display control device including a display unit arranged between a plurality of users when in use, a voice acquisition unit arranged in a position capable of acquiring voices of at least one of the plurality of users, a communication unit configured to be able to communicate with a server, and an input unit for receiving input from an operation unit, acquiring a voice signal from the voice acquisition unit via the communication unit; displaying character information into which the voice signal is converted on the display unit; transmitting the character information displayed on the display unit from when the first operation is accepted until when the second operation is accepted to the server based on a first operation and a second operation on the operation unit, and acquiring connection information for connecting to the server and displaying the connection information on the display unit; a display control program that causes a control unit of the display control device to execute processing including the steps of:

Images