Display system, display control device, and display control program

Abstract

This display system comprises a display unit, a speech acquisition unit, a display control device, an operation unit, and a server. The display unit is disposed between a plurality of users while in use. The speech acquisition unit is disposed at a position at which the speech of any of one or more people among the plurality of users can be acquired. The display control device receives an input of speech signals from the speech acquisition unit, and causes the display unit to display character information obtained by converting the speech signals. The operation unit receives operations for the display control device. The display control device: on the basis of a first operation and a second operation for the operation unit, transmits, to the server, the character information displayed on the display unit from the reception of the first operation to the reception of the second operation and acquires connection information for connecting to the server; and causes the display unit to display the connection information.

Description

Display system, display control device, and display control program CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent Application No. 2024-30588, filed February 29, 2024, the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to a display system, a display control device, and a display control program.

[0003] Conventionally, in situations where users such as a counter clerk and a visitor are conversing across a partition, a technology has been proposed in which the speech uttered by at least one party is converted into text information and displayed on a transparent display that is visible to the other speaker (see, for example, Patent Document 1).

[0004] Japanese Patent Application Laid-Open No. 2023-124615

[0005] In one embodiment, (1) a display system includes a display unit that is positioned between multiple users when in use, an audio acquisition unit that is positioned in a position where it can acquire the audio of one or more of the multiple users, a display control device that receives an audio signal from the audio acquisition unit and displays text information converted from the audio signal on the display unit, an operation unit that accepts operations on the display control device, and a server, wherein the display control device transmits the text information displayed on the display unit from when the first operation is accepted to when the second operation is accepted based on a first operation and a second operation on the operation unit, and acquires connection information for connecting to the server and displays the connection information on the display unit.

[0006] (2) In the display system of (1) above, the display control device can display a notification regarding the handling of personal information based on the first operation on the operation unit, and can transmit the character information to the server based on the third operation on the operation unit in response to the notification and the second operation.

[0007] (3) In the display system of (1) or (2) above, the connection information may include a two-dimensional code and a password for connecting to the server.

[0008] (4) In any of the display systems (1) to (3) above, the display control device may not require the audio acquisition unit to acquire audio while the connection information is being displayed on the display unit.

[0009] (5) In any of the display systems (1) to (4) above, the server can use the connection information to display the character information on a user terminal connected to the server, and when a summary instruction is received from the user terminal, generate or obtain summary information that summarizes the character information and display the summary information on the user terminal.

[0010] (6) In the display system of (5) above, the server determines whether or not there is summary information that has been generated or acquired in the past, and if there is summary information that has been generated or acquired in the past, when it receives the summary instruction from the user terminal, it does not generate or acquire the summary information, but can display the summary information that has been generated or acquired in the past on the user terminal.

[0011] In one embodiment, (7) the display control device includes a display unit that is positioned between multiple users when in use, a voice acquisition unit that is positioned in a position where it can acquire the voice of one or more of the multiple users, a communication unit that is configured to be able to communicate with a server, an input unit that receives input from an operation unit, and a control unit that acquires a voice signal from the voice acquisition unit via the communication unit and displays character information into which the voice signal is converted on the display unit, and the control unit transmits the character information displayed on the display unit from the time the first operation is accepted until the time the second operation is accepted to the server based on a first operation and a second operation on the operation unit, and acquires connection information for connecting to the server and displays the connection information on the display unit.

[0012] In one embodiment, (8) the display control program is a control program for a display control device that includes a display unit that is placed between multiple users when in use, a voice acquisition unit that is placed in a position where it can acquire the voice of one or more of the multiple users, a communication unit that is configured to be able to communicate with a server, and an input unit that receives input from an operation unit, and causes a control unit of the display control device to execute processing that includes acquiring a voice signal from the voice acquisition unit via the communication unit, displaying character information into which the voice signal is converted on the display unit, and transmitting the character information displayed on the display unit from the time the first operation is accepted to the time the second operation is accepted based on a first operation and a second operation on the operation unit to the server, and acquiring connection information for connecting to the server and displaying the connection information on the display unit.

[0013] 7 is a diagram illustrating an example of a usage scene of a display system according to an embodiment. FIG. 1 is a diagram illustrating the display system of FIG. 1 as viewed from above. FIG. 2 is a block diagram illustrating a schematic configuration of the display system of FIG. 1. FIG. 3 is a diagram illustrating an example of a display on a transparent screen as viewed from a first user's side. FIG. 4 is a diagram illustrating an example of a display on a transparent screen as viewed from a second user's side. FIG. 5 is a flowchart illustrating an example of processing executed by a control unit of a display control device. FIG. 6 is a flowchart illustrating an audio display process executed in FIG. 7. FIG. 8 is a diagram illustrating an example of connection information for checking a conversation record displayed to a second user. FIG. 9 is a flowchart of processing executed by a first server when a second user checks the content of the conversation.

[0014] With conventional technology, it was sometimes not possible to record the conversation between the counter staff and the visitor and check it later.

[0015] The display system, display control device, and display control program disclosed herein enable a user to check a conversation record in a display system that converts a spoken dialogue into text and displays it.

[0016] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. The drawings used in the following description are schematic. The dimensions and ratios in the drawings do not necessarily correspond to the actual dimensions and ratios.

[0017] (Configuration of the Display System) As shown in Figures 1 and 2, the display system 1 according to one embodiment is a system that supports dialogue between a first user U1 and a second user U2. As an example, the first user U1 is an employee of a government, local government, or public institution. The second user U2 is, for example, an ordinary citizen visiting a government, local government, or public institution. This system is particularly effective when the second user U2 is hearing impaired, for example. The situations in which the display system 1 is used are not limited to those described above. The display system 1 can also be used at the counters of financial institutions, medical institutions, public transportation facilities, etc., at the business offices of private businesses, and in office conference rooms, etc.

[0018] An acrylic plate, a vinyl curtain, or the like is installed between the first user U1 and the second user U2. In recent years, acrylic plates or vinyl curtains have sometimes been installed to prevent droplets from spreading. In the display system 1, the acrylic plate, the vinyl curtain, or the like is used as a base material 10, and a transparent screen 5 is disposed on the base material 10. The projector 30 projects the speech content of the first user U1 and / or the second user U2 onto the transparent screen 5. This allows the first user U1 and / or the second user U2, who are conversing across the transparent screen 5, to visually confirm the speech content even if they cannot hear what the other user is saying.

[0019] The transparent screen 5 can be placed anywhere on the base material 10. The transparent screen 5 may be placed in a position that does not prevent the first user U1 and the second user U2 from seeing each other's faces. For example, the transparent screen 5 may be placed in a position where the first user U1 and the second user U2 lower their lines of sight downward from the horizontal direction.

[0020] As shown in FIGS. 1 to 3 , the display system 1 includes a transparent screen 5, a display control device 20, an operation unit 25, a projector 30, a first microphone 40a, a second microphone 40b, and a first server 61, which is a cloud server located in a remote location and performs voice recognition processing. The transparent screen 5 and the projector 30 constitute a display unit. Hereinafter, the transparent screen 5 and the projector 30 may be collectively referred to as the display unit. The first microphone 40a and the second microphone 40b constitute a voice acquisition unit. The operation unit 25, the first microphone 40a, and the second microphone 40b are each connected to the display control device 20 via a wired or wireless connection so as to be able to communicate with each other. The display control device 20 and the projector 30 are connected via a wired or wireless connection so as to be able to communicate with each other. The display control device 20 is configured to be able to communicate with the first server 61 via a network 60. The display system 1 may further include a second server 62, which is a cloud server having a text summarization function. The display control device 20 and the first server 61 are configured to be able to communicate with a second server 62 via a network 60 .

[0021] The transparent screen 5 can be a film-, sheet-, or plate-like member for projector projection that can be attached to the base material 10. The transparent screen 5 diffuses a portion of the light incident from the projector 30 to the incident side and the exit side. The first user U1 and the second user U2 can recognize the image projected from the projector 30 when the light diffused by the transparent screen 5 enters their field of vision. The shape of the transparent screen 5 can be, for example, rectangular, but is not limited to this. The transparent screen 5 can be in various shapes.

[0022] The first microphone 40a is arranged on the first user U1 side of the transparent screen 5. For example, the first microphone 40a may be arranged on a desk in front of the first user U1. The first microphone 40a mainly converts the voice uttered by the first user U1 into an audio signal and outputs it to the display control device 20. The second microphone 40b is arranged on the second user U2 side of the transparent screen 5. For example, the second microphone 40b may be arranged on a desk in front of the second user U2. The second microphone 40b mainly converts the voice uttered by the second user U2 into an audio signal and outputs it to the display control device 20. The number of microphones constituting the audio acquisition unit is not limited to two. The display system 1 may be configured to have one microphone arranged on the first user U1 side in order to display only the speech of the first user U1 as text. The display system 1 may also be configured to have three or more microphones arranged across the transparent screen 5.

[0023] The display control device 20 transmits the audio signals acquired from the first microphone 40a and the second microphone 40b to the first server 61. In order to distinguish between the audio of the first user U1 and the audio of the second user U2, the display control device 20 may transmit the audio signal from the first microphone 40a and the audio signal from the second microphone 40b separately to the first server 61.

[0024] The display control device 20 acquires character information obtained by converting a voice signal through voice recognition processing in the first server 61. The display control device 20 can process the acquired character information. The display control device 20 can acquire or generate image information based on the character information. The display control device 20 transmits an image signal based on the acquired character information to the projector 30, which projects and displays a character string corresponding to the character information on the transparent screen 5.

[0025] The display control device 20 can also receive operations from the first user U1 via the operation unit 25 and execute processing in accordance with the operations from the first user U1. The display control device 20 can also compile character information acquired as a result of voice recognition and transmit it to the first server 61, acquire connection information for the second user U2 to connect to the first server, and display it on the projector 30.

[0026] The display control device 20 may be a general-purpose information device such as a mobile phone (smartphone) or a personal computer, or a dedicated information device. The display control device 20 includes a communication unit 21, a control unit 22, a storage unit 23, and an input unit 24.

[0027] The communication unit 21 communicates with devices external to the display control device 20 via wireless or wired communication means. The external devices include the operation unit 25, the first microphone 40a, the second microphone 40b, the projector 30, and the first server 61. The communication unit 21 may support communication using various communication methods, such as a serial communication standard, a wired local area network (LAN) standard, a wireless LAN standard such as Wi-Fi, and mobile communication standards such as 4G (4th Generation) and 5G (5th Generation).

[0028] The control unit 22 includes one or more processors. Processors include general-purpose processors that load specific programs to execute specific functions, and dedicated processors specialized for specific processing. Dedicated processors include application-specific integrated circuits (ASICs). Processors include programmable logic devices (PLDs). PLDs include field-programmable gate arrays (FPGAs). The control unit 22 may be either a system-on-a-chip (SoC) or a system in a package (SiP) in which one or more processors work together.

[0029] The control unit 22 controls the entire display control device 20 and executes various processes as the display control device 20. The control unit 22 is configured to transmit, for example, audio signals acquired from the first microphone 40a and the second microphone 40b to the first server 61 via the communication unit 21. The control unit 22 is configured to acquire, via the communication unit 21, character information converted from the audio signals by the first server 61. The control unit 22 causes the projector 30 to display a character string based on the acquired character information.

[0030] The control unit 22 executes a display process to display the character information acquired from the first server 61 on the projector 30. The control unit 22 can control the display position and display mode of the character string on the transparent screen 5.

[0031] The control unit 22 can reverse the characters included in the character information and project them on the projector 30. In the present application, a state in which the displayed characters appear as "normal characters" to the first user U1 is considered to be a normal projection state. In this case, the characters viewed by the second user U2 are "mirror characters." "Normal characters" are characters displayed in a normal manner. "Mirror characters" are characters in which the left and right sides of normal characters are reversed. Characters projected from the projector 30 with the left and right reversed appear as "mirror characters" to the first user U1 and as "normal characters" to the second user U2.

[0032] The control unit 22 may be configured to highlight a predetermined word to be emphasized in the display process when the text information includes the predetermined word to be emphasized. Hereinafter, the word to be emphasized will be referred to as an emphasized word. When the text information includes an emphasized word, the control unit 22 can change the color of the character string corresponding to the word to, for example, a conspicuous color. Keywords that are important in a conversation are stored in advance in the storage unit 23 as emphasized words.

[0033] When the text information includes a word to be illustrated, the control unit 22 can display a still image or a moving image related to the illustrated object through an illustration process. Hereinafter, the word to be illustrated will be referred to as an illustrated word. An illustrated word is a specific word associated with a still image and / or a moving image for explanation. Data on the still image and the moving image corresponding to the illustrated word is stored in the storage unit 23 as illustration display data.

[0034] The storage unit 23 may include at least one of a semiconductor storage device, a magnetic storage device, and an optical storage device. The semiconductor storage device may include volatile memory such as DRAM (Dynamic Random Access Memory) and SRAM (Static Random Access Memory), and non-volatile memory such as ROM (Read Only Memory) and flash memory. The semiconductor storage device includes an SSD (Solid State Drive) that uses flash memory. The magnetic storage device includes a magnetic tape, a floppy disk (registered trademark), a hard disk, etc. The optical storage device includes, for example, a CD (Compact Disc), a DVD (Digital Versatile Disc), and a Blu-ray (registered trademark).

[0035] The storage unit 23 is configured to store programs executed by the control unit 22, information necessary for the processing executed by the control unit 22, and information obtained as a result of the execution by the control unit 22. The storage unit 23 may store emphasis words used when emphasizing text information received from the first server 61, as well as graphic words and graphic display data used when displaying graphic information based on the text information. If permission is given by the second user, the storage unit 23 may store, as a conversation record, text information generated by voice recognition processing from voice information sequentially received from the first server 61.

[0036] The input unit 24 accepts operations by the first user U1 on the display control device 20. The input unit 24 includes, for example, a touch panel, a keyboard, a mouse, and a pen input device. The input unit 24 may be built into the display control device 20. The input unit 24 may also be an external device attached to the display control device 20.

[0037] The operation unit 25 is externally attached to the display control device 20 and accepts operations from the first user U1. A signal from the operation unit 25 is input to the display control device 20 via the input unit 24. The operation unit 25 can be considered to be part of the input unit 24. As an example, the operation unit 25 may include one or more switches Sw1 to Sw3. The switches Sw1 to Sw3 are, for example, push button switches. The first user U1 can instruct the display control device 20 to perform various operations by operating the switches Sw1 to Sw3.

[0038] The projector 30 is disposed on the side of the first user U1, and projects images of characters and illustrations onto the transparent screen 5 based on the image signal received from the display control device 20.

[0039] The first server 61 is a server that can be accessed via a network 60 such as the Internet. In this embodiment, the first server 61 is a voice recognition server that provides a voice recognition service. When the first server 61 receives a voice signal, it performs voice recognition processing on the received voice signal. The first server 61 transmits text information obtained as a result of the voice recognition to the sender of the voice signal. The first server 61 can further acquire the text information converted by the voice recognition from the display control device 20 as a conversation record and display the acquired conversation record on the user terminal 50 of the second user. The first server 50 can also transmit the conversation record to the second server 62 to acquire a summarized conversation record and transmit it to the user terminal 50. Note that the voice recognition processing of the first server 61 and the processing of acquiring and transmitting the conversation record may each be performed by separate servers.

[0040] The second server 62 is a cloud server available via the network 60. The second server 62 provides a function for summarizing text using technologies such as natural language processing and AI (Artificial Intelligence).

[0041] (Display Example of Display System) Next, an example of an image displayed on the transparent screen 5 by the projector 30 based on an image signal from the display control device 20 will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is an example of a view of the transparent screen 5 from the side of the first user U1. Fig. 5 is an example of a view of the transparent screen 5 from the side of the second user U2.

[0042] 4 and 5, the transparent screen 5 is divided into a first display area A1, a second display area A2, and a third display area A3. The first display area A1 is an area for displaying text information generated by speech recognition of speech uttered by the first user U1 and the second user U2 while scrolling sequentially. In the first display area A1, the text information may be projected in a horizontally inverted manner so that it appears in normal characters when viewed from the second user U2.

[0043] The second display area A2 is an area where still images and / or moving images are displayed when a graphical display is performed. The second display area A2 is also referred to as a graphical display area. The second display area A2 is displayed so as to be oriented correctly when viewed from the second user U2.

[0044] The third display area A3 is an area for displaying the most recent speech content. The third display area A3 is used by the first user U1 to confirm that the speech he or she uttered has been correctly recognized. Therefore, in the third display area A3, text information is displayed in correct characters as seen by the first user U1.

[0045] 4 and 5 are merely examples of the arrangement of the first display area A1, the second display area A2, and the third display area A3. The positions, sizes, shapes, etc. of the first display area A1, the second display area A2, and the third display area A3 may be set as appropriate. Furthermore, it is not essential to provide the second display area A2 and the third display area A3. For example, the second display area A2 may be provided only when there is an illustration to be displayed.

[0046] (Display Control Method by Display Control Device) A display control method executed by the control unit 22 of the display control device 20 will be described below with reference to the flow chart of FIG. 6 . The display control device 20 may be configured to read and implement a display control program recorded on a non-transitory computer-readable medium to perform the processing performed by the control unit 22 described below. Non-transitory computer-readable media include, but are not limited to, magnetic storage media, optical storage media, magneto-optical storage media, and semiconductor storage media. Magnetic storage media include magnetic disks, hard disks, and magnetic tapes. Optical storage media include optical disks such as CDs (Compact Discs), DVDs, and Blu-ray (registered trademark) Discs. Semiconductor storage media include read-only memories (ROMs), electrically erasable programmable read-only memories (EEPROMs), flash memories, etc.

[0047] First, when the first user U1 starts the display system 1, the control unit 22 starts an audio display process of the audio acquired from the first microphone 40a and the second microphone 40b via the communication unit 21 (step S101). The display system 1 may be started by turning on / off a switch provided in the display control device 20, or by turning on / off the first microphone 40a and the second microphone 40b.

[0048] The audio display process is executed according to the flowchart of Fig. 7. The control unit 22 continues to execute the audio display process of Fig. 7 until an instruction to stop or end the process is received. The processes from step S102 onward in the flowchart of Fig. 6 may be executed in parallel with the audio display process of Fig. 7 while the audio display process of Fig. 7 is being executed. The audio display process of Fig. 7 will be described below.

[0049] The control unit 22 acquires an audio signal from the first microphone 40a and / or the second microphone 40b via the communication unit 21 (step S201).

[0050] The control unit 22 transmits the audio signal acquired from the first microphone 40a and / or the second microphone 40b to the first server 61 via the communication unit 21 (step S202).

[0051] The control unit 22 acquires, via the communication unit 21, character information converted from the voice signal by the voice recognition process in the first server 61 (step S203).

[0052] If necessary, the control unit 22 performs highlighting and illustration processing on the character information acquired in step S203. The control unit 22 transmits an image signal for display to the projector 30 via the communication unit 21, and displays the character information as a string of characters on the transparent screen 5 (step S204).

[0053] If the control unit 22 is set to record the conversation between the first user U1 and the second user U2 (step S205: Yes), the control unit 22 stores character information corresponding to the character string displayed on the transparent screen 5 in the storage unit 23 (step S206). If the control unit 22 is not set to store the conversation (step S205: No), the control unit 22 does not store the character information in the storage unit 23.

[0054] The control unit 22 repeats the processes of steps S201 to S206 unless an instruction to stop or end is received (step S207: No).

[0055] 6, after the audio display process is started, the control unit 22 waits for an operation to start recording the conversation (step S102). When the first user U1 wishes to keep a conversation record, the first user U1 performs a first operation on the operation unit 25. The first operation is performed, for example, by operating the switch Sw1 of the operation unit 25.

[0056] When the first user U1 performs an operation to start recording the conversation (step S102: Yes), the control unit 22 causes the projector 30 to display a consent confirmation screen for recording the conversation on the transparent screen 5 (step S103). The consent confirmation screen for recording the conversation includes, for example, an explanation of how personal information will be handled and a request for consent to save the conversation record from the second user U2.

[0057] If the second user U2 agrees to the saving of the conversation record, the first user U1 performs a third operation on the operation unit 25, which is an operation to permit the conversation record. The third operation is executed, for example, by operating the switch Sw3 on the operation unit 25. When the operation to permit the conversation record is received (step S104: Yes), the control unit 22 starts saving the conversation record (step S105). When the saving of the conversation record is started in step S105, the conversation record is in progress (step S205: Yes) in step S205 of the flowchart in FIG. 7, and the character information acquired from the first server 61 thereafter is recorded in the storage unit 23 (step S206).

[0058] If there is no operation to start conversation recording in step S102 (step S102: No), and if there is no operation to permit conversation recording in step S104 (step S104: No), the control unit 22 continues to wait for an operation to start conversation recording.

[0059] If permission to record the conversation has been obtained in advance from the second user U2, steps S103 and S104 may be omitted. In this case, the conversation recording is started in step S105 when the first user U1 performs an operation to start the conversation recording (step S102: Yes).

[0060] When conversation recording is started in step S105, the control unit 22 displays the fact that the conversation is being recorded in part of the image projected onto the transparent screen 5 by the projector 30 (step S106). This allows the second user U2 to constantly confirm that the conversation is being recorded during the conversation.

[0061] The control unit 22 waits for an instruction from the first user U1 to end the conversation recording (step S107). When the first user U1 wishes to end the saving of the conversation recording, the first user U1 performs a second operation on the operation unit 25. The second operation is executed, for example, by operating the switch Sw2 of the operation unit 25. The control unit 22 continues to display the message indicating that the conversation is being recorded in step S106 unless the first user U1 performs the second operation (step S107: No).

[0062] The first user U1 may perform the second operation when the conversation with the second user U2 ends or ends. When the first user U1 performs the second operation, the control unit 22 determines that an instruction to end the conversation recording has been issued (step S107: Yes), and ends the conversation recording, i.e., the process of storing the character information in the storage unit 23 (step S108).

[0063] After completing the conversation recording, the control unit 22 transmits all conversation records stored in the storage unit 23 to the first server 61 (step S109). The control unit 22 transmits the conversation records recorded between the third operation and the second operation of the first user U1 to the first server 61. If the operations of steps S103 and S104 are not performed, the control unit 22 may transmit the conversation records recorded between the first operation and the second operation of the first user U1 to the first server 61. Once the conversation records are transmitted to the first server 61, the first user U1 can check the contents of the conversation records transmitted to the first server 61 by, for example, accessing the website of the first server 61 using any information terminal. The first user U1 may use the display control device 20 as an information terminal for checking the information of the first server 61.

[0064] After step S109, the control unit 22 waits for an instruction to present the conversation record to the second user U2 (step S110). The first user U1 can operate the information terminal to select whether or not to present the conversation record to the second user U2 for the second user U2's confirmation. When the first user U1 instructs the second user U2 via the information terminal to present the conversation content to the second user U2, the first server 61 transmits to the display control device 20 information indicating that the conversation record will be presented to the second user U2 and connection information for connecting to the first server 61.

[0065] When the control unit 22 receives an instruction to present the conversation record to the second user U2 (step S110: Yes), the control unit 22 causes the projector 30 to display on the transparent screen 5 a message indicating that the conversation record can be viewed. The control unit 22 also displays this message along with a two-dimensional code and one-time password, which are connection information for connecting to the first server 61 (step S111). The two-dimensional code is, for example, a QR code (registered trademark). As an example, the control unit 22 causes the projector 30 to display an image such as that shown in FIG. 8 on the transparent screen 5. For example, the two-dimensional code corresponds to the address of a website for viewing the conversation record provided by the first server 61, and the one-time password is the password for accessing this website. The one-time password may be a number or a string of characters with any number of digits. The one-time password may include two or more of numbers, letters, and symbols. The second user U2 can view the conversation record by reading this connection information on any user terminal 50.

[0066] While displaying the connection information, the control unit 22 does not need to acquire audio using the first microphone 40a and the second microphone 40b and / or transmit audio signals to the first server 61. This is to avoid confusion in the display content caused by simultaneously displaying the connection information and displaying based on the audio signal.

[0067] In step S110, when the control unit 22 receives an instruction indicating that the conversation record should not be presented to the second user U2 (step S110: No), the control unit 22 does not display the connection information.

[0068] The first user U1 may thereafter select whether to continue displaying the connection information on the transparent screen 5 (step S112). If the first user U1 selects to continue displaying the connection information on the transparent screen 5 (step S112: Yes), the connection information continues to be displayed on the transparent screen 5 until the dialogue between the first user U1 and the second user U2 ends (step S113). If the first user U1 selects not to continue displaying the connection information (step S112: No), the control unit 22 may erase the connection information displayed on the transparent screen 5 by the projector 30 after a predetermined time has elapsed.

[0069] When the first user U1 instructs the control unit 22 to end the processing by, for example, turning off the switch of the display control device 20, the control unit 22 ends all processing including the audio display processing.

[0070] (Confirming Conversation Record) Next, the process executed by the first server 61 when the second user U2 checks the conversation record using the connection information will be described with reference to the flowchart in Fig. 9. The flowchart in Fig. 9 is executed by the control unit of the first server 61. The control unit of the first server 61 is configured to include one or more processors.

[0071] First, the second user U2 accesses a dedicated site for displaying conversation records on the first server 61 by reading a two-dimensional code using a user terminal 50. The user terminal 50 is, for example, a smartphone or a portable information terminal owned by the second user U2. When the two-dimensional code is read using the user terminal 50, a login screen is displayed on the display of the user terminal 50 (step S301).

[0072] The second user U2 enters the one-time password acquired in step S111 into the displayed login screen. If the correct password is entered (step S302: Yes), the first server 61 proceeds to the next step S303. If the entered password is incorrect (step S302: No), the first server 61 displays an error message and returns to displaying the login screen (step S301).

[0073] In step S303, the first server 61 accepts the log-in. The first server 61 displays the entire text of the conversation record on the display of the user terminal 50 (step S304).

[0074] The entire conversation record is long, takes time to read, and can be difficult to grasp the gist of. Therefore, being able to present a summary of the conversation record contributes to improving convenience for the second user U2. The first server 61 waits for an instruction to summarize the conversation record from the second user U2 (step S305). The first server 61 can display a button or the like on the user terminal 50 to instruct the second user U2 to summarize the conversation record.

[0075] When the first server 61 receives an instruction to summarize a conversation record from the user terminal 50 (Yes in S305), it checks whether summary information that previously summarized the conversation record is available in the first server 61 (Step S306). If there is no previous summary information (No in Step S306), the first server 61 summarizes the entire conversation record (Step S307).

[0076] The first server 61 may transmit the entire text of the conversation record to the second server 62 to summarize the conversation record. The second server 62 generates a summary sentence from the entire text of the conversation record using technology such as generation AI. The summary sentence is also called summary information. The first server 61 obtains the summary information of the conversation record from the second server 62. Note that the first server 61 itself may have a text summarization function and generate summary information from the entire text of the conversation record. Once summarized, the summary information is stored in the first server 61 for a certain period of time.

[0077] In step S306, if there is previously acquired or generated summary information (step S306: Yes), the first server 61 uses the previously acquired or generated summary information without performing a new summarization process. This makes it possible to effectively utilize the summary information that has already been acquired and to prevent waste caused by performing duplicate summarization processes.

[0078] The first server 61 displays the newly acquired or generated summary information or the past summary information on the user terminal 50 (step S308).

[0079] On the other hand, if there is no instruction to summarize the conversation record in step S305 (step S305: No), the first server 61 proceeds to step S309 without displaying the summary information.

[0080] In step S309, the first server 61 waits for a download instruction from the user terminal 50. At this time, the first server 61 may cause the user terminal 50 to display a button for instructing the user terminal 50 to download.

[0081] When the first server 61 receives a download instruction from the user terminal 50 (step S309: Yes), it generates a file in which the conversation record and / or summary information is converted into a portable document format (PDF) (step S310). Note that the format of the download file is not limited to PDF and may be any format. The second user U2 may be able to select, via the user terminal 50, whether to download the conversation record or the summary information.

[0082] The first server 61 transmits the file generated in step S310 to the user terminal 50 (step S311). In this way, the second user U2 can record the content of the conversation with the first user U1 and check it later.

[0083] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art would easily be able to make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included within the scope of the present disclosure. For example, the functions included in each component can be rearranged so as not to cause logical inconsistencies, and multiple components can be combined or divided into one.

[0084] In this disclosure, descriptions such as "first" and "second" are identifiers for distinguishing the configuration. In this disclosure, the configurations distinguished by descriptions such as "first" and "second" can have their numbers swapped. For example, the first microphone 40a can swap the identifiers "first" and "second" with the second microphone 40b. The identifier swapping is performed simultaneously. The configurations remain distinguished even after the identifier swapping. Identifiers may be deleted. A configuration from which an identifier has been deleted is distinguished by a symbol. The identifiers "first" and "second" used in this disclosure should not be used solely to interpret the order of the configurations or to justify the existence of an identifier with a smaller number.

[0085] The components and functional blocks included in each embodiment of the present disclosure may be arranged in the same hardware or different hardware as appropriate. The first microphone 40a in the embodiment of the present disclosure may be built into the display control device 20. The display control device 20 and the projector 30 may be integrally configured with the same hardware. Some or all of the communication unit 21, control unit 22, storage unit 23, and input unit 24 of the display control device 20 in the embodiment of the present disclosure may be included in the projector 30.

[0086] The display unit of the present disclosure is not limited to a configuration including the transparent screen 5 and the projector 30. The display unit may include, for example, a transparent organic light-emitting diode (OLED) display in which light-emitting elements are arranged within a transparent substrate.

[0087] In the above embodiment, the speech recognition process and the summarization process are performed by a cloud server located in a remote location, but the present invention is not limited to this. The speech recognition process and the summarization process may be performed by separate servers rather than the same server located in a remote location. At least one of the speech recognition process and the summarization process may be performed by a server located near the display control device 20 or by the display control device 20 itself.

[0088] REFERENCE SIGNS LIST 1 Display system 5 Transparent screen (display unit) 10 Base material 20 Display control device 21 Communication unit 22 Control unit 23 Memory unit 24 Input unit 25 Operation unit 30 Projector (display unit) 40a First microphone (audio acquisition unit) 40b Second microphone (audio acquisition unit) 50 User terminal 60 Network 61 First server (server) 62 Second server (server) U1 First user U2 Second user A1 First display area A2 Second display area A3 Third display area

Claims

1. A display system comprising: a display unit that is placed between multiple users when in use; a voice acquisition unit that is placed in a position where it can acquire the voice of one or more of the multiple users; a display control device that receives an audio signal from the voice acquisition unit and displays text information into which the audio signal is converted on the display unit; an operation unit that accepts operations for the display control device; and a server, wherein the display control device transmits to the server the text information that has been displayed on the display unit from when the first operation is accepted until when the second operation is accepted, based on a first operation and a second operation on the operation unit, and acquires connection information for connecting to the server and displays the connection information on the display unit.

2. The display system of claim 1, wherein the display control device displays a notification regarding the handling of personal information based on the first operation on the operation unit, and transmits the character information to the server based on the third operation on the operation unit in response to the notification and the second operation.

3. A display system according to claim 1 or 2, wherein the connection information includes a two-dimensional code and a password for connecting to the server.

4. A display system according to any one of claims 1 to 3, wherein the display control device does not cause the audio acquisition unit to acquire audio while the connection information is being displayed on the display unit.

5. A display system as described in any one of claims 1 to 4, wherein the server displays the character information on a user terminal connected to the server using the connection information, and when a summary instruction is received from the user terminal, generates or obtains summary information that summarizes the character information, and displays the summary information on the user terminal.

6. The display system of claim 5, wherein the server determines whether or not summary information has been generated or acquired in the past, and if such summary information has been generated or acquired in the past, when the server receives the summary instruction from the user terminal, it does not generate or acquire the summary information, but instead displays the previously generated or acquired summary information on the user terminal.

7. A display control device comprising: a display unit that is placed between multiple users when in use; a voice acquisition unit that is placed in a position where it can acquire the voice of one or more of the multiple users; and a communication unit that is configured to be able to communicate with a server; an input unit that receives input from an operation unit; and a control unit that acquires a voice signal from the voice acquisition unit via the communication unit and causes character information into which the voice signal is converted to be displayed on the display unit, wherein the control unit transmits to the server the character information that has been displayed on the display unit from when the first operation is accepted until when the second operation is accepted, based on a first operation and a second operation on the operation unit, and acquires connection information for connecting to the server and causes the connection information to be displayed on the display unit.

8. A control program for a display control device including a display unit arranged between multiple users when in use, a voice acquisition unit arranged in a position capable of acquiring voices of one or more of the multiple users, a communication unit configured to be able to communicate with a server, and an input unit that receives input from an operation unit, the display control program causing a control unit of the display control device to execute processes including: acquiring a voice signal from the voice acquisition unit via the communication unit; displaying character information into which the voice signal is converted on the display unit; transmitting to the server, based on a first operation and a second operation on the operation unit, the character information displayed on the display unit from when the first operation is accepted until when the second operation is accepted, and acquiring connection information for connecting to the server and displaying the connection information on the display unit.

Images