Information processing device, information processing system, information processing method, and program

Abstract

Reduce the effort required to create documents. [Solution] The information processing device comprises a format acquisition unit that acquires format information of a document, a speech recognition unit that acquires speech recognition results of a dialogue between multiple users, a text generation unit that generates text data to be included in the document based on the speech recognition results, and a document generation unit that generates document data representing the document based on the text data and the format information.

Description

Technical Field

[0006] , , ,

[0001] The present invention relates to an information processing apparatus, an information processing system, an information processing method, and a program.

Background Art

[0002] Techniques for generating materials for proposing products to customers are known. For example, Patent Document 1 discloses a proposal material generation system that generates proposal materials based on design information obtained from the website of a proposed customer in order to propose a product to the proposed customer.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, in the prior art, there is room for reducing the labor of creating documents. For example, when proposal materials cannot be immediately prepared at the negotiation or meeting site, it is necessary to take the materials back, newly consider the configuration, prepare the proposal materials again, and then make a re-visit or hold a meeting again. If it takes time and effort to prepare the proposal materials, there is a risk of losing the order due to the competing company seizing the contract opportunity during the period until the re-visit.

[0004] One embodiment of the present invention aims to reduce the labor of creating documents in view of the above technical problems.

Means for Solving the Problems

[0007] [Figure 1] This figure shows an example of the system configuration of a dialogue system according to one embodiment. [Figure 2] This figure shows an example of a computer hardware configuration according to one embodiment. [Figure 3] This figure shows an example of the hardware configuration of a terminal device according to one embodiment. [Figure 4] This figure shows an example of the functional configuration of a dialogue system according to one embodiment. [Figure 5] This is a sequence diagram showing an example of an interaction method according to one embodiment. [Figure 6] This figure shows an example of template information according to one embodiment. [Figure 7] This flowchart shows an example of a text generation process according to one embodiment. [Figure 8] This figure shows an example of product information according to one embodiment. [Figure 9] This figure shows an example of text data according to one embodiment. [Figure 10] This figure shows an example of a dialogue screen according to one embodiment. [Figure 11] This figure shows an example of a proposal document according to one embodiment. [Modes for carrying out the invention]

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, components having the same function are numbered the same, and redundant explanations are omitted.

[0009] [Embodiment] The first embodiment of the present invention is an information processing system that provides a dialogue service. Hereinafter, the information processing system according to this embodiment will be referred to as the "dialogue system." The dialogue service is an information communication service that enables dialogue with a dialogue agent.

[0010] In a dialogue service, the dialogue between the user and the dialogue agent progresses through automatic responses to messages from the user. The dialogue service can have one user or multiple users. In other words, the dialogue service can facilitate one-on-one dialogues between a user and the dialogue agent, or it can facilitate multi-person dialogues involving multiple users and the dialogue agent.

[0011] <System Configuration> Figure 1 shows an example of the system configuration of a dialogue system according to one embodiment. In the example in Figure 1, the dialogue system 1 includes a server device 10 connected to a communication network N such as the Internet or a LAN (Local Area Network), and a terminal device 50.

[0012] Figure 1 shows an example of a usage scenario for the dialogue system 1, where sales representative 2 conducts a business negotiation with customer 3. Sales representative 2 is carrying a terminal device 50 that displays a virtual salesperson 51. Sales representative 2 conducts the business negotiation with customer 3, with the virtual salesperson 51 participating. Here, the negotiation is an example of a dialogue. Sales representative 2 and customer 3 are examples of users of the dialogue service. Virtual salesperson 51 is an example of a dialogue agent.

[0013] Terminal device 50 is an information terminal used by sales representative 2, such as a PC (Personal Computer), tablet, or smartphone. Terminal device 50 acquires the spoken voice of sales representative 2 or customer 3, who are participants in the business negotiation, and transmits the acquired spoken voice data to server device 10. The spoken voice is an example of utterances in a dialogue. The utterances in a dialogue may also include utterances in the form of text data, such as chat. Here, the explanation will be given assuming that the utterances are spoken voice.

[0014] The server device 10 is, for example, an information processing device having a computer configuration or a system including a plurality of computers. The server device 10 acquires the voice data transmitted by the terminal device 50 and generates a response content for the uttered voice indicated in the voice data. This response content includes, for example, the language information, non-verbal information, recommendation information, and explanation of the recommendation information of the virtual salesperson 51.

[0015] Further, the server device 10 controls the virtual salesperson 51 displayed on the terminal device 50 according to this response content. For example, the server device 10 controls the utterance of the virtual salesperson 51 based on the language information of the virtual salesperson 51. Also, the server device 10 controls the display of the virtual salesperson 51 and the output of paralanguage, etc. based on the non-verbal information of the virtual salesperson 51. Here, the non-verbal information includes, for example, one or more pieces of information among the expression, gaze, posture, emotion, movement, and paralanguage of the virtual salesperson 51. Also, the paralanguage includes voice information other than language such as the tone of voice, speaking speed, voice height, voice strength, coughing, sighing, laughing, or silence.

[0016] The server device 10 implements a generative AI (Artificial Intelligence). The generative AI generates text data corresponding to the input prompt, such as ChatGPT (registered trademark), and outputs it to the server device 10. Note that the prompt in this embodiment is text data for expressing the dialogue content such as conversations, instructions, and questions for the generative AI implemented by the server device 10. Also, the generative AI does not necessarily have to be inside the server device 10 and may be implemented as a device separate from the server device 10. Also, the server device 10 may be implemented by one or more information processing devices or may be implemented by a plurality of information processing devices.

[0017] The terminal device 50 displays the virtual sales 51 controlled by the server device 10, and outputs the speech and paralanguage of the virtual sales 51. Thereby, the virtual sales 51 can propose, for example, recommendation information according to the needs of the customer 3 to the customer 3, the salesperson 2, etc. according to the flow of the business negotiation.

[0018] As a specific example, when it is determined that the need of the customer 3 is "electronic invoicing", the virtual sales 51 proposes commercial materials related to the electronic invoice issuing system, such as "What about product A if you go for electronic invoicing?" Also, as an explanation of the commercial material, the virtual sales 51 may show the basis of the proposal, such as "Product A is the best-selling in the manufacturing industry within the prefecture this year.", or may compare it with other products, etc.

[0019] The server device 10 may generate different response contents for each speaker according to the acquired speech audio. For example, when the speaker is the salesperson 2, the virtual sales 51 displays the materials of the commercial materials proposed to the customer 3 on the terminal device 50 and waits for the instructions of the salesperson 2. Thereby, the salesperson 2 can explain the commercial material based on the materials displayed by the virtual sales 51, or can also let the virtual sales 51 explain the commercial material. On the other hand, when the speaker is the customer 3, the virtual sales 51 signals, for example, by a gesture such as raising a hand, that there are commercial materials that can be proposed. Thereby, when the proposal by the virtual sales 51 is necessary, the salesperson 2 can let the virtual sales 51 propose the commercial material, and when the proposal of the commercial material by the virtual sales 51 is not necessary, the salesperson 2 can suppress the proposal of the commercial material by the virtual sales 51.

[0020] Note that dialogue is a means of transmitting information, opinions, feelings, etc. between individuals or groups. This means may include language information using words and non-verbal information such as gestures, expressions, or body language. The dialogue system 1 may, for example, execute other dialogues such as meetings, interviews, lectures, lessons, medical care, nursing care, or counseling.

[0021] <Hardware Configuration> (Computer hardware configuration) The server device 10 has, for example, a hardware configuration of a computer 200 as shown in Figure 2. Alternatively, the server device 10 is composed of multiple computers 200. Furthermore, the terminal device 50 may have, for example, a hardware configuration of a computer 200 as shown in Figure 2.

[0022] Figure 2 shows an example of the hardware configuration of a computer according to one embodiment. The computer 200 includes, for example, a CPU (Central Processing Unit) 201, ROM (Read Only Memory) 202, RAM (Random Access Memory) 203, HD (Hard Disk) 204, HDD (Hard Disk Drive) controller 205, display 206, external device connection I / F (Interface) 207, network I / F 208, keyboard 209, pointing device 210, DVD-RW (Digital Versatile Disk Rewritable) drive 212, media I / F 214, and bus line 215, as shown in Figure 2.

[0023] Furthermore, if the computer 200 is the terminal device 50, the computer 200 further includes a microphone 221, a speaker 222, an audio input / output interface 223, a CMOS (Complementary Metal Oxide Semiconductor) sensor 224, and an image sensor interface 225, etc.

[0024] Of these components, the CPU 201 controls the overall operation of the computer 200. The ROM 202 stores programs used to start the computer 200, such as the IPL (Initial Program Loader). The RAM 203 is used, for example, as a work area for the CPU 201. The HD 204 stores programs such as the OS (Operating System), applications, device drivers, and various data. The HDD controller 205 controls the reading or writing of various data to the HD 204, for example, according to the control of the CPU 201. Note that the HD 204 and the HDD controller 205 are examples of storage devices.

[0025] Display 206 displays various information, such as a cursor, menu, window, text, or image. Note that display 206 may be located outside the computer 200. External device connection I / F 207 is an interface for connecting various external devices to the computer 200. Network I / F 208 is an interface for connecting the computer 200 to a communication network N and communicating with other devices.

[0026] The keyboard 209 is a type of input device equipped with multiple keys for inputting characters, numbers, and various instructions. The pointing device 210 is a type of input device for selecting and executing various instructions, selecting processing targets, moving the cursor, and so on. The keyboard 209 and the pointing device 210 may be located outside the computer 200.

[0027] The DVD-RW drive 212 controls the reading or writing of various data to the DVD-RW 211, which is an example of a removable recording medium. Note that the DVD-RW 211 is not limited to DVD-RW; it may be any other removable recording medium. The media interface 214 controls the reading or writing (storage) of data to the media 213, such as flash memory. The bus line 215 includes an address bus, a data bus, and various control signals for electrically connecting the above components.

[0028] Microphone 221 is a built-in circuit that converts sound into electrical signals. Speaker 222 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and speech. Audio input / output interface 223 is a circuit that processes the input and output of audio signals between microphone 221 and speaker 222 according to the control of CPU 201.

[0029] The CMOS sensor 224 is a type of built-in imaging means that captures an image of a subject (e.g., a self-portrait) and obtains image data according to the control of the CPU 201. The computer 200 may have other imaging means, such as a CCD (Charge Coupled Device) sensor, instead of the CMOS sensor 224. The image sensor interface 225 is a circuit that controls the driving of the CMOS sensor 224.

[0030] (An example of terminal device hardware configuration) Figure 3 shows an example of the hardware configuration of a terminal device according to one embodiment. Here, we will describe an example of the hardware configuration of the terminal device 50 when the terminal device 50 is an information terminal such as a smartphone or a tablet device.

[0031] In the example shown in Figure 3, the terminal device 50 includes a CPU 301, ROM 302, RAM 303, storage device 304, CMOS sensor 305, image sensor I / F 306, acceleration / direction sensor 307, media I / F 309, and GPS (Global Positioning System) receiver 310.

[0032] Of these, the CPU 301 controls the operation of the entire terminal device 50 by executing a predetermined program. The ROM 302 stores programs used to start the CPU 301, such as an IPL. The RAM 303 is used as the work area for the CPU 301. The storage device 304 is a large-capacity storage device that stores programs such as the OS and applications, and various data, and is implemented by, for example, an SSD (Solid State Drive) or flash ROM.

[0033] The CMOS sensor 305 is a type of built-in imaging means that captures an image of a subject (mainly a self-portrait) and obtains image data according to the control of the CPU 301. The terminal device 50 may have an imaging means such as a CCD sensor instead of the CMOS sensor 305. The image sensor interface 306 is a circuit that controls the driving of the CMOS sensor 305. The acceleration and orientation sensor 307 is a type of sensor such as an electronic magnetic compass, gyrocompass, or acceleration sensor that detects the Earth's magnetic field. The media interface 309 controls the reading or writing (storage) of data to or from the media (storage medium) 308, such as flash memory. The GPS receiver 310 receives GPS signals (positioning signals) from GPS satellites.

[0034] Furthermore, the terminal device 50 includes a long-range communication circuit 311, an antenna 311a for the long-range communication circuit 311, a CMOS sensor 312, an image sensor interface 313, a microphone 314, a speaker 315, an audio input / output interface 316, a display 317, an external device connection interface 318, a short-range communication circuit 319, an antenna 319a for the short-range communication circuit 319, and a touch panel 320.

[0035] Of these, the long-distance communication circuit 311 is a circuit that communicates with other devices, for example, via a communication network N. The CMOS sensor 312 is a type of built-in imaging means that captures an image of a subject and obtains image data according to the control of the CPU 301. The image sensor interface 313 is a circuit that controls the driving of the CMOS sensor 312. The microphone 314 is a built-in circuit that converts sound into electrical signals. The speaker 315 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and speech. The sound input / output interface 316 is a circuit that processes the input and output of sound signals between the microphone 314 and the speaker 315 according to the control of the CPU 301.

[0036] The display 317 is a type of display means, such as a liquid crystal or organic EL (electroluminescence), that displays images of the subject and various icons. The external device connection I / F 318 is an interface for connecting various external devices. The short-range communication circuit 319 includes a circuit for performing short-range wireless communication. The touch panel 320 is a type of input means that allows the user to operate the terminal device 50 by pressing the display 317.

[0037] Furthermore, the terminal device 50 is equipped with a bus line 321. The bus line 321 includes an address bus, a data bus, and various control signals for electrically connecting each component, such as the CPU 301 shown in Figure 3.

[0038] Note that the hardware configuration of the terminal device 50 shown in Figure 3 is just one example. The terminal device 50 may have other hardware configurations as long as it includes a computer, communication circuits, a display, a microphone, and a speaker, etc.

[0039] <Functional Configuration> The functional configuration of the dialogue system 1 will be explained with reference to Figure 4. Figure 4 is a diagram showing an example of the functional configuration of a dialogue system according to one embodiment.

[0040] ≪Server Equipment≫ As shown in Figure 4, the server device 10 includes a format information storage unit 101, a product information storage unit 102, a recognition result storage unit 103, a text storage unit 104, a document storage unit 105, a communication control unit 110, a storage control unit 120, a format acquisition unit 130, a speech recognition unit 140, a product search unit 150, a text generation unit 160, a product presentation unit 170, a document generation unit 180, and a document transmission unit 190.

[0041] The format information storage unit 101, the product information storage unit 102, the recognition result storage unit 103, the text storage unit 104, and the document storage unit 105 are implemented, for example, using the HD204 shown in Figure 2. Reading or writing data stored in the HD204 is performed, for example, via the HDD controller 205.

[0042] The communication control unit 110, memory control unit 120, format acquisition unit 130, speech recognition unit 140, product search unit 150, text generation unit 160, product presentation unit 170, document generation unit 180, and document transmission unit 190 are realized, for example, by a program deployed from ROM 202 to RAM 203 as shown in Figure 2, which is then executed by the CPU 201 and network I / F 208.

[0043] Format information is stored in the format information storage unit 101. Format information may already be stored in the format information storage unit 101. Format information received from the terminal device 50 may also be stored in the format information storage unit 101.

[0044] Formatting information is an example of electronic data that indicates the format of a document. Formatting information may also be, for example, document template information. Documents may include, for example, proposals, meeting minutes, reports, or contracts. Template information may include, for example, the page structure of the document, document headings, items to be included on each page, and the arrangement of items included on each page. Template information may also include placeholders for embedding data. Placeholders may be able to embed any data, such as text data, image data, or audio data.

[0045] Product information is stored in the product information storage unit 102. Product information may already be stored in the product information storage unit 102. Product information received from the terminal device 50 may also be stored in the product information storage unit 102.

[0046] Product information is an example of electronic data that shows information about a product. A product is an example of an object that may be mentioned in a business negotiation. A product may include, for example, the goods or services handled by sales representative 2. Product information may also include key information used for searching. Key information may include, for example, keywords related to the product, or document vectors extracted from the product information.

[0047] The recognition result storage unit 103 stores the speech recognition results of the dialogue. The speech recognition results stored in the recognition result storage unit 103 may be generated by the speech recognition unit 140.

[0048] The speech recognition result is an example of text data recognized from speech data received from the terminal device 50. The speech recognition result of a dialogue may also be data accumulated in chronological order from multiple text data recognized from speech data received during the dialogue.

[0049] The recognition result storage unit 103 may store the speech recognition result for each dialogue. The recognition result storage unit 103 may store the speech recognition result for each customer 3. The recognition result storage unit 103 may store the speech recognition results from multiple business negotiations with the same customer 3 in association with each other.

[0050] The text storage unit 104 stores text data to be included in the document. The text data stored in the text storage unit 104 may be generated by the text generation unit 160.

[0051] The text storage unit 104 may store text data for each dialogue. The text storage unit 104 may store text data for each customer 3. The text storage unit 104 may store text data for each product.

[0052] Document data is stored in the document storage unit 105. The document data stored in the document storage unit 105 may be generated by the document generation unit 180. The document data stored in the document storage unit 105 may also be edited by the document generation unit 180.

[0053] Document data is an example of electronic data representing a document. Document data may be stored in any file format. File formats may include, for example, presentation format, word processing format, text format, or spreadsheet format.

[0054] The communication control unit 110 controls communication with the terminal device 50. The communication control unit 110 may receive format information entered into the terminal device 50 from the terminal device 50. The communication control unit 110 may receive audio data indicating spoken voice entered into the terminal device 50 from the terminal device 50. The communication control unit 110 may transmit the response content to the spoken voice to the terminal device 50. The communication control unit 110 may transmit screen data for displaying a dialogue screen on the terminal device 50 to the terminal device 50.

[0055] Screen data is electronic data used to display a screen on the display 317 of the terminal device 50. Screen data is electronic data written in, for example, HTML (HyperText Markup Language), and may also include programs written in a scripting language such as JavaScript (registered trademark).

[0056] The memory control unit 120 controls the storage of data in the storage device of the server device 10. The memory control unit 120 may read format information to and write format information to the format information storage unit 101. The memory control unit 120 may read product information from the product information storage unit 102. The memory control unit 120 may read speech recognition results to and write recognition result storage unit 103. The memory control unit 120 may read text data to and write text data to and from the text storage unit 104. The memory control unit 120 may read document data to and from the document storage unit 105.

[0057] The format acquisition unit 130 acquires format information. The format acquisition unit 130 may acquire format information received from the terminal device 50. The format acquisition unit 130 may also perform control to store the format information in the format information storage unit 101.

[0058] The speech recognition unit 140 recognizes the user's spoken voice. The speech recognition unit 140 may recognize the user's spoken voice based on the voice data received from the terminal device 50. The speech recognition unit 140 may also recognize the spoken voices of multiple users participating in the dialogue. The speech recognition unit 140 may generate a speech recognition result for the dialogue by accumulating the speech recognition results for each spoken voice in chronological order.

[0059] The speech recognition unit 140 may perform control to store the speech recognition results of the dialogue in the recognition result storage unit 103. The speech recognition unit 140 may also generate speech recognition results of a dialogue by accumulating the speech recognition results generated during the dialogue in the recognition result storage unit 103 in chronological order.

[0060] The product search unit 150 searches for product information. The product search unit 150 may search for product information based on the speech recognition results generated by the speech recognition unit 140. The product search unit 150 may also search for product information stored in the product information storage unit 102.

[0061] The product search unit 150 may extract keywords from the speech recognition results of the dialogue. The product search unit 150 may also extract keywords based on a language model. The keywords may include words or sentences that indicate the topic of the dialogue. The topic of the dialogue may include the needs of customer 3. The needs of customer 3 may include, for example, requests, issues, worries, problems, etc. The product search unit 150 may also extract product information that matches the keywords from the product information stored in the product information storage unit 102.

[0062] The product search unit 150 may generate a search query based on the speech recognition results of the dialogue. The product search unit 150 may generate a search query based on a language model. The product search unit 150 may generate a search query that indicates the needs of customer 3. The product search unit 150 may extract product information that is close in distance to the search query from the product information stored in the product information storage unit 102. In other words, the product search unit 150 may extract product information similar to the search query from the product information stored in the product information storage unit 102.

[0063] The language model is an example of a machine learning model trained to perform a predetermined language processing task. The language model may be a large language model (LLM) trained to perform various language processing tasks, or a small language model (SLM) trained to perform a specific language processing task. The language model may be built into the server device 10, or it may be held by another information processing device that can communicate via the communication network N.

[0064] The text generation unit 160 generates text data to be included in the document. The text generation unit 160 may generate text data based on a language model. The text generation unit 160 may generate text data based on the speech recognition results of the dialogue. The text generation unit 160 may generate text data based on product information retrieved by the product search unit 150.

[0065] The text generation unit 160 may generate text data for each product. The text generation unit 160 may omit generating text data for products for which text data has already been generated. The text generation unit 160 may also perform control to store the text data in the text storage unit 104.

[0066] The product presentation unit 170 presents product information to the user. The product presentation unit 170 may also present product information for which text data has already been generated by the text generation unit 160. The product presentation unit 170 may also present product information relating to products mentioned in the conversation. For example, the product presentation unit 170 may present product information relating to a product explained by sales representative 2 to customer 3 during a business negotiation to sales representative 2 or customer 3.

[0067] The product presentation unit 170 may generate screen data containing product information to be presented to the user. The product presentation unit 170 may also control the transmission of the screen data containing product information to the terminal device 50. The terminal device 50 may output an interactive screen displaying the product information to the display 317 based on the screen data containing product information.

[0068] The document generation unit 180 generates document data. The document generation unit 180 may generate document data based on format information read from the format information storage unit 101. The document generation unit 180 may generate document data based on text data read from the text storage unit 104. The document generation unit 180 may generate document data based on product information read from the product information storage unit 102.

[0069] For example, the document generation unit 180 may write text data according to the format shown in the format information. The document generation unit 180 may embed text data in the placeholders of the template information. The document generation unit 180 may embed product information in the placeholders of the template information.

[0070] The document generation unit 180 may generate document data relating to the product specified by the user. The user may specify the product on a screen where the product presentation unit 170 presents product information to the user.

[0071] The document generation unit 180 may edit the document data. The document generation unit 180 may edit the document data based on user instructions. The document generation unit 180 may delete parts of the document data. The document generation unit 180 may change the text data contained in the document data. For example, the document generation unit 180 may delete one or more pages contained in the document data. For example, the document generation unit 180 may modify the text data embedded in placeholders.

[0072] The document generation unit 180 may perform control to store the generated document data in the document storage unit 105. The document generation unit 180 may also perform control to store the edited document data in the document storage unit 105.

[0073] The document transmission unit 190 transmits document data. The document transmission unit 190 may also transmit the document data to the terminal device 50. The document transmission unit 190 may also control the transmission of screen data, including the document data, to the terminal device 50. The terminal device 50 may output a screen displaying the document data to the display 317 based on the screen data.

[0074] The document transmission unit 190 may transmit document data to one or more users. For example, the document transmission unit 190 may transmit document data to users via communication services such as email, messaging services, or SNS (Social Networking Service).

[0075] The document transmission unit 190 may transmit document data to sales representative 2. Sales representative 2 may forward the document data received from the server device 10 to customer 3. The document transmission unit 190 may transmit document data to customer 3. The document transmission unit 190 may obtain customer information, including customer 3's contact information (e.g., email address), in order to transmit document data to customer 3.

[0076] <Terminal Devices> As shown in Figure 4, the terminal device 50 includes a communication control unit 510, a format input unit 520, an audio input unit 530, and a display control unit 540.

[0077] The communication control unit 510, format input unit 520, voice input unit 530, and display control unit 540 are implemented, for example, by a process in which a program loaded from the ROM 302 to the RAM 303, as shown in Figure 3, is executed by the CPU 301.

[0078] The communication control unit 510 controls communication with the server device 10. The communication control unit 510 may transmit format information entered into the terminal device 50 to the server device 10. The communication control unit 510 may transmit audio data indicating spoken voice entered into the terminal device 50 to the server device 10. The communication control unit 510 may receive response content to spoken voice from the server device 10. The communication control unit 510 may receive screen data from the server device 10 to display a dialogue screen on the terminal device 50.

[0079] The format input unit 520 accepts format information input. The format input unit 520 may accept format information entered by the user into the terminal device 50. The format information input may be performed on the screen displayed on the display 317. The format input unit 520 may transmit the format information to the server device 10.

[0080] The voice input unit 530 accepts input of voice data representing spoken speech. The voice input unit 530 may detect the user's spoken speech based on the acoustic signal picked up by the microphone 314. The voice input unit 530 may generate voice data including the voice segment detected from the acoustic signal. The voice input unit 530 may transmit the voice data to the server device 10.

[0081] The display control unit 540 controls the display 317 to show the screen provided by the dialogue system 1. The display control unit 540 may be, for example, an application with browser functionality. The application with browser functionality may be a web browser pre-installed on the terminal device 50, or an application that displays web content using a web view function.

[0082] The display control unit 540 may display an interactive screen based on screen data received from the server device 10. The display control unit 540 may display an interactive screen that includes an interactive agent. The display control unit 540 may display an interactive screen that includes product information. The display control unit 540 may display an interactive screen that includes document data.

[0083] <Processing Procedure> The processing procedure of the dialogue method executed by the dialogue system 1 will be explained with reference to Figures 5 to 11. Figure 5 is a sequence diagram showing an example of a dialogue method according to one embodiment. Hereafter, an example will be described in which a proposal document, which is an example of a document, is created based on template information, which is an example of format information, during a business negotiation between a sales representative 2 and a customer 3.

[0084] In step S101, sales representative 2 inputs template information into terminal device 50. The format input unit 520 of terminal device 50 receives the input template information. The format input unit 520 sends the received template information to the communication control unit 510.

[0085] In step S102, the communication control unit 510 of the terminal device 50 receives template information from the format input unit 520. The communication control unit 510 transmits the template information to the server device 10.

[0086] In step S103, the communication control unit 110 of the server device 10 receives template information from the terminal device 50. The communication control unit 110 sends the received template information to the format acquisition unit 130. The format acquisition unit 130 sends the template information received from the communication control unit 110 to the storage control unit 120. The storage control unit 120 stores the template information received from the format acquisition unit 130 in the format information storage unit 101.

[0087] Figure 6 shows an example of template information according to one embodiment. Figure 6 shows template information PT for a proposal document created in presentation format. As shown in Figure 6, template information PT may include multiple pages P1 to P6. Pages P1 to P5 may each have a heading. Page headings may include, for example, cover page, greeting, proposal outline, product details, summary, etc. Each heading may include multiple pages. For example, the template information PT shown in Figure 6 includes two pages of product details.

[0088] Each page (P1-P6) of the template information PT may have a predetermined layout for the items contained on that page. The items on a page may include text, shapes, tables, animations, still images, videos, audio, etc. The items on a page may include fixed information or placeholders for embedding data. Placeholders may be assigned identification information to identify them.

[0089] Figure 6 shows an example of page P4, which contains product details. Page P4 shown in Figure 6 includes multiple placeholders with item names enclosed in angle brackets ("<" and ">"). Page P4 includes placeholders for embedding the title, customer needs, product effects, usage scenarios, etc.

[0090] Let's return to Figure 5 for explanation. In step S104, sales representative 2 and customer 3 begin negotiations. The voice input unit 530 of the terminal device 50 receives voice data representing the speech of either sales representative 2 or customer 3. The voice input unit 530 sends the received voice data to the communication control unit 510.

[0091] In step S105, the communication control unit 510 of the terminal device 50 receives voice data from the voice input unit 530. The communication control unit 510 transmits the voice data to the server device 10.

[0092] In step S106, the communication control unit 110 of the server device 10 receives voice data from the terminal device 50. The communication control unit 110 sends the received voice data to the voice recognition unit 140. The voice recognition unit 140 recognizes the speech of the sales representative 2 or the customer 3 based on the voice data. The voice recognition unit 140 sends the voice recognition result to the storage control unit 120. The storage control unit 120 stores the voice recognition result in the recognition result storage unit 103.

[0093] In step S107, the server device 10 generates text data to be included in the proposal document based on the speech recognition results generated in step S106. The server device 10 may repeatedly execute step S107 at predetermined time intervals. The predetermined time interval may be, for example, about 15 seconds.

[0094] The text generation process performed by the server device 10 (step S107 in Figure 5) will be described in more detail with reference to Figures 7 to 9. Figure 7 is a flowchart showing an example of the text generation process according to one embodiment.

[0095] In step S201, the storage control unit 120 of the server device 10 reads the speech recognition results from the recognition result storage unit 103. Based on the read speech recognition results, the server device 10 determines whether or not there are speech recognition results to be processed. The speech recognition results to be processed are those generated after the last text generation process was executed.

[0096] If the server device 10 determines that there are speech recognition results to be processed (YES), it proceeds to step S202. On the other hand, if it determines that there are no speech recognition results to be processed (NO), the server device 10 terminates the text generation process.

[0097] In step S202, the product search unit 150 of the server device 10 obtains the speech recognition results to be processed from the storage control unit 120. Based on the speech recognition results to be processed, the product search unit 150 extracts the needs of customer 3. For example, the product search unit 150 may extract keywords indicating the needs of customer 3 from the speech recognition results to be processed. Alternatively, for example, the product search unit 150 may generate a search query indicating the needs of customer 3 based on the speech recognition results to be processed.

[0098] In step S203, the product search unit 150 of the server device 10 searches for product information related to the needs of customer 3 from the product information stored in the product information storage unit 102. For example, the product search unit 150 may extract product information that matches keywords from the product information storage unit 102. Alternatively, for example, the product search unit 150 may extract product information from the product information storage unit 102 that is close in distance between the search query and the document vector associated with each product. Alternatively, for example, the product search unit 150 may search and extract products based on a space created from text information about products stored in the product information storage unit. It may also search and extract using a method generally known as RAG.

[0099] A document vector is an embedding vector generated from product information. A document vector may be generated based on at least one of the product name or product description. A document vector may also be generated based on a language model.

[0100] Figure 8 shows an example of product information according to one embodiment. As shown in Figure 8, product information may include data items such as product ID, product name, product description, keywords, and document vectors. The product ID is an example of identification information that identifies a product. The product name is the name of the product. The description is an example of text that explains the contents of the product. Note that product information may include only keywords or only document vectors.

[0101] Keywords are those related to the product. Keywords may include words or phrases that describe, for example, problems solved by the product, effects obtained by introducing the product, or usage scenarios for the product. Keywords may also be extracted from the product name or product description.

[0102] Let's return to Figure 7 for explanation. In step S204, the product search unit 150 of the server device 10 determines whether or not product information related to the customer 3's needs exists. In other words, it determines whether or not one or more product information items were found in step S203.

[0103] If the server device 10 determines that product information related to customer 3's needs exists (YES), it proceeds to step S205. On the other hand, if the server device 10 determines that product information related to customer 3's needs does not exist (NO), it terminates the text generation process.

[0104] In step S205, the product search unit 150 of the server device 10 sends the identification information (e.g., product ID) of the product information extracted in step S204 to the storage control unit 120. Based on the identification information of the product information, the storage control unit 120 reads the product information from the product information storage unit 102. The storage control unit 120 then sends the read product information to the text generation unit 160.

[0105] The text generation unit 160 determines whether or not text data has already been generated for the product information received from the storage control unit 120. Specifically, the text generation unit 160 determines whether or not text data related to that product information is stored in the text storage unit 104.

[0106] If it is determined that the text data has already been generated (YES), the server device 10 terminates the text generation process. On the other hand, if it is determined that the text data has not yet been generated (NO), the server device 10 proceeds to step S206.

[0107] In step S206, the text generation unit 160 of the server device 10 generates text data to be included in the proposal document based on the speech recognition results read in step S201 and the product information read in step S205. Specifically, the text generation unit 160 may cause the language model to generate text data to be included in the proposal document by inputting prompts containing the speech recognition results and product information into the language model.

[0108] The prompt may include instructional information that instructs the generation of text data. For example, the instructional information may be something like, "To introduce products that can appeal to the customer's concerns, write the content to be inserted into the document based on the customer's conversation, using polite language." The prompt may also include constraints that specify the number of characters or word endings of the text data.

[0109] The text generation unit 160 sends the generated text data to the storage control unit 120. The storage control unit 120 stores the text data received from the text generation unit 160 in the text storage unit 104.

[0110] Figure 9 shows an example of text data according to one embodiment. As shown in Figure 9, the text data is data that indicates the text to be included in the document. The text data may include text data to be embedded in placeholders in template information. The text data may be associated with the item names of the placeholders.

[0111] Figure 9 shows an example where multiple text data separated by commas ("") are stored in a single data item, but each of the multiple text data may be stored in a different data item. Also, Figure 9 shows an example where a placeholder item name and text data are associated with an equals sign ("="), but the placeholder item name and text data may each be stored in a different data item.

[0112] Text data may be associated with a product ID. That is, text data may be generated for each product. Text data may also be associated with an explained flag. The explained flag is a truth value indicating whether or not the product was explained during the sales negotiation. The explained flag may be set to true (explained) by the server device 10 when the virtual salesperson 51 speaks an explanation of the product, or it may be set to true (explained) by the salesperson 2 when the salesperson 2 explains the product to the customer 3.

[0113] Let's return to Figure 5 for explanation. In step S108, the sales representative 2 or the customer 3 initiates the creation of the proposal document. This can be done by speaking an instruction to create the proposal document, or by using the screen displayed on the terminal device 50's display 317. For example, the sales representative 2 might say, "Create the proposal document." The communication control unit 510 of the terminal device 50 sends a creation start request to the server device 10 in response to the proposal document creation start operation.

[0114] In step S109, the communication control unit 110 of the server device 10 receives a creation start request from the terminal device 50. When the communication control unit 110 receives the creation start request, the storage control unit 120 reads the product information for which text data has already been generated from the product information storage unit 102. Specifically, the storage control unit 120 extracts text data from the text storage unit 104 for which the explained flag is set to true (explained). The storage control unit 120 also obtains the product ID associated with the extracted text data. Then, the storage control unit 120 reads the product information including the obtained product ID from the product information storage unit 102.

[0115] In step S110, the memory control unit 120 sends the product information read in step S109 to the product presentation unit 170. The product presentation unit 170 generates screen data containing the product information received from the memory control unit 120. The product presentation unit 170 sends the screen data containing the product information to the communication control unit 110. The communication control unit 110 transmits the screen data containing the product information to the terminal device 50.

[0116] In step S111, the communication control unit 510 of the terminal device 50 receives screen data containing product information from the server device 10. When the communication control unit 510 receives the screen data containing product information, the display control unit 540 outputs an interactive screen displaying product information to the display 317 based on the screen data containing product information. In other words, the terminal device 50 presents the user with product information that can be used to create proposal materials.

[0117] Figure 10 shows an example of a dialogue screen according to one embodiment. As shown in Figure 10, the dialogue screen 600 may have an agent display area 610, a message display area 620, and a product presentation area 630.

[0118] The agent display area 610 is the area that displays the virtual salesperson 51. The message display area 620 is the area that displays messages to notify the user. In the example shown in Figure 10, the message display area 620 displays a message requesting the user to select the products for which to create a proposal document.

[0119] The product presentation area 630 includes one or more product display areas 635 (635-1 to 635-4). The product display area 635 is an area that displays product information capable of generating proposal materials.

[0120] Specifically, the product display area 635 displays product information for which text data is stored in the text storage unit 104. Figure 10 shows a product presentation area 630 that includes four product display areas 635-1 to 635-4, but the number of product display areas 635 may be three or less, or five or more.

[0121] The product presentation area 630 displays product display areas 635 in a selectable format. The product presentation area 630 may display multiple product display areas 635 in a selectable format. When a product display area 635 is selected in the product presentation area 630, the product information displayed in that product display area 635 is designated as product information to be included in the proposal document.

[0122] Let's return to Figure 5 for explanation. In step S112, the user of the dialogue system 1 selects the product information for which to create a proposal document on the dialogue screen 600 displayed on the terminal device 50. The display control unit 540 of the terminal device 50 accepts the specified product information.

[0123] In step S113, the display control unit 540 of the terminal device 50 sends the identification information of the product specified by the user in step S112 to the communication control unit 510. The communication control unit 510 sends a request to create a proposal document to the server device 10. The request to create a proposal document includes the identification information of the product specified by the user.

[0124] In step S114, the communication control unit 110 of the server device 10 receives a request to create a proposal document from the terminal device 50. When the communication control unit 110 receives the request to create a proposal document, the storage control unit 120 reads text data from the text storage unit 104 based on the identification information included in the request to create a proposal document. The storage control unit 120 also reads template information from the format information storage unit 101.

[0125] The memory control unit 120 sends text data and template information to the document generation unit 180. The document generation unit 180 generates proposal documents based on the text data and template information received from the memory control unit 120. Specifically, the document generation unit 180 embeds text data into placeholders in the template information. The document generation unit 180 sends the generated proposal documents to the memory control unit 120. The memory control unit 120 stores the proposal documents received from the document generation unit 180 in the document storage unit 105.

[0126] In step S115, the document generation unit 180 of the server device 10 sends the proposal document generated in step S114 to the document transmission unit 190. The document transmission unit 190 generates screen data including the proposal document. The document transmission unit 190 sends the screen data including the proposal document to the communication control unit 110. The communication control unit 110 transmits the screen data including the proposal document to the terminal device 50.

[0127] In step S116, the communication control unit 510 of the terminal device 50 receives screen data including the proposal document from the server device 10. When the communication control unit 510 receives the screen data including the proposal document, the display control unit 540 outputs an interactive screen that displays the proposal document to the display 317 based on the screen data including the proposal document. In other words, the terminal device 50 presents the user with the proposal document related to the product information selected in step S112.

[0128] Figure 11 shows an example of a proposal document according to one embodiment. Figure 11 shows a proposal document PM in which the text data shown in Figure 9 is embedded in the template information PT shown in Figure 6. As shown in Figure 11, it can be seen that in the proposal document PM, text data is embedded in each of the placeholders included in the template information PT.

[0129] Let's return to Figure 5 for explanation. In step S117, the sales representative 2 or customer 3 performs an operation to modify the proposal document. The operation to modify the proposal document may be done by speaking an instruction to modify the proposal document, or it may be done on the screen displayed on the display 317 of the terminal device 50. For example, customer 3 may say something like, "I selected this product, but I don't need it, so please delete it." The communication control unit 510 of the terminal device 50 sends a request to modify the proposal document to the server device 10 in response to the operation to modify the proposal document. The request to modify the proposal document includes information indicating the content of the modification.

[0130] In step S118, the communication control unit 110 of the server device 10 receives a request to revise the proposal document from the terminal device 50. When the communication control unit 110 receives the request to revise the proposal document, the document generation unit 180 revises the proposal document generated in step S114 based on the information indicating the revisions included in the request to revise the proposal document. The document generation unit 180 sends the revised proposal document to the storage control unit 120. The storage control unit 120 stores the revised proposal document received from the document generation unit 180 in the document storage unit 105.

[0131] In step S119, the document generation unit 180 of the server device 10 sends the proposal document modified in step S118 to the document transmission unit 190. The document transmission unit 190 generates screen data including the modified proposal document. The document transmission unit 190 sends the screen data including the modified proposal document to the communication control unit 110. The communication control unit 110 transmits the screen data including the modified proposal document to the terminal device 50.

[0132] In step S120, the communication control unit 510 of the terminal device 50 receives screen data containing the revised proposal document from the server device 10. When the communication control unit 510 receives the screen data containing the revised proposal document, the display control unit 540 updates the proposal document displayed on the interactive screen to the revised proposal document based on the screen data containing the revised proposal document. In other words, the terminal device 50 presents the user with the proposal document that was revised in step S118.

[0133] Note that the processes from step S117 to step S120 may be performed at will. For example, the processes from step S117 to step S120 do not need to be performed. Also, the processes from step S117 to step S120 may be performed multiple times.

[0134] Furthermore, in step S115 or step S119, the document transmission unit 190 may transmit the proposal materials (including the revised proposal materials) to sales representative 2 or customer 3. The document transmission unit 190 may transmit the proposal materials to sales representative 2 or customer 3 in accordance with the instructions of sales representative 2 or customer 3. The document transmission unit 190 may transmit the proposal materials to the user in step S115 or step S119, or it may transmit the proposal materials to the user after step S119.

[0135] <Effects of the Embodiment> The server device 10 according to this embodiment acquires document formatting information, generates text data to be included in the document based on the speech recognition results of a dialogue between multiple users, and generates document data representing the document based on the text data and formatting information. In one aspect, according to this embodiment, since a document based on the content of the dialogue can be automatically created in a predetermined format, the effort required to create the document can be reduced.

[0136] The server device 10 may generate text data based on information about the object mentioned in the dialogue. In one aspect, according to this embodiment, document data about the object mentioned in the dialogue can be generated.

[0137] The server device 10 may search for information about the object based on the dialogue topic extracted from the speech recognition result. In one aspect, according to this embodiment, document data about the object related to the dialogue topic can be generated.

[0138] Multiple users may include customers and sales representatives. Documents may include proposals, meeting minutes, reports, or contracts. In one respect, according to this embodiment, various documents can be created based on dialogue between customers and sales representatives.

[0139] The topics may include customer needs. The subjects may include the products handled by sales representatives. In one respect, this embodiment allows for the creation of documents related to products that address customer needs.

[0140] The server device 10 may present the user with a list of products mentioned in the dialogue. The server device 10 may generate document data showing proposal materials related to products specified by the user. In one aspect, according to this embodiment, proposal materials related to products desired by the user can be created.

[0141] The server device 10 may transmit document data to at least one of multiple users. In one respect, according to this embodiment, users can easily obtain the created documents.

[0142] [supplement] Each of the functions of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules designed to execute each of the functions described above.

[0143] The apparatus described in the examples represents only one of several computing environments for carrying out the embodiments disclosed herein. In one embodiment, the server apparatus 10 includes multiple computing devices, such as a server cluster. The multiple computing devices are configured to communicate with each other via any type of communication link, including a network or shared memory, and perform the processing disclosed herein.

[0144] Furthermore, each functional configuration of the server device 10 may be combined into a single server device, or it may be divided among multiple devices. In addition, at least some of the functional configurations of the server device 10 may be provided by the terminal device 50.

[0145] Examples of the present invention are as follows:

[0146] (Note 1) A format acquisition unit that acquires format information of a document, A speech recognition unit that acquires speech recognition results of conversations conducted by multiple users, A text generation unit generates text data to be included in the document based on the speech recognition results, A document generation unit generates document data representing the document based on the text data and the format information, An information processing device equipped with the following features.

[0147] (Note 2) The text generation unit generates the text data based on the information about the object mentioned in the dialogue. The document generation unit generates the document data relating to the object. The information processing device described in Appendix 1.

[0148] (Note 3) The system further includes a search unit that searches for information about the object based on the topic of the dialogue extracted from the speech recognition results. The information processing device described in Appendix 2.

[0149] (Note 4) The aforementioned users include customers and sales representatives, The aforementioned documents include proposal materials, meeting minutes, reports, or contracts. The information processing device described in Appendix 3.

[0150] (Note 5) The aforementioned topic includes the needs of the aforementioned customer, The aforementioned items include the merchandise handled by the sales representative, The information processing device described in Appendix 4.

[0151] (Note 6) The system further includes a display unit that presents to the user a list of the products mentioned in the aforementioned dialogue. The document generation unit generates the document data that shows the proposal materials relating to the product specified by the user. The information processing device described in Appendix 5.

[0152] (Note 7) The system further includes a document transmission unit that transmits the document data to at least one of the aforementioned users. An information processing device as described in any of the appendices 1 to 6.

[0153] (Note 8) An information processing system in which a terminal device and an information processing device can communicate via a network, The aforementioned terminal device is A format input unit that transmits document format information to the information processing device, A voice input unit that transmits voice data representing the spoken voices of a dialogue between multiple users to the information processing device, Equipped with, The aforementioned information processing device is A format acquisition unit that receives the aforementioned format information from the terminal device, A speech recognition unit that acquires the speech recognition results of the aforementioned dialogue, A text generation unit generates text data to be included in the document based on the speech recognition results, A document generation unit generates document data representing the document based on the text data and the format information, An information processing system equipped with the following features.

[0154] (Note 9) Computers Procedure for obtaining document formatting information, Procedure for obtaining speech recognition results from a conversation between multiple users, A procedure for generating text data to be included in the document based on the speech recognition results, A procedure for generating document data representing the document based on the text data and the format information, An information processing method that performs the following.

[0155] (Note 10) On the computer, Procedure for obtaining document formatting information, Procedure for obtaining speech recognition results from a conversation between multiple users, A procedure for generating text data to be included in the document based on the speech recognition results, A procedure for generating document data representing the document based on the text data and the format information, A program to execute.

[0156] Although embodiments of the present invention have been described above, the present invention is not limited to these specific embodiments, and various modifications and applications are possible within the scope of the gist of the present invention as described in the claims. [Explanation of Symbols]

[0157] 1. Dialogue System 2 Sales representatives 3 customers 10 Server devices 50 Terminal devices 51 Virtual Sales 101 Format information storage unit 102 Product Information Storage Unit 103 Recognition result storage unit 104 Text Storage Unit 105 Document Storage Unit 110 Communication Control Unit 120 Memory Control Unit 130 Format Acquisition Unit 140 Voice Recognition Unit 150 Product Search Department 160 Text generation unit 170 Product Presentation Department 180 Document Generation Unit 190 Document Transmission Department 510 Communication Control Unit 520 Format Input Section 530 Voice Input Section 540 Display Control Unit 600 Dialogue screen [Prior art documents] [Patent Documents]

[0158] [Patent Document 1] Japanese Patent Publication No. 2023-151545

Claims

1. A format acquisition unit that acquires format information of a document, A speech recognition unit that acquires speech recognition results of conversations conducted by multiple users, A text generation unit generates text data to be included in the document based on the speech recognition results, A document generation unit generates document data representing the document based on the text data and the format information, An information processing device equipped with the following features.

2. The text generation unit generates the text data based on the information about the object mentioned in the dialogue. The document generation unit generates the document data relating to the object. The information processing apparatus according to claim 1.

3. The system further includes a search unit that searches for information about the object based on the topic of the dialogue extracted from the speech recognition results. The information processing apparatus according to claim 2.

4. The aforementioned users include customers and sales representatives, The aforementioned documents include proposal materials, meeting minutes, reports, or contracts. The information processing apparatus according to claim 3.

5. The aforementioned topic includes the needs of the aforementioned customer, The aforementioned items include the merchandise handled by the sales representative, The information processing apparatus according to claim 4.

6. The system further includes a display unit that presents to the user a list of the products mentioned in the aforementioned dialogue. The document generation unit generates the document data that shows the proposal materials relating to the product specified by the user. The information processing apparatus according to claim 5.

7. The system further includes a document transmission unit that transmits the document data to at least one of the aforementioned users. An information processing apparatus according to any one of claims 1 to 6.

8. An information processing system in which a terminal device and an information processing device can communicate via a network, The aforementioned terminal device is A format input unit that transmits document format information to the information processing device, A voice input unit that transmits voice data representing the spoken voices of a dialogue between multiple users to the information processing device, Equipped with, The aforementioned information processing device is A format acquisition unit that receives the aforementioned format information from the terminal device, A speech recognition unit that acquires the speech recognition results of the aforementioned dialogue, A text generation unit generates text data to be included in the document based on the speech recognition results, A document generation unit generates document data representing the document based on the text data and the format information, An information processing system equipped with the following features.

9. Computers Procedure for obtaining document formatting information, Procedure for obtaining speech recognition results from a conversation between multiple users, A procedure for generating text data to be included in the document based on the speech recognition results, A procedure for generating document data representing the document based on the text data and the format information, An information processing method that performs the following.

10. On the computer, Procedure for obtaining document formatting information, Procedure for obtaining speech recognition results from a conversation between multiple users, A procedure for generating text data to be included in the document based on the speech recognition results, A procedure for generating document data representing the document based on the text data and the format information, A program to execute.

Images