Information processing device and program

Abstract

This technology makes it easier to have an information processing device act as the master of ceremonies. [Solution] The information processing device 1 acquires communication information, including information about the deceased and information about the officiant, etc., transmitted from the input terminal 2. Using the acquired communication information, it generates text data for each scene of the funeral service. Using the generated text, it further generates audio data for outputting the text as sound. The generation of audio data for reading condolence telegrams is performed using condolence telegrams that have been converted into image data by the scanner 3. Playback of the audio data generated in this scene-by-scene manner is performed by determining whether the start conditions for starting playback have been met, using image data from cameras 5 that photograph the entrance 9 and the seated officiant 8, sound data from microphone 6, or operation information from input device 7 that represents the operation of a physical button 7A.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a program.

Background Art

[0002] Various ceremonies such as weddings and farewell ceremonies are conventionally progressed by a master of ceremonies. At present, instead of a human master of ceremonies, an information processing apparatus is caused to act as an emcee proxy, that is, the information processing apparatus is caused to output voice acting as the master of ceremonies (see, for example, Patent Document 1).

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] Conventionally, voice data for voice output as an emcee proxy is generated using input sentences (text data). However, creating the input sentences requires a cumbersome work. For this reason, it is considered important to enable an information processing apparatus to more easily act as an emcee proxy.

[0005] An object of the present invention is to provide a technology that enables an information processing apparatus to more easily perform emcee proxy for a ceremony.

Means for Solving the Problem

[0006] An information processing apparatus according to one aspect of the present invention includes: an information input unit that inputs transmission information to be conveyed to participants participating in a ceremony; a sentence generation unit that uses the transmission information to generate sentences representing content that a master of ceremonies of the ceremony should speak in units of scenes; and a voice data generation unit that generates, in units of said scenes, voice data for voice output of the sentences generated by said sentence generation unit. [Effects of the Invention]

[0007] This invention makes it easier to have an information processing device act as the master of ceremonies. [Brief explanation of the drawing]

[0008] [Figure 1] This figure illustrates an example of an information processing device according to one embodiment of the present invention, and an overview of a master of ceremonies service provided by the information processing device. [Figure 2] This figure shows an example of the hardware configuration of a server, which is an information processing device according to one embodiment of the present invention. [Figure 3] This figure shows an example of a functional configuration implemented on a server, which is an information processing device according to one embodiment of the present invention. [Figure 4] This diagram illustrates examples of audio data generated during various stages of a funeral service where the master of ceremonies is acting, as well as examples of how to play that audio data. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a diagram illustrating an example of an information processing device according to one embodiment of the present invention, and an overview of a master of ceremonies service provided by the information processing device.

[0010] Information Processing Device 1 is installed by an organization such as a company that operates a facility where ceremonies such as weddings or funerals are held, for the purpose of providing a master of ceremonies service. This information processing device 1 is installed, for example, within a facility equipped with a ceremony venue, or it is implemented using a cloud service. Hereafter, the master of ceremonies service will be referred to as "this service." Also, to avoid confusion, the term "ceremony" will be explained using "funeral ceremony" as an example. For funeral ceremonies, a "Buddhist funeral ceremony" will be assumed. "Participants" is used as a general term for bereaved family members, relatives, and attendees who participate in the funeral ceremony. Note that the term "ceremony" is not limited to funeral ceremonies. The ceremony can be any ceremony that is conducted by a master of ceremonies according to a predetermined schedule.

[0011] To act as a substitute moderator, it is necessary to determine what the moderator should say and to output that determined content as audio. Therefore, the necessary functions are a generation function that determines what content to output as audio and generates audio data for that output, and a playback function that plays back the generated data. Information processing device 1 is equipped with both of these functions. The generation function is broadly divided into a text generation function that generates the content to be output as text, and an audio data generation function that generates audio data from the generated text.

[0012] As shown in Figure 1 as an example, the information processing device 1 has a functional configuration comprising a communication unit 1A, a transmission / reception processing unit 1B, a character recognition unit 1C, a natural language processing unit 1D, a text generation unit 1E, an image recognition unit 1F, a sound recognition unit 1G, a voice data generation unit 1H, and a playback processing unit 1J. The generation function is mainly realized by the character recognition unit 1C, the natural language processing unit 1D, the text generation unit 1E, and the voice data generation unit 1H. The playback function is mainly realized by the natural language processing unit 1D, the image recognition unit 1F, the sound recognition unit 1G, and the playback processing unit 1J.

[0013] The communication unit 1A enables communication with various devices. The terminal 2, scanner 3, sound system 4, multiple cameras (photography devices) 5, microphone (hereinafter abbreviated as "microphone") 6, and input device 7 shown in Figure 1 are all examples of various devices. Terminal 2 is an information processing device equipped with communication capabilities, such as a PC (Personal Computer) or a tablet PC. Hereafter, it will be referred to as "Input Terminal 2".

[0014] There is specific information that needs to be conveyed to participants at a funeral service. This information may include information about the deceased, information about the officiant, and a will or message from the deceased addressed to specific or unspecified individuals. In this service, other information that can be included is a photograph of the deceased (actually, an image such as a photograph in which the face exists) and audio recordings of the deceased speaking. Input terminal 2 is assumed to be a terminal used for inputting such information and transmitting it to information processing device 1. It should be noted that the deceased can leave audio recordings as a will. To avoid confusion, unless otherwise specified, audio recordings will refer to recordings that are not for the purpose of a will. Audio data left as a will will be referred to as "will audio data." The deceased is considered an important person at the funeral service.

[0015] In addition to the information that can be entered at input terminal 2, there is also specified information. This specified information mainly specifies the flow (schedule) of the funeral service and the content to be output by voice. The flow of the funeral service may be specified by the type of service pattern, including whether or not there are eulogies. Personal information of the person who will speak the eulogy, the designation of the condolence telegram to be read, etc., are treated as specified information that specifies the content to be output by voice. The entered specified information is transmitted to information processing device 1. If multiple types of services are to be supported, the type of service should be treated as specified information.

[0016] A will can be left as a handwritten or printed document. Condolence messages are delivered as printed documents. Scanner 3 is used to convert such documents into image data. The image data is transmitted to information processing device 1. Wills left as written documents will be referred to as "will documents" from now on to distinguish them from will audio data. The image data read by scanner 3 may also be transmitted to information processing device 1 via input terminal 2.

[0017] Data respectively transmitted from the input terminal 2 and the scanner 3 is received by the communication unit 1A of the information processing apparatus 1, and passed to the transmission / reception processing unit 1B. Transmission information and designation information are passed to the sentence generation unit 1E, and image data is passed to the character recognition unit 1C. The type of sentence represented by the image data, that is, whether the sentence is a will or a condolence telegram, is designated by, for example, the input terminal 2.

[0018] The character recognition unit 1C recognizes characters present on an image represented by the image data, and passes the recognition result to the natural language processing unit 1D. The recognition result includes position information representing positions where the recognized characters are present. The natural language processing unit 1D refers to the position information to identify characters that should be validated and the sequence of validated characters among the recognized characters, and generates natural sentences. Thereby, for example, in the case of a condolence telegram, the main text and the sender's name are extracted and formed into sentences. If it is a will, for example, all content is formed into sentences. The generated sentences are passed to the sentence generation unit 1E.

[0019] The sentence generation unit 1E generates sentences representing content to be spoken per scene along the flow of a formula specified by the designation information. Deceased information and officiant information input as transmission information are used for generating sentences. Thereby, sentences that are contextually suitable for a funeral service are automatically generated per scene. Note that sentence generation is performed per scene in order to enable timely emceeing substitution as required for each individual scene. Sentences generated per scene are hereinafter referred to as "scene sentences" for distinction. Sentences for the entire funeral service, that is, a group of scene sentences, are referred to as "overall sentences" for distinction. In many cases, scene sentences correspond to transition scenes that assume scene transitions in a funeral service, for example, a transition from an officiant entrance scene to an opening ceremony scene, a transition from a sutra reading end scene to a condolence speech reading start scene, and the like.

[0020] Deceased information and officiant information are normally known at the stage when it is decided that a funeral service will be held. In contrast, it is normally the case that the existence itself of condolence telegrams is not confirmed until they arrive. For this reason, in the present service, text generation for the condolence telegram reading scene is configured to be performed separately from other scenes. In other words, text generation for the entire funeral service is made possible to be performed in two separate stages.

[0021] The text generation itself is performed using AI (Artificial Intelligence). Text generation for scenes other than the condolence telegram reading scene is performed by prompt input including, for example, deceased information, officiant information, the pattern type of the ceremony schedule including the presence or absence of a condolence address, and personal information of the person giving the condolence address. Text generation for the condolence telegram reading scene may be performed by prompt input including the sender name of each condolence telegram to be read and a text containing the sender name of each condolence telegram. The text for the condolence telegram reading scene is, for example, a scene following the transition from the sutra reading end scene to the condolence telegram reading start scene (FIG. 4).

[0022] Each finally generated scene text is passed to the audio data generation unit 1H. The audio data generation unit 1H generates audio data for outputting the text as speech. Generation of the audio data completes preparation for the information processing apparatus 1 to act as an emcee. Since audio data is generated for each individual scene text, an audio data group is generated for one funeral service.

[0023] The final generated document serves as the master of ceremonies' script for the entire funeral service. The information used to generate such a document is the minimum information that the family organizing the funeral service needs to prepare. This is because information about the deceased and the officiant is difficult to obtain unless it is prepared by the family organizing the funeral service. By preparing such information and determining the content of the items to be requested as specified information, a document equivalent to the script for the entire funeral service can be generated. Therefore, regardless of whether or not Information Processing Device 1 is used as a substitute for the master of ceremonies, the amount of work required for the family to prepare the funeral service will be reduced. If Information Processing Device 1 is used as a substitute for the master of ceremonies, the family can perform that role more easily.

[0024] The audio data is intended for audio output on a scene-by-scene basis. Therefore, the playback processing unit 1J performs processing for playback of the audio data on a scene-by-scene basis. Audio output is controlled by the sound system 4. This sound system 4 includes, for example, multiple speakers, an amplifier that outputs audio signals to the speakers and emits sound, one or more displays, and a communication terminal capable of outputting analog audio signals and video (image) data. The communication terminal is capable of processing audio data transmitted from the information processing device 1 to generate analog audio signals and outputting the generated audio signals to the amplifier. It is also capable of converting image data transmitted from the information processing device 1 into image data for display and outputting it to the display. Thus, playback of audio data and output (display) of image data are realized by transmitting audio data and image data to the sound system 4.

[0025] The playback of scene-by-scene audio data by the playback processing unit 1J may be performed according to the instructions of the person in charge of the funeral service. However, it is not always possible for the person in charge to give these instructions appropriately. For this reason, this service allows for partial automatic playback of scene-by-scene audio data. To this end, the output format, which specifies either text display or audio output, and whether or not automatic playback is enabled are also included in the specified information.

[0026] If automatic playback is selected, this service sets start conditions for beginning playback of audio data for each scene in accordance with the progress of the funeral service. Therefore, playback of the audio data for each scene begins when it is determined that the corresponding start condition has been met. The multiple cameras 5, microphones 6, and input devices 7 shown in Figure 1 are all devices that can be used to determine whether or not the start condition has been met. They are used, for example, in the following applications. To distinguish them, the camera 5 used to film the entrance / exit 9 of the venue will be referred to as "camera 5A," and the camera 5 used to film the officiant 8 chanting will be referred to as "camera 5B."

[0027] The officiant 8 enters through the entrance 9, for example, before the start of the funeral service. Camera 5A is used to automatically recognize when the entrance 9 is open and when the officiant 8 is inside the open entrance 9. This automatic recognition is performed by the image recognition unit 1F. For this purpose, the video (image) data transmitted by the camera 5A is passed to the image recognition unit 1F via the communication unit 1A and the transmission / reception processing unit 1B. As a result, when the image recognition unit 1F recognizes the officiant 8 inside the open entrance 9 from the video data, this fact is notified to the playback processing unit 1J. Upon this notification, the playback processing unit 1J starts playing the audio data of the officiant's entrance. In this case, the automatic recognition of the officiant 8 inside the entrance 9 becomes the start condition for starting the playback of the audio data of the officiant's entrance. The determination of whether or not this start condition has been met can be made by either the image recognition unit 1F or the playback processing unit 1J. For this reason, they can be positioned as start condition determination units in this embodiment.

[0028] Camera 5B is used to determine the end of the scene where the officiant enters and when the officiant leaves. Camera 5B and microphone 6 are used to determine the end of the chanting. Upon entering, the officiant 8 takes a seat in the place where he will chant sutras. Therefore, the moment when the officiant 8 takes a seat can be considered the end of the officiant's entrance scene. After sitting and chanting sutras, the officiant 8 stands up and exits. Camera 5B makes it possible to determine whether or not the officiant 8 has performed such actions. For this purpose, the video data transmitted by camera 5B is also passed to the image recognition unit 1F via the communication unit 1A and the transmission / reception processing unit 1B. If the image recognition unit 1F recognizes, for example, the officiant 8 being seated or standing from the video data, this fact is notified to the playback processing unit 1J. The playback processing unit 1J can then start playing the audio data for the opening scene or the officiant's exit scene based on this notification. Therefore, the automatic recognition of the officiant 8 sitting down and the officiant 8 standing up can be used as the start conditions for starting the playback of the audio data for the opening scene and the officiant's exit scene. Here too, both the image recognition unit 1F and the playback processing unit 1J can be positioned as the start condition determination unit in this embodiment.

[0029] Just before the chanting of sutras ends, the officiant 8 usually rings a bell loudly several times in quick succession. Microphone 6 is designed to receive input such as the sound emitted from the bell, in addition to the voice of the officiant 8. Microphone 6 transmits, for example, digital sound data representing the input sound to the information processing device 1. The sound data received by the communication unit 1A is passed to the sound recognition unit 1G via the transmission / reception processing unit 1B.

[0030] The sound recognition unit 1G analyzes the sound data and determines whether or not a sound originating from a bell is present. If it determines that the sound is present, it further determines the loudness of the sound. If it determines that the loudness of the sound increased or decreased two or more times while the sound was present, it recognizes that the bell was rung continuously and then determines whether or not there is any subsequent audio that continues within the time set by the master 8. The set time is a predetermined time, assuming that the master 8 will inform the participants that the chanting has ended. As a result, after recognizing the sound of the bell being rung continuously, the sound recognition unit 1G recognizes the audio that continues within the time set by the master 8 as words indicating the end of the chanting. The sound recognition unit 1G's recognition of these two events in succession is the determination of the end of the chanting scene and one of the conditions for starting playback of the audio data for the next scene. The recognition result is notified to the playback processing unit 1J. Note that the end of the chanting may also be determined by recognizing the meaning of words spoken by the master 8. This determination can be performed, for example, by having the sound recognition unit 1G convert the words spoken by the master 8 into text data, and then having the natural language processing unit 1D process the resulting text data.

[0031] The playback processing unit 1J, for example, when the image recognition unit 1F recognizes that the officiant 8 is seated, and receives notification from the sound recognition unit 1G, determines that the conditions for considering the chanting to be finished have been met, and starts playing audio data for a scene that assumes the chanting has finished. At this time, the audio data to be started to be played is for expressing gratitude to the officiant 8 after the chanting (chanting scene), and for guiding the audience to the scene that follows the chanting, such as the eulogy scene. In this case, the playback processing unit 1J corresponds to the start condition determination unit in this embodiment.

[0032] Incense offering is usually started in the middle of chanting. However, in reality, it is difficult for anyone other than the officiant 8 to know when to start offering incense. The input device 7, placed near the officiant 8, is designed to allow the officiant 8 to instruct when to start offering incense. For this purpose, the input device 7 is equipped with a physical button 7A that should be operated to instruct the start of offering incense. When the input device 7 detects an operation on the physical button 7A, it transmits operation information to the information processing device 1 indicating that operation has been performed.

[0033] In addition to the input device 7 intended for operation by the master 8, there is also one intended for operation by the staff member. Therefore, the user of the input device 7 is pre-configured. For example, the input device 7 designated for the master 8 is configured to only accept the instruction to start the incense offering, while the input device 7 designated for the staff member is configured to accept the instruction to start playback of audio data for scenes that are not subject to automatic playback. Therefore, when an operation to the input device 7 is notified, the playback processing unit 1J refers to the configuration information of that input device 7 and identifies the content of the instruction resulting from that operation.

[0034] The operation information transmitted from the input device 7 is received by the communication unit 1A and passed to the playback processing unit 1J via the transmission / reception processing unit 1B. The playback processing unit 1J processes the operation information from the setting information of the input device 7 as an instruction to start the incense offering. As a result, the playback processing unit 1J starts playing back the audio data for the incense offering scene. This audio data is, for example, for guidance (explanation) about the incense offering that is about to begin.

[0035] Thus, this service not only allows the person in charge to instruct the playback of audio data on a scene-by-scene basis, but also enables automatic playback of most of the audio data. As a result, the information processing device 1 can perform the role of officiant for the ceremony even more easily. It is not necessary to process both the video data transmitted from cameras 5A and 5B. For example, when the master enters, the image data transmitted from camera 5A is processed. However, after the master 8 is recognized, it is no longer necessary to process the image data transmitted from camera 5A. Instead, it becomes necessary to process the image data transmitted from camera 5B to confirm that the master 8 is seated. For this reason, only one of the video data transmitted from cameras 5A or 5B is processed. This selection of which data to process can be performed, for example, by the playback processing unit 1J.

[0036] The starting conditions for each scene do not fundamentally change depending on the funeral service. For example, the officiant 8 enters through entrance 9, and the officiant 8 sits in a pre-designated place to chant sutras. The actions of the officiant 8 when the chanting is finished are also fundamentally the same. For this reason, the starting conditions for each scene can be automatically selected and set according to the predetermined schedule of the funeral service. As described above, depending on the starting conditions, what is positioned in the starting condition determination unit, that is, what determines whether or not the starting conditions have been met, may change.

[0037] Hereafter, embodiments of the present invention will be described in detail with further reference to the drawings. Figure 2 shows an example of the hardware configuration of a server, which is an information processing device according to one embodiment of the present invention. This hardware configuration example is just one example and is not particularly limited. For example, only one CPU (Central Processing Unit) 21 and one GPU (Graphics Processing Unit) 24 are shown, but multiple units of each may be installed.

[0038] Information processing device 1 is implemented, for example, as a server installed within a facility by an organization operating a facility, or as a server installed using a cloud service, for the purpose of providing this service. Therefore, the server is assigned the code "1". From now on, "information processing device" will also be referred to as "server".

[0039] As shown in Figure 2, Server 1 has a configuration in which a CPU 21, ROM (Read Only Memory) 22, RAM (Random Access Memory) 23, GPU 24, NIC (Network Interface Card) 25, auxiliary storage device 26, media drive 27, and I / FC (Interface Controller) group 28 are connected to bus 29. VRAM (Video RAM) 24A is connected to GPU 24.

[0040] NIC25 is a communication device capable of wired and wireless communication, enabling communication with input terminal 2, scanner 3, sound system 4, and other various devices 5-7. Thus, NIC25 corresponds to communication unit 1A shown in Figure 1. Here, it is assumed that the various devices 5-7 communicate directly with server 1.

[0041] The auxiliary storage device 26 is a device capable of permanently storing data, such as a hard disk drive or an SSD (Solid State Drive). The media drive 27 is a device on which the recording medium 27A can be attached and detached. The media 27A is such as a CD (Compact Disc)-ROM, DVD-ROM, DVD-RAM, etc.

[0042] The I / FC group 28 includes various I / FCs that enable communication with various peripheral devices, including the input device 28A and the display device 28B, or with external devices. The input device 28A and the display device 28B are temporarily connected to the I / FC group 28 as needed. The auxiliary storage device 26 stores the OS (Operating System) and various application programs that run on that OS as programs. Among these various application programs is an application program that enables the provision of this service. Hereafter, this application will be referred to as the "MC Substitute Service App".

[0043] ROM22 is also a device capable of permanently storing data, such as firmware and various other data. The CPU21 reads the firmware stored in ROM22 into RAM23 and executes it. Subsequently, the firmware reads the OS stored in auxiliary storage device 26 into RAM23 and executes it. Various application programs, including some matching service applications, are read into RAM13 by the OS and executed. The GPU24 can execute various application programs, including some MC proxy service applications, that are stored in auxiliary storage device 26 and read into VRAM24A.

[0044] The chairmanship代理 service application may be stored in a medium 27A for distribution. It may also be distributed via a network such as the Internet. In the case of distribution via a network, the chairmanship代理 service application may be stored in a recording medium that can be accessed directly or indirectly by an information processing apparatus that distributes the application. That is, the recording medium may be one that can be accessed directly or indirectly by another information processing apparatus capable of communicating with the information processing apparatus that performs distribution.

[0045] FIG. 3 is a diagram showing an example of a functional configuration implemented on a server, which is an information processing apparatus according to an embodiment of the present invention. This example functional configuration is mainly implemented by the CPU 21 and the GPU 24 respectively executing different parts of the aforementioned chairmanship代理 service application. The functional configuration is not particularly limited, and various modifications are possible.

[0046] As shown in FIG. 3, on the CPU 21 of the server 1, as functional configurations, in addition to the transmission / reception processing unit 1B, character recognition unit 1C, natural language processing unit 1D, sound recognition unit 1G, and reproduction processing unit 1J, a screen generation unit 211, a text generation instruction unit 212, an audio data generation instruction unit 213, a sound quality conversion instruction unit 214, a face video generation instruction unit 215, and a sound quality conversion learning instruction unit 216 are implemented.

[0047] On the GPU 24, as functional configurations, a text generation unit 241, an audio data generation unit 242, a sound quality conversion unit 243, a sound quality conversion learning unit 244, and a face video generation unit 245 are implemented. Artificial intelligence is used for all of these. While such a functional configuration is implemented on the CPU 21 and the GPU 24, an explanation information storage unit 261, a text storage unit 262, an audio data storage unit 263, an image storage unit 264, and a generation result storage unit 265 are secured as data storage areas on the auxiliary storage device 26.

[0048] Many of the monks requested by the bereaved family to serve as officiant 8 live relatively close to the venue or have visited it at least once. For this reason, in this embodiment, information about monks who are likely to be requested as officiant 8 is stored as explanatory information in the explanatory information storage unit 261. Using this explanatory information and RAG (Retrieval Augmented Generation), it is possible to generate a text that supplements the necessary information to introduce officiant 8 with minimal input such as the officiant's name. This makes it easier for the bereaved family to input officiant information. This explanatory information can also be used as an introductory text introducing officiant 8.

[0049] The document storage unit 262 is a storage area reserved for saving various types of documents (text data). The main documents are the generated scene documents and the recognition results of documents present on images represented by image data transmitted from the scanner 3. The documents saved as recognition results include documents such as wills and condolence telegrams.

[0050] The audio data storage unit 263 is a storage area reserved for saving various types of audio data. The audio data stored in this storage unit 263 can be broadly divided into data prepared by the bereaved family and data for the officiant. The audio data that the bereaved family can prepare is, as mentioned above, either a will left by the deceased (will audio data) or simply a recording of the deceased speaking (recorded audio data).

[0051] The image storage unit 264 is a storage area reserved for saving image data of the deceased's face, image data of condolence messages, and the like. The generation result storage unit 265 is a storage area reserved for storing generation result management information, which is information for managing the generation results of data for the substitute master of ceremonies. This generation result management information includes, for example, the output format, whether or not autoplay is enabled, the specified pattern type, the generated scene text according to the flow of the funeral represented by that pattern type, and URLs (Uniform Resource Locators) that represent the storage locations of the audio data generated from that scene text. The output format, whether or not autoplay is enabled, and the pattern type are input as specified information.

[0052] The various data stored in each memory unit 261-265 are actually read out and processed in RAM 23 or VRAM 24A. Data transfer between the CPU 21 and GPU 24 is also actually performed via RAM 23. Communication with the input terminal 2, scanner 3, sound system 4, and various devices (cameras 5, microphones 6, and input devices 7) 5-7 is performed via NIC 25. These are conveniently ignored in Figure 3. This will also be the case in subsequent explanations.

[0053] Communication with external devices, including input terminal 2, scanner 3, sound system 4, and various devices 5-7, may be conducted via wired or wireless connection. Server 1 is actually equipped with multiple communication devices with different communication standards, such as those for external devices and those for network communication. NIC25 in this context is positioned as a collection of such communication devices.

[0054] The parts 241-245 implemented on GPU24 have the following functions. The text generation unit 241 is an AI for text generation. As described above, it generates text for scenes other than the reading of condolence messages based on prompt input including transmission information and specified information. In the reading of condolence messages scene, for example, specified information that specifies the text generation for the reading of condolence messages scene, the name of the sender of the condolence message to be read, and the recognized text of the condolence message are entered as prompt input.

[0055] The voice data generation unit 242 is a generation AI that converts text (text data) into speech (voice data). The sound quality conversion unit 243 is an AI that generates audio data to output audio in the deceased person's sound quality. Recordings of the deceased person are used to reproduce their sound quality. A well-known model that makes such reproduction of sound quality, or voice cloning, possible is, for example, the "adaptive limited Boltzmann machine".

[0056] Adaptive restricted Boltzmann machines use the voices of multiple speakers and simultaneously estimate speaker-independent and speaker-dependent parameters to separate "phonological" and "speaker" information. This separation allows for voice transformation by simply switching the speaker information to the desired one while keeping the phonological information unchanged for the input voice. This model can reproduce the voice of a target speaker from audio data of about 5 seconds. The sound quality conversion learning unit 244 receives the deceased person's voice data (recorded data) as input and performs learning to realize phonological information on the sound quality conversion unit 243 with the deceased person as the target speaker.

[0057] The facial video generation unit 245 uses image data representing the deceased's face and audio data representing what they are saying to generate a moving image (facial video) that simulates the facial movements of the deceased when they speak the content represented by the audio data. The audio data representing what the deceased is saying can be audio data generated by the sound quality conversion unit 243 or audio data of a will.

[0058] The transmission / reception processing unit 1B, implemented on the CPU 21, performs processing for sending and receiving data between the input terminal 2, scanner 3, sound system 4, and various devices 5-7. The screen generation unit 211 generates screens for various data inputs. The generated screens are transmitted to the input terminal 2 by the transmission / reception processing unit 1B. As a result, the person using the input terminal 2 can input transmission information and specified information, upload files containing image data representing the deceased's face and audio data files, specify condolence messages to be read, instruct the scanner 3 to read the condolence messages, and instruct the generation of the entire text on the transmitted screen. Thus, the information input unit in this embodiment is implemented by the NIC 25, the transmission / reception processing unit 1B, and the screen generation unit 211. The data acquisition unit in this embodiment is implemented by the NIC 25 and the transmission / reception processing unit 1B.

[0059] Image data generated by reading the condolence telegram and transmitted from scanner 3 is stored, for example, in image storage unit 264 by transmission / reception processing unit 1B, and this is notified to character recognition unit 1C. Upon notification, character recognition unit 1C recognizes the characters represented by the stored image data and passes the recognition result, along with location information, to natural language processing unit 1D. Natural language processing unit 1D refers to the location information to identify the connections between the recognized characters and forms a sentence. The sentence thus obtained is stored in sentence storage unit 262.

[0060] The text generation instruction unit 212 instructs the text generation unit 241 to generate text. Text for each scene other than the reading of the condolence telegram is generated together based on prompt inputs, such as transmission information and specified information. Text for the reading of the condolence telegram is generated based on prompt inputs, such as specified information to generate text for the reading of the condolence telegram, the name of the sender of the condolence telegram to be read, and the recognized text of the condolence telegram. Thus, the text generation unit 1E shown in Figure 1 is realized by the text generation instruction unit 212 and the text generation unit 241. Note that the text actually generated is basically a set of predefined phrases.

[0061] Each generated scene text is saved in the text storage unit 262 by, for example, the text generation instruction unit 212. The text generation instruction unit 212 generates generation result management information and stores it in the generation result storage unit 265, and also updates the generation result management information by adding the URLs of each scene text saved in the text storage unit 262.

[0062] As described above, the image recognition unit 1F is primarily used to automatically recognize the presence or absence of Master 8, the state of Master 8, etc. The recognition result is passed to the playback processing unit 1J. As described above, the sound recognition unit 1G is for automatically recognizing the sound of the bell that the master 8 rings multiple times in succession at the end of the chanting. The recognition result is passed to the playback processing unit 1J.

[0063] The audio data generation instruction unit 213 instructs the audio data generation unit 242 to generate audio data. Audio data is generated for each scene sentence. The scene sentence is, for example, used as a prompt input to generate audio data. The generated audio data is stored in the audio data storage unit 263 by the audio data generation instruction unit 213. The audio data generation instruction unit 213 also updates the generation result management information stored in the generation result storage unit 265 by adding the URL of the audio data storage location.

[0064] The sound quality conversion instruction unit 214 instructs the sound quality conversion unit 243 to generate audio data that reproduces the deceased's voice. The input audio data is the audio data generated from the recognition result of the will. The audio data that reproduces the deceased's voice is stored in the audio data storage unit 263 by the sound quality conversion instruction unit 214, for example. The generation result management information stored in the generation result storage unit 265 is updated by the sound quality conversion instruction unit 214, for example, by adding the URL of the audio data storage location. Therefore, the audio data generation unit 1H shown in Figure 1 is implemented by the audio data generation instruction unit 213, the sound quality conversion instruction unit 214, and the audio data generation unit 242.

[0065] The face video generation instruction unit 215 instructs the face video generation unit 245 to generate a video of the deceased person's facial movements as they speak. To this end, the face video generation unit 245 receives an image representing the deceased person's face and audio data generated by the sound quality conversion unit 243. Based on the face image represented by the image, the face video generation unit 245 reproduces the changes in facial expression that produce the sound output by the audio data. The generated face video is saved in the image storage unit 264 by the face video generation instruction unit 215. The generation result management information stored in the generation result storage unit 265 is updated, for example, by the face video generation instruction unit 215, which adds the URL of the save location of the face video (video data). In this embodiment, the moving image generation unit is realized by the face video generation instruction unit 215 and the face video generation unit 245.

[0066] Audio output in the deceased's own voice quality can be expected to evoke memories of the deceased for those who knew them. Such audio output of a will can be expected to make listeners feel as if the deceased is actually speaking. If a video of the deceased's face is displayed along with such audio output, it can be expected that the feelings of remembrance will be intensified through both auditory and visual means. This, in turn, can be expected to help listeners remember the contents of the will more strongly.

[0067] The playback processing unit 1J refers to the generation result management information stored in the generation result storage unit 265 and performs processing to display scene text or play audio data according to the specified output format. The sound quality conversion learning instruction unit 216 instructs the sound quality conversion learning unit 244 to perform learning using the recorded data. Through this learning, the sound quality conversion unit 243 becomes capable of generating audio data that reproduces the voice of the deceased.

[0068] Figure 4 illustrates examples of the content of audio data generated at each stage of a funeral service where the master of ceremonies is acting, and examples of how to play back that audio data. Referring to Figure 4, we will now specifically explain the content of the audio output generated by the audio data at each stage of the funeral service, as well as the playback processing performed by the playback processing unit 1J for playing back the audio data at each stage.

[0069] The example of the funeral service flow shown in Figure 4 involves the following sequence of events: entrance of the officiant → opening of the service → chanting of sutras including incense offering by participants → reading of condolence messages → departure of the officiant → farewell ceremony → presentation of the will → address by the chief mourner → closing of the service. This sequence of events in the funeral service can be specified using designated information.

[0070] When the master guide enters, it is desirable to provide an explanation before the master guide 8 actually enters, including information about the master guide 8's entry and any precautions to take during their entry. Therefore, in the scene before the master guide enters, audio data is generated to output the scene text for this explanation. This audio data is played back by the person in charge, such as by operating the physical button 7A on the input device 7. To this end, when the playback processing unit 1J is notified by the transmission / reception processing unit 1B that the physical button 7A on the input device 7 has been operated, it starts playing back the audio data for the scene before the master guide enters. The audio data for that scene is identified by referring to the generation result management information. Playback of this audio data is performed by having the transmission / reception processing unit 1B transmit the audio data to the sound system 4. The condition for starting playback of the audio data for the scene before the master guide enters is the operation of the physical button 7A on the input device 7 by the person in charge.

[0071] After the playback processing unit 1J starts outputting audio data, it instructs the image recognition unit 1F to recognize the master 8 using the image data transmitted from the camera 5A. Then, the playback processing unit 1J waits for notification from the image recognition unit 1F that the master 8 has been recognized, and then starts playing the audio data for the master's entrance scene. The audio output during the playback of the audio data for that scene includes the master 8's entrance and an introduction of the master 8. The condition for starting the playback of the audio data for the master's entrance scene is the recognition of the master 8.

[0072] In Figure 4, the content generated (played) upon the master's entrance is described as "Entrance guidance (instructed by the person in charge) → Introduction of the master (automatic playback)." Here, "instructed by the person in charge" means playback based on the instructions of the person in charge, and "automatic playback" means automatic playback based on automatic recognition. "→" indicates a transition between scenes. Here, it represents the transition from the scene before the master's entrance to the scene of the master's entrance. This transition is performed automatically when a substitute master of ceremonies is in place.

[0073] Upon entering, the officiant 8 moves to the place where he will chant sutras and takes a seat. Therefore, after receiving notification from the image recognition unit 1F that the officiant 8 has been recognized, the playback processing unit 1J instructs the image recognition unit 1F to recognize the seated state of the officiant 8 using the image data transmitted from the camera 5B. As a result, the playback processing unit 1J waits for notification from the image recognition unit 1F that the seated state of the officiant 8 has been recognized, and then begins playback of the audio data for the opening ceremony. The opening address is output as audio during the playback of this audio data. The opening address contains information about the deceased. The condition for starting playback of the audio data for the opening ceremony is the recognition of the seated state of the officiant 8.

[0074] The seated master (8) will, for example, announce that they will begin chanting and explain any precautions to take during the chanting before starting. The time between sitting down and actually beginning chanting varies depending on the master (8) and the circumstances at the time. For this reason, no audio output is provided at the start of the chanting.

[0075] After the master 8 begins chanting, he operates the physical button 7A on the input device 7 at the appropriate time to start the incense offering. This operation causes the playback processing unit 1J to play audio data for the incense offering scene. The audio output from the playback of the audio data includes explanations about the incense offering, etc. The condition for starting the playback of this audio data is the operation of the physical button 7A on the input device 7 by the master 8.

[0076] As described above, when the master 8 finishes chanting, he rings a bell repeatedly. After that, he informs the participants that the chanting has finished. The sound of the bell and the voice of the master 8 that follows are converted into sound data by microphone 6 and sent to server 1. As a result, the sound recognition unit 1G recognizes the end of the chanting by sound. The image recognition unit 1F, which uses image data from camera 5B to recognize the seated position of the master 8, is used to correct any misrecognition of the end of the chanting by the sound recognition unit 1G. Thus, the sound recognition unit 1G's recognition of the end of the chanting becomes the starting condition for playing the audio data for the next scene, and the playback processing unit 1J plays the audio data for the next scene. The audio data that starts playback is a thank you to the master 8 who has finished chanting (this is also actually part of the chanting scene), and it is intended to guide the transition to the scene following the chanting, in this case the scene of reading condolence messages. After the audio output of the announcement ends, the playback processing unit 1J transitions to the condolence telegram reading scene and automatically starts playing the audio data for reading the condolence telegram.

[0077] If there are many condolence messages, it will not be possible to read all of them within the limited time. For this reason, only the condolence messages designated by the bereaved family will be read. For condolence messages that are not read, only the sender's name will be announced, for example. After the playback of the audio data for reading the condolence messages is finished, the playback processing unit 1J automatically starts playing the audio data for announcing the remaining condolence messages.

[0078] After the audio data for introducing the condolence messages has finished playing, the playback processing unit 1J waits for notification from the image recognition unit 1F that the officiant 8 has recognized a standing position based on the image data from the camera 5B, and then starts playing the audio data for the officiant's departure. Thus, the change from a seated state to a standing state of the officiant 8 is the trigger condition for starting the playback of the audio data for the officiant's departure. The audio output from the playback audio data includes, for example, a statement that the officiant 8 is leaving, as well as explanations of precautions to take when the officiant leaves.

[0079] The end of the scene where the master departs can be considered to be the point when the image recognition unit 1F can no longer recognize the master departs 8 as they exit the entrance 9, based on the image data transmitted by camera 5A. If the scene that follows is the farewell ceremony, playback of the audio data for that scene may be started only after the image recognition unit 1F notifies the playback processing unit 1J that the master departs 8 can no longer be recognized. In other words, the starting condition for playback of the audio data for the farewell ceremony scene may be when the image recognition unit 1F can no longer recognize the master departs 8.

[0080] During the farewell ceremony, flowers are placed in the coffin as part of a farewell flower arrangement, and mementos are placed inside. Finally, nails are driven into the coffin. The audio output for this scene is intended to provide guidance to ensure that such a farewell ceremony is carried out smoothly. In this embodiment, the audio data for the scene following the farewell ceremony is set to start playback when the person in charge operates the physical button 7A on the input device 7. This is because the farewell ceremony is the final farewell to the deceased, and it is considered desirable to take into consideration the emotional state of the participants.

[0081] In the example shown in Figure 4, the scene of the farewell ceremony is followed by the scene of the presentation of the will. The audio data output as the will in this scene is either audio data prepared in advance by the deceased as a will, or a written will reproduced in the deceased's voice. If a facial video is displayed, the facial video is altered based on the audio data to show the deceased's face. After the playback of such audio data and facial video is finished, the playback processing unit 1J automatically starts playing the audio data for the next scene.

[0082] As shown in Figure 4, if the next scene is the scene for the chief mourner's speech, the audio output from the audio data is, for example, to inform the audience that the chief mourner will be giving a speech. The chief mourner then only needs to deliver their speech after the audio data has finished playing. Alternatively, the transition to the chief mourner's speech scene may be initiated by the chief mourner operating a physical button 7A or the like on the input device 7.

[0083] The voice for the chief mourner's speech is converted into sound data by microphone 6 and sent to server 1. The sound data received by server 1 is passed to sound recognition unit 1G and used to recognize the end of the speech. This end recognition could be performed, for example, when no voice can be recognized from the sound data for a certain period of time. This is based on the fact that the chief mourner usually turns off the voice input of microphone 6 after finishing their speech. The playback processing unit 1J starts playback of the next closing speech's audio data, using the notification of end recognition from sound recognition unit 1G as the starting condition. The audio output from this audio data is for the closing remarks. Note that the recognition of the end of the speech may also be performed comprehensively by the playback processing unit 1J, including the results of semantic interpretation by the natural language processing unit 1D.

[0084] Thus, in this embodiment, the system supports automatic playback of most of the audio data in each scene. Therefore, even an inexperienced person can easily act as the moderator. Even if the person in charge gives most of the instructions for playing the audio data, they only need to give those instructions, so the workload for acting as the moderator is very light. As a result, even if the audio data is played according to the instructions of the person in charge, acting as the moderator can be easily done.

[0085] While this embodiment primarily envisions the use of voice output for acting as a master of ceremonies, it can also be used to create scripts for the master of ceremonies to read. When used for script creation, the master of ceremonies may be allowed to switch between the displayed text for each scene. Even when used for script creation, it eliminates the need for the bereaved family, including the master of ceremonies, to create the script, or at least reduces the effort involved. This makes it easier to conduct the funeral service.

[0086] In this embodiment, a funeral service is assumed, but as described above, it can be widely adapted to any ceremony that follows a predetermined schedule. For ceremonies of the same type, the starting conditions for scenes that exist in other ceremonies can be basically the same. In other words, the starting conditions can be standardized except for special scenes that are limited to that particular ceremony. As a result, it can be widely adapted to both the generation of audio data, including text generation, and the playback of the generated audio data, for ceremonies that follow a predetermined schedule.

[0087] Furthermore, in this embodiment, both the generation function and the playback function are installed on a single information processing device (server), but these functions may be installed on different information processing devices. The generation function and the playback function may also be implemented by distributing the necessary functions across multiple information processing devices. Including these modifications, this embodiment is capable of various variations. [Explanation of Symbols]

[0088] 1 Information processing unit (server), 1A Communication unit, 1B Transmission / reception unit, 1C Character recognition unit, 1D Natural language processing unit, 1E Text generation unit, 1F Image recognition unit, 1G Sound recognition unit, 1H Audio data generation unit, 1J Playback processing unit, 2 Input terminal, 3 Scanner, 4 Sound system, 5 Camera, 6 Microphone, 7 Input device.

Claims

1. An information input unit for inputting information to be conveyed to participants attending the ceremony, A text generation unit that uses the aforementioned communication information to generate sentences on a scene-by-scene basis that represent the content that the moderator of the aforementioned formula should speak, A voice data generation unit generates voice data for audio output of the text generated by the text generation unit on a scene-by-scene basis, An information processing device equipped with the following features.

2. The information input unit is capable of inputting other sentences representing words from the important person in the formula as the transmitted information. The aforementioned audio data generation unit is capable of generating audio data for the audio output of the other text. The system further includes a sound quality conversion unit that, when the sound data generation unit generates the sound data from the other text, can convert the sound data into another sound data that is output with the sound quality of the important person. The information processing apparatus according to claim 1.

3. The aforementioned information input unit is capable of inputting image data representing the facial image of the important person, The system further includes a motion image generation unit capable of generating motion images that dynamically change the face image in accordance with the other audio data when the aforementioned image data is input. The information processing apparatus according to claim 2.

4. A data acquisition unit that acquires data from various devices, including multiple camera equipment, installed at the venue of the ceremony, A start condition determination unit processes data from the various devices and determines whether the start conditions for starting playback of the audio data generated for each scene in the formula have been met, The system further includes a playback processing unit that plays back the audio data generated for each scene based on the scene-specific determination result by the start condition determination unit. The information processing apparatus according to any one of claims 1 to 3.

5. A data acquisition unit that acquires data from various devices, including multiple camera equipment, installed at the venue of the ceremony, A start condition determination unit processes data from the various devices mentioned above and determines whether the start conditions for starting playback of the audio data generated on a scene-by-scene basis have been met for each scene in the formula. A playback processing unit plays back the audio data generated for each scene based on the scene-specific determination result by the start condition determination unit, An information processing device equipped with the following features.

6. A program for causing an information processing device to function as the information processing device described in claim 1.

Images