Sutra recitation support device, sutra recitation support method and program

Abstract

To provide a sutras-reading support device which can present, in real time, the evaluation of the voice and the evaluation of the emotion of a user who is reading sutras.SOLUTION: The sutras-reading support device includes: a presentation unit for presenting a desirable speed or a desirable length of time of reading a passage from sutras when a user reads a passage from sutras, by using an image or voice; a music interval evaluation unit for evaluating the music interval of the reading voice when the user reads a passage from sutras on a letter section-by-letter section basis of the passage from sutras; an emotion evaluation unit for evaluating the emotion of the user on the basis of a continuously obtained image of the user who reads the passage from sutras on a letter section-by-letter section basis; and an output unit for outputting the evaluation of the music interval and the evaluation of the emotion every time a letter section is read.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a sutra recitation assistance device, a sutra recitation assistance method, and a program. [Background technology]

[0002] In recent years, chanting sutras, which are Buddhist scriptures, has been attracting attention as a form of training to improve mental health. When chanting sutras, it is preferable that the sutras be recited at an appropriate pitch. Furthermore, it is preferable that sutra chanting as training is performed under stable emotions.

[0003] Patent document 1 describes an audio information display device that allows a speaker to understand the difference between the speaker's voice and the model voice by displaying the speaker's voice waveform data reading a string of characters and the model voice waveform data on a display. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-156943 Summary of the Invention [Problem to be solved by the invention]

[0005] To support sutra chanting as training, it is necessary to evaluate the reciter's emotions in real time in addition to the pitch of their voice.

[0006] The present invention has been made to solve the above-mentioned problems, and aims to provide a sutra recitation assistance device, a sutra recitation assistance method, and a program that enable evaluation of the recitation voice and emotions of a user reciting sutras to be presented in real time. [Means for solving the problem]

[0007] A sutra recitation assistance device according to an embodiment of the present invention is characterized by having a presentation unit that presents, by means of an image or sound, the speed at which a user should recite a sutra or the period during which the sutra should be recited; a pitch evaluation unit that evaluates the pitch of the recitation voice when the user recites a sutra for each character segment of the sutra; an emotion evaluation unit that evaluates the emotion of the user for each character segment based on successively obtained images of the user reciting the sutra; and an output unit that outputs an evaluation of the pitch and evaluation of the emotion each time a character segment is recited.

[0008] Furthermore, it is preferable that the sutra chanting assistance device further has a reception unit that receives the user's selection of a sect, and the presentation unit presents the speed at which the user should recite the sutra or the period during which the sutra should be recited, by presenting the timing for the user to take a breath or by outputting the sound of a mokugyo at predetermined intervals, depending on the selection of the sect.

[0009] It is also preferable that the output unit displays the pitch evaluation and the emotion evaluation in different display formats in a single display area.

[0010] Preferably, the output unit displays the pitch evaluation and the emotion evaluation using a single flame color and shape.

[0011] It is also preferable that the sutra recitation assistance device further has an emotion determination unit that determines the emotions of the user reciting the sutra text, a proficiency evaluation unit that evaluates the user's proficiency in reciting the sutra based on a combination of pitch evaluation, emotional evaluation, and emotion determination, and a proficiency output unit that outputs the evaluation of the proficiency.

[0012] A sutra recitation assistance method according to an embodiment of the present invention is a sutra recitation assistance method executed by a sutra recitation assistance device, characterized in that it includes the steps of: presenting, by image or audio, the speed at which a user should recite a sutra or the period during which the sutra should be recited; evaluating the pitch of the recitation voice when the user recites the sutra for each character segment of the sutra; evaluating the user's emotions for each character segment based on successively obtained images of the user reciting the sutra; and outputting an evaluation of the pitch and the evaluation of the emotions each time a character segment is recited.

[0013] A program according to an embodiment of the present invention is characterized in that it causes a computer to perform the following: present the speed at which a user recites a sutra or the period during which the sutra should be recited using images or audio; evaluate the pitch of the audio of the user reciting the sutra for each character segment of the sutra; evaluate the user's emotions for each character segment based on successively obtained images of the user reciting the sutra; and output an evaluation of the pitch and the emotion each time a character segment is recited. [Effects of the Invention]

[0014] The sutra recitation assistance device, sutra recitation assistance method and program according to the present invention make it possible to present an evaluation of the recitation voice and emotions of a user reciting sutras in real time. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a functional block diagram of a sutra recitation assistance device 1. FIG. [Figure 2] FIG. 10 is a diagram showing the data structure of a denomination table T1. [Figure 3] FIG. 10 is a diagram showing a setting screen G1. [Figure 4] FIG. 10 is a diagram showing a first sutra reading screen G2. [Figure 5] 10 is a schematic diagram for explaining changes in an image displayed in a period display area G22. FIG. [Figure 6] FIG. 10 is a diagram showing a second sutra reading screen G3. [Figure 7] FIG. 10 shows a proficiency evaluation screen G4. [Figure 8] FIG. 10 is a flowchart showing the flow of sutra recitation support processing. [Figure 9] FIG. 10 is a flow chart showing the flow of the first sutra recitation process. [Figure 10] FIG. 10 is a schematic diagram for explaining a method for evaluating a pitch. [Figure 11] FIG. 1 is a schematic diagram for explaining a method for evaluating emotions. [Figure 12] FIG. 10 is a flow chart showing the flow of the first sutra recitation process. [Figure 13] FIG. 10 is a flowchart showing the flow of a proficiency evaluation process. DETAILED DESCRIPTION OF THE INVENTION

[0016] Various embodiments of the present invention will be described below with reference to the drawings. Please note that the technical scope of the present invention is not limited to these embodiments, but extends to the inventions set forth in the claims and their equivalents.

[0017] FIG. 1 is a functional block diagram of a sutra recitation assistance device 1 according to an embodiment of the present invention. The sutra recitation assistance device 1 presents the tempo or rhythm of the recitation by presenting the speed at which the user should recite the sutra (recitation speed) or the period during which the sutra should be recited (recitation period). The sutra recitation assistance device 1 also evaluates in real time the pitch of the user's recitation voice and the user's emotions as the user recites the sutra according to the presented tempo or rhythm. By checking the output evaluation, the user can continue reciting at an appropriate pitch while maintaining appropriate emotions. The sutra recitation assistance device 1 also determines the emotions of the user reciting the sutra, and evaluates the user's level of proficiency in reciting the sutra based on the evaluation of pitch, emotion, and emotion. By checking the evaluation of the user's level of proficiency in reciting, the user can challenge themselves to recite sutras at an appropriate level of difficulty. The sutra recitation assistance device 1 is an information processing device such as a PC (Personal Computer), a mobile phone, or a smartphone, and has a memory unit 11, an imaging unit 12, an audio input unit 13, an audio output unit 14, a display unit 15, an operation unit 16, a processing unit 17, etc.

[0018] The storage unit 11 is configured to store programs or data, and includes, for example, a semiconductor memory device. The storage unit 11 stores an operating system program, a driver program, an application program, data, etc., used in processing by the processing unit 17. Programs are installed into the storage unit 11 from a computer-readable, non-transitory, portable storage medium such as a CD-ROM (Compact Disc Read Only Memory) or a DVD-ROM (Digital Versatile Disc Read Only Memory).

[0019] The imaging unit 12 is configured to capture an image of the user reciting sutras and includes a camera. The camera includes an imaging optical system for forming an image on a light-receiving surface, photoelectric conversion elements such as CCD (Charge Coupled Device) sensors that are arranged two-dimensionally on the light-receiving surface and output an electrical signal according to the amount of incident light, and an image generation circuit that generates an image based on the output of the photoelectric conversion elements. The imaging unit 12 generates an image based on a control signal supplied from the processing unit 17 and supplies the generated image to the processing unit 17.

[0020] The voice input unit 13 is configured to acquire the voice of the user reciting the sutra, and is equipped with a microphone. The voice input unit 13 converts vibrations caused by the voice into an electrical signal. The voice input unit 13 converts the voice into an electrical signal based on a control signal supplied from the processing unit 17, and supplies the electrical signal to the processing unit 17.

[0021] The audio output unit 14 is configured to output audio and includes a speaker. The audio output unit 14 converts an electrical signal supplied from the processing unit 17 into mechanical vibrations to output audio.

[0022] The display unit 15 is configured to display an image, and includes, for example, a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 15 displays an image based on the display data supplied from the processing unit 17.

[0023] The operation unit 16 is configured to accept operations from the user and includes, for example, a keyboard, a keypad, and a mouse. The operation unit 16 includes a touch panel and may be integrated with the display unit 15. The operation unit 16 generates an operation signal in response to the user's operation and supplies it to the processing unit 17.

[0024] The processing unit 17 is a device that comprehensively controls the operation of the sutra recitation assistance device 1, and includes one or more processors and their peripheral circuits. The processing unit 17 includes, for example, a CPU (Central Processing Unit). The processing unit 17 may include a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc. The processing unit 17 controls the operation of each component and executes various processes so that the various processes of the sutra recitation assistance device 1 are executed in an appropriate order based on the programs stored in the storage unit 11 and the operation signals from the operation unit 16.

[0025] The processing unit 17 includes, as its functional blocks, a reception unit 171, a presentation unit 172, a pitch evaluation unit 173, an emotion evaluation unit 174, an output unit 175, a feeling determination unit 176, a proficiency evaluation unit 177, and a proficiency output unit 178. Each of these units is a functional module realized by a program executed by the processing unit 17. Each of these units may be implemented in the sutra recitation assistance device 1 as firmware.

[0026] 2 is a diagram showing the data structure of the sect table T1 stored in the storage unit 11. The sect table T1 stores sects, sutras, methods, and sutra categories in association with one another.

[0027] The sect and sutra are information identifying a Buddhist sect and a type of sutra, respectively. Because the sutra a user should recite varies depending on the user's sect, a sutra is associated with each sect. Because some sutras are used in multiple sects, the same sutra may be associated with multiple sects. The method is information indicating the method for determining the tempo or rhythm of the sutra chanting, and is either the breath method or the mokugyo method. The breath method is a method in which a recitation period during which the user should recite the sutra and a breathing period during which the user can take a breath are predetermined. The rhythm of the sutra chanting is determined by the user reciting a passage of sutra during the recitation period and taking a breath during the breathing period. The mokugyo method is a method in which the sound of a mokugyo is output at a predetermined tempo while the user is reciting the sutra. By reciting the sutra while listening to the sound of the mokugyo, the user's recitation speed is guided to the tempo of the mokugyo sound, thereby determining the tempo of the sutra chanting. The sutra classification is information indicating each section of a sutra segmented based on the division positions predetermined for each sutra, and the identification information of each section is associated with the string of sutra text contained in each section.

[0028] FIG. 3 is a diagram showing an example of a setting screen G1 displayed on the display unit 15. The setting screen G1 is a screen for setting the user's sect and sutra chanting speed. The setting screen G1 is displayed, for example, when the operation unit 16 receives an operation to execute a sutra chanting assistance application program. The setting screen G1 includes a sect selection area G11, a speed selection area G12, and a sutra chanting start object G13.

[0029] The sect selection area G11 is an area for selecting the user's sect. The sutra text that a user should recite and the method of reciting the sutra vary depending on the user's sect. Therefore, before reciting the sutra, the user selects his / her own sect from among multiple sects. For example, the sect selection area G11 includes multiple objects corresponding to each of the multiple sects, and the user selects the object corresponding to his / her own sect from among the multiple objects. The sect selection area G11 displays the selected object in an emphasized manner. In the example shown in Figure 3, the object corresponding to "Jodo Shinshu" selected by the user is displayed in an emphasized manner.

[0030] The speed selection area G12 is an area for selecting the chanting speed. For example, the speed selection area G12 includes radio buttons that allow the user to select one speed from a plurality of preset speeds. In the example shown in FIG. 3, the speed selection area G12 allows the user to select one of "Excitement" corresponding to a fast speed, "Everyday" corresponding to a medium speed, and "Calm" corresponding to a slow speed. The speed selection area G12 may also include an object that allows the user to specify the chanting speed numerically.

[0031] If the timing of taking a breath when chanting is specified, the faster the chanting speed, the shorter the recitation period. If the timing of taking a breath when chanting is not specified, the faster the chanting speed, the shorter the interval between the mokugyo sounds output from the audio output unit 14. In the example shown in Figure 3, the user has selected "Excitement," which corresponds to a fast speed.

[0032] The sutra chanting start object G13 is an object for starting sutra chanting. After selecting the sect and the sutra chanting speed, the user starts the sutra chanting by selecting the sutra chanting start object G13.

[0033] 4 is a diagram showing an example of the first sutra recitation screen G2 displayed on the display unit 15. The first sutra recitation screen G2 is a screen for reciting sutras using the breath-taking method. The first sutra recitation screen G2 is displayed when the sutra recitation start object G13 is selected while a sect using the breath-taking method is selected in the sect selection area G11 of the setting screen G1. The first sutra recitation screen G2 includes a sutra text display area G21, a period display area G22, and an evaluation display area G23.

[0034] The sutra display area G21 displays the sutra that the user is reciting. For example, the sutra display area G21 displays the sutra that the user is currently reciting by scrolling an image in which the sutra is displayed vertically to the right at a speed that corresponds to the recitation speed. In the example shown in Figure 4, the sutra is broken up into lines at the end of each verse, so the sutra display area G21 scrolls the image of the sutra by one line between each breathing period and recitation period.

[0035] The period display area G22 displays the recitation period during which the user should recite the sutra displayed in the sutra display area G21 and the breath period during which the user should take a breath. In the example shown in Fig. 2, the period display area G22 indicates whether the current period is a recitation period or a breath period using a character string ST, and displays the progress of the recitation period and the breath period using a circular progress bar PB arranged to surround the character string ST.

[0036] FIG. 5 is a schematic diagram illustrating changes in the image displayed in the period display area G22. First, a breathing period begins. Simultaneously with the start of the breathing period, the character string "Inhale" is displayed as the character string ST, as shown in the upper left of FIG. 5. At this time, the length of the progress bar PB is zero. As time passes from this state, the progress bar PB extends clockwise from the 12 o'clock position, as shown in the upper center of FIG. 5. As more time passes from this state and the end time of the breathing period arrives, the tip of the progress bar PB reaches the 12 o'clock position, as shown in the upper right of FIG. 5. When the breathing period ends, the reading period begins. Simultaneously with the start of the reading period, the character string "Read" is displayed as the character string ST, as shown in the lower right of FIG. 5. At this time, the length of the progress bar PB is zero. As time passes from this state, the progress bar PB again extends clockwise from the 12 o'clock position, as shown in the lower center of FIG. 5. As time passes from this state and the end of the recitation period arrives, the tip of the progress bar PB reaches the 12 o'clock position, as shown in the lower left of Figure 5. When the recitation period ends, a breathing period begins again. The period display area G22 repeats the above display until the recitation ends.

[0037] The length of the recitation period varies depending on the recitation speed set on the setting screen G1. For example, the length of the recitation period is 2 seconds when "Excitement" is selected, 4 seconds when "Everyday" is selected, and 6 seconds when "Calm" is selected. The length of the breathing period is constant, for example, 2 seconds, regardless of the recitation speed.

[0038] In this way, the sutra recitation assistance device 1 presents the recitation period and the breathing period by means of images.

[0039] 4, the evaluation display area G23 displays the pitch evaluation and the emotion evaluation using the color and shape of a lit candle flame FL. For example, the evaluation display area G23 displays the pitch evaluation using the shape of the flame FL, and displays the emotion evaluation using the color of the flame FL.

[0040] It is considered preferable to read sutras without any intonation. Therefore, the smaller the fluctuation in pitch during the recitation period, the better the pitch evaluation. The flame FL is displayed so that its shape periodically expands and contracts, and the smaller the fluctuation in pitch, the smaller the range of expansion and contraction. When the fluctuation in pitch is small, the flame FL may be displayed so that its shape remains constant regardless of the passage of time.

[0041] Emotions are evaluated based on the pulse rate of the user reciting the sutra. From the perspective of improving mental health, it is considered preferable for the pulse rate to fluctuate less, i.e., for emotions to be low. Therefore, the lower the emotion, the better the emotion evaluation. The flame FL is displayed in a color that corresponds to the magnitude of the emotion. For example, the flame FL is displayed closer to white as the emotion becomes weaker, and closer to red as the emotion becomes stronger.

[0042] The pitch and emotion evaluations are performed for each recitation period. During the recitation period, the user recites a passage of sutra displayed in the sutra display area G21. That is, the pitch and emotion evaluations are performed for each passage of sutra. Furthermore, each time a passage of sutra is recited, the evaluation display area G23 outputs the pitch and emotion evaluations by updating the shape and color of the flame FL based on the pitch and emotion evaluations. A sutra passage is an example of a character segment of a sutra.

[0043] In this way, the evaluation display area G23 displays the pitch evaluation and the emotion evaluation using a single flame color and shape, allowing the user to easily check both the pitch evaluation and the emotion evaluation.

[0044] 6 is a diagram showing an example of the second sutra recitation screen G3 displayed on the display unit 15. The second sutra recitation screen G3 is a screen for performing sutra recitation in the Mokugyo style. The second sutra recitation screen G3 is displayed when the chanting start object G13 is selected with the Mokugyo style sect selected on the setting screen G1. The second sutra recitation screen G3 includes a sutra text display area G31, an animation display area G32, and an evaluation display area G33.

[0045] The sutra display area G31 displays the sutra to be recited by the user and indicates the speed at which the user recites the sutra. In the example shown in Fig. 5, the sutra display area G31 displays the sutra in white characters, like karaoke lyrics, and switches the sutra characters from white characters to black characters from front to back at a predetermined speed. The user recites the sutra in time with the speed at which the sutra display switches from white characters to black characters, so that the sutra is recited at the predetermined speed.

[0046] The animation display area G32 repeatedly displays an animation of a stick called a mokugyo bai striking a mokugyo. In mokugyo-style recitation, the sound of a mokugyo is output from the audio output unit 14 at predetermined intervals. The animation display area G32 displays the animation so that the timing at which the mokugyo sound is output and the timing at which the stick strikes the mokugyo are synchronized. The tempo at which the mokugyo sound is output varies depending on the chanting speed selected on the setting screen G1. For example, the tempo of the mokugyo sound is 100 bpm when "Excitement" is selected, 80 bpm when "Everyday" is selected, and 65 bpm when "Calmness" is selected.

[0047] In this way, the sutra recitation assistance device 1 presents the sutra recitation speed with images and sounds.

[0048] The evaluation display area G33 displays the pitch evaluation and the emotion evaluation, similar to the evaluation display area G23. The pitch and emotion evaluations are performed for each evaluation period, which is separated by two consecutive mokugyo sounds. The user recites the portion of the sutra display area G31 where the characters change from outlined to black while the two consecutive mokugyo sounds are being output. In other words, the pitch and emotion evaluations are performed for each evaluation period. Furthermore, the evaluation display area G33 outputs the pitch evaluation and the emotion evaluation by updating the shape and color of the flame based on the pitch evaluation and the emotion evaluation each time a portion of the sutra corresponding to the evaluation period is recited. The portion of the sutra corresponding to the evaluation period is an example of a character segment of the sutra.

[0049] FIG. 7 is a diagram showing the proficiency evaluation screen G4 displayed on the display unit 15. The proficiency evaluation screen G4 is a screen that displays an evaluation of the user's proficiency in sutra chanting. The proficiency evaluation screen G4 is displayed when sutra chanting using the first sutra chanting screen G2 or the second sutra chanting screen G3 is completed. The proficiency evaluation screen G4 includes an item-specific evaluation display area G41 and a proficiency display area G42.

[0050] The item-specific evaluation display area G41 displays the pitch evaluation, emotion evaluation, and emotion judgment for the entire period from the start to the end of the chanting. The pitch evaluation for the entire period is the ratio of the period during which the pitch fluctuation was determined to be small to the period from the start to the end of the chanting. The emotion evaluation for the entire period is the ratio of the period during which the emotion was determined to be small to the period from the start to the end of the chanting. The emotion judgment for the entire period is the ratio of the period during which the user was determined to have positive emotions to the period from the start to the end of the chanting.

[0051] The proficiency display area G42 displays an evaluation of the user's chanting proficiency. In the example shown in Figure 7, the text indicates that the user is at a proficiency level appropriate for the current chanting. The user's chanting proficiency is evaluated based on a combination of pitch evaluation, emotional evaluation, and emotion judgment.

[0052] 8 is a flow diagram showing the flow of the sutra recitation assistance process executed by the sutra recitation assistance device 1. The sutra recitation assistance process is executed, for example, when the operation unit 16 receives an operation to start a sutra recitation assistance application program. The sutra recitation assistance process is realized by the processing unit 17 working in cooperation with each component of the sutra recitation assistance device 1 based on the program stored in the storage unit 11.

[0053] First, the reception unit 171 receives a selection of a sect and a selection of a recitation speed by the user (step S101). The reception unit 171 displays the setting screen G1 on the display unit 15 by supplying display data for the setting screen G1, which has been stored in advance in the storage unit 11, to the display unit 15. The reception unit 171 receives an operation on the operation unit 16 to select a sect and an operation to select a chanting speed. Based on the received operation, the reception unit 171 changes the display of the sect selection area G11 and the speed selection area G12 on the setting screen G1. The reception unit 171 also receives an operation on the operation unit 16 to select a chanting start object G13. The reception unit 171 stores the sect and chanting speed selected at the time of the operation in the storage unit 11 as the sect and chanting speed selected by the user.

[0054] Next, the reception unit 171 refers to the sect table T1 and determines whether the sutra chanting of the user's sect is a breathing method (step S102).

[0055] If the sutra chanting of the user's sect is the breath-taking method (step S102-Yes), a first sutra chanting process for supporting the breath-taking method of sutra chanting is executed (step S103). The first sutra chanting process will be described in detail later.

[0056] If the chanting of the user's sect is not the breath-taking method (step S102-No), that is, if it is the mokugyo method, a second chanting process is executed to support the mokugyo method chanting (step S104). The second chanting process will be described in detail later.

[0057] After the first sutra recitation process or the second sutra recitation process, a skill evaluation process is executed to evaluate the skill level of the user's sutra recitation (step S105). The skill evaluation process will be described in detail later. This completes the sutra recitation assistance process.

[0058] 9 is a flow diagram showing the flow of the first sutra recitation process. The first sutra recitation process is executed in step S103 of the sutra recitation assistance process.

[0059] First, the presenting unit 172 displays the first sutra recitation screen G2 on the display unit 15 by supplying the display data of the first sutra recitation screen G2 stored in advance in the storage unit 11 to the display unit 15 (step S201). After that, the presenting unit 172 updates the display of the sutra text displayed in the sutra text display area G21 of the first sutra recitation screen G2 as time passes.

[0060] Next, the presentation unit 172 controls the audio input unit 13 to start acquiring the recitation voice of the user reciting the sutra (step S202). After that, the presentation unit 172 generates recitation voice data in which the recitation voice of the user is recorded in chronological order based on the electrical signal converted from the recitation voice of the user supplied from the audio input unit 13, and stores the generated data in the storage unit 11.

[0061] Furthermore, the presenting unit 172 controls the imaging unit 12 to start capturing an image of the user reciting the sutra (step S203). Thereafter, the presenting unit 172 acquires data of the moving images of the user supplied from the imaging unit 12 and stores the data in the storage unit 11. Note that the moving images are an example of continuously obtained images.

[0062] Next, the presentation unit 172 presents, by an image, a breathing period for the user to take a breath (step S204). The presentation unit 172 starts the breathing period. The presentation unit 172 displays "inhale" as the character string ST in the period presentation area G22. The presentation unit 172 also extends the progress bar PB in the period presentation area G22 so that it reaches the 12 o'clock position at the end time of the breathing period. In this way, the presentation unit 172 presents the breathing period by an image displayed in the period presentation area G22.

[0063] When the breathing period ends, the presentation unit 172 presents, by an image, a recitation period during which the user should recite the sutra (step S205). The presentation unit 172 starts the recitation period. The presentation unit 172 displays "Read" as the character string ST in the period presentation area G22. The presentation unit 172 also extends the progress bar PB in the period presentation area G22 so that it reaches the 12 o'clock position at the end of the recitation period. In this way, the presentation unit 172 presents the recitation period by an image displayed in the period presentation area G22.

[0064] When the recitation period ends, the pitch evaluation unit 173 evaluates the pitch of the user's recitation voice (step S206). The pitch evaluation unit 173 acquires the recitation voice data of the most recently ended breathing period and recitation period from the storage unit 11. The pitch evaluation unit 173 calculates the amount of fluctuation in the pitch of the recitation voice based on the recitation voice data, and evaluates the pitch of the recitation voice based on the amount of fluctuation in pitch.

[0065] FIG. 10 is a schematic diagram illustrating a pitch evaluation method. The vertical axis of the graph in FIG. 10 represents frequency f, and the horizontal axis represents time t. The pitch evaluation unit 173 applies FFT (Fast Fourier Transform) to the reading voice data and performs spectral analysis of the reading voice data to calculate the time change PT of the fundamental frequency F0 corresponding to the pitch of the reading voice. In FIG. 9, the time change PT of the fundamental frequency F0 during the reading period P1 and the breathing period P2 is shown by a curve. The pitch evaluation unit 173 extracts the maximum and minimum values ​​of the fundamental frequency F0 during the reading period P1, and calculates the difference Δf between the maximum and minimum values ​​as the amount of pitch fluctuation.

[0066] The pitch evaluation unit 173 evaluates the pitch of the reading voice by comparing the amount of pitch fluctuation with a predetermined threshold. For example, if the difference Δf is less than 20 Hz, the pitch evaluation unit 173 evaluates the amount of pitch fluctuation as small. If the difference Δf is 20 Hz or more and less than 50 Hz, the pitch evaluation unit 173 evaluates the amount of pitch fluctuation as medium. If the difference Δf is 50 Hz or more, the pitch evaluation unit 173 evaluates the amount of pitch fluctuation as large.

[0067] In this way, the pitch evaluation unit 173 evaluates the pitch of the recitation voice for each recitation period, that is, for each passage of the sutra.

[0068] Returning to FIG. 9 , the emotion evaluation unit 174 evaluates the emotion of the user (step S207). The emotion evaluation unit 174 acquires from the storage unit 11 video data of the user for the breathing period and reading period that ended immediately before. The emotion evaluation unit 174 estimates the user's pulse rate based on the video data of the user, and evaluates the user's emotion based on the number of fluctuations in the pulse rate. For example, the emotion evaluation unit 174 estimates the user's pulse rate using the method described in WO 2022 / 065446 or WO 2018 / 74371.

[0069] FIG. 11 is a schematic diagram illustrating a method for evaluating emotions. The vertical axis of the graph in FIG. 11(A) represents pulse wave amplitude A, and the horizontal axis represents time t. The vertical axis of the graph in FIG. 11(B) represents pulse rate B, and the horizontal axis represents time t. The emotion evaluation unit 174 extracts multiple frame images from the user's video data. The emotion evaluation unit 174 applies a contour detection algorithm or a feature point extraction algorithm to the multiple frame images to identify an area in the frame images that corresponds to the user's forehead. The emotion evaluation unit 174 extracts pixels included in the area that corresponds to the forehead in each frame image, and calculates the change in the G (green) pixel value over time by obtaining the G pixel value from the RGB pixel values ​​of the extracted pixels. Capillaries are concentrated on the forehead, and the G pixel value reflects the user's blood flow, so the pulse wave is detected based on the change in the G pixel value over time. The emotion evaluation unit 174 extracts the pulse wave signal PW shown in Fig. 11(A) by applying a bandpass filter having a transmission band of 0.5 Hz to 3 Hz, which corresponds to a human pulse wave, to the signal indicating the temporal change in pixel values ​​of G. Note that the emotion evaluation unit 174 may also extract an area in the frame image that corresponds to an exposed part of the human skin other than the forehead.

[0070] The emotion evaluation unit 174 calculates the change in the user's pulse rate over time based on the extracted pulse wave signal PW. For example, the emotion evaluation unit 174 identifies the peak point P(n) of the pulse wave signal. The emotion evaluation unit 174 calculates the interval between two adjacent peak points (P(n), P(n+1)) as the pulse wave interval d(n). The emotion evaluation unit 174 calculates the change in the pulse rate over time PR shown in FIG. 10(B) by multiplying the reciprocal of the pulse wave interval by 60 as the pulse rate per minute.

[0071] The emotion evaluation unit 174 calculates the number of times the pulse rate fluctuates based on the time change PR of the pulse rate. The emotion evaluation unit 174 extracts the pulse rate b2 at the point where the time change PR of the pulse rate during the reading period P1 is at its maximum value, and the pulse rate b1 a predetermined time before the point at which the pulse rate is at its maximum value. The emotion evaluation unit 174 determines that the pulse rate has fluctuated if the pulse rate b2 is greater than the average pulse rate bav of an average person and the difference Δb between the pulse rate b2 and the pulse rate b1 is equal to or greater than a predetermined value. The emotion evaluation unit 174 calculates the number of times the pulse rate fluctuates during the reading period P1 by making the above-mentioned determination for each point at which the pulse rate is at its maximum value during the reading period P1.

[0072] The emotion evaluation unit 174 evaluates the emotion by comparing the number of times the pulse rate fluctuates during the reading period with a predetermined threshold. For example, the emotion evaluation unit 174 evaluates the emotion as low when the number of times the pulse rate fluctuates during the reading period is one or less. The emotion evaluation unit 174 evaluates the emotion as medium when the number of times the pulse rate fluctuates during the reading period is two or more and four or less. The emotion evaluation unit 174 evaluates the emotion as high when the number of times the pulse rate fluctuates during the reading period is five or more.

[0073] In this way, the emotion evaluation unit 174 evaluates the emotion of the user for each recitation period, that is, for each passage of the sutra.

[0074] Returning to FIG. 9 , the output unit 175 outputs the pitch evaluation and the emotion evaluation using the color and shape of a single flame (step S208). The output unit 175 generates display data for a flame having a shape corresponding to the pitch evaluation and a color corresponding to the emotion evaluation. The output unit 175 displays the flame in the evaluation display area G23 based on the generated display data. Thereafter, the output unit 175 enlarges or reduces the shape of the flame displayed in the evaluation display area G23 as time passes. In this way, the output unit 175 outputs the pitch evaluation and the emotion evaluation every time the recitation period ends, that is, every time a passage of sutra is recited.

[0075] Next, the presenting unit 172 determines whether the sutra recitation has ended (step S209). For example, the presenting unit 172 determines that the sutra recitation has ended when a predetermined time has elapsed since the first sutra recitation screen G2 was displayed and all sutras have been displayed in the sutra display area G21.

[0076] If the sutra recitation has not ended (step S209-No), the first sutra recitation process returns to step S204, and the presenter 172 presents the breath period again. If the sutra recitation has ended (step S209-Yes), the first sutra recitation process ends.

[0077] 12 is a flow diagram showing the flow of the second sutra recitation process. The second sutra recitation process is executed in step S104 of the sutra recitation assistance process.

[0078] First, the presenting unit 172 displays the second sutra recitation screen G3 on the display unit 15 based on the display data previously stored in the storage unit 11 (step S301). After that, the presenting unit 172 updates the display of the sutra text displayed in the sutra text display area G31 of the second sutra recitation screen G3 as time passes. In addition, the presenting unit 172 updates the animation displayed in the animation display area G32 as time passes.

[0079] Next, the presentation unit 172 controls the voice input unit 13 to start acquiring the recitation voice when the user recites the sutra (step S302).

[0080] Furthermore, the presenting unit 172 controls the imaging unit 12 to start capturing an image of the user reciting the sutra (step S303).

[0081] Next, the presentation unit 172 determines whether the output time for outputting the sound of the mokugyo has arrived (step S304). The output time is the time when a time determined based on the speed of the sutra chanting has elapsed since the time when the second sutra chanting screen G3 was displayed or the time when the sound of the mokugyo was output immediately before.

[0082] If the output time has not arrived (step S304-No), the second sutra recitation process returns to step S304. That is, the presenter 172 waits until the output time arrives.

[0083] If the output time has arrived (step S304-Yes), the presentation unit 172 controls the audio output unit 14 to output the sound of a mokugyo (step S305). The presentation unit 172 outputs the sound of a mokugyo by supplying audio data of the sound of a mokugyo stored in advance in the storage unit 11 to the audio output unit 14. In this way, the presentation unit 172 presents the chanting speed by an image displayed in the animation display area G32 and audio output from the audio output unit 14.

[0084] Next, the pitch evaluation unit 173 evaluates the pitch of the user's recitation voice (step S306). The pitch evaluation unit 173 acquires recitation voice data for the period between the immediately preceding output time and the output time before that from the storage unit 11. As in step S206 of the first sutra recitation process, the pitch evaluation unit 173 performs spectral analysis of the recitation voice data to calculate the change in the fundamental frequency F0 over time. The pitch evaluation unit 173 also detects a period in which the recitation voice data contains the sound of a mokugyo, and extracts a period in which the recitation voice data does not contain the sound of a mokugyo as the evaluation period. The period in which the sound of a mokugyo is contained may be detected by comparing the audio waveform indicated by the recitation voice data with a pre-stored audio waveform of the sound of a mokugyo, or may be detected based on time.

[0085] The pitch evaluation unit 173 extracts the maximum and minimum values ​​of the fundamental frequency during the evaluation period and calculates the difference between the maximum and minimum values ​​as the amount of pitch fluctuation. By excluding the period that includes the sound of the mokugyo from the evaluation period, the time change of the fundamental frequency can be calculated appropriately without being affected by the sound of the mokugyo. The pitch evaluation unit 173 evaluates the pitch of the reading voice by comparing the amount of pitch fluctuation with a predetermined threshold.

[0086] In this way, the pitch evaluation unit 173 evaluates the pitch of the recited voice for each evaluation period, that is, for each part of the sutra corresponding to the evaluation period.

[0087] Furthermore, the emotion evaluation unit 174 evaluates the emotion of the user (step S307). The emotion evaluation unit 174 acquires from the storage unit 11 moving image data of the user for the period between the immediately preceding output time and the output time before that. The emotion evaluation unit 174 calculates the change in the user's pulse rate over time based on the moving image data of the user, in the same manner as in step S207 of the first sutra recitation evaluation process. The emotion evaluation unit 174 calculates the number of times the pulse rate fluctuates during the evaluation period. The emotion evaluation unit 174 evaluates the emotion of the user by comparing the number of times the pulse rate fluctuates with a predetermined threshold.

[0088] In this way, the emotion evaluation unit 174 evaluates the pitch of the recited voice for each evaluation period, that is, for each part of the sutra corresponding to the evaluation period.

[0089] Next, the output unit 175 outputs the pitch evaluation and the emotion evaluation in the same manner as in step S208 of the first sutra recitation process (step S308). The output unit 175 outputs the pitch evaluation and the emotion evaluation every time the evaluation period ends, that is, every time a part of the sutra corresponding to the evaluation period is recited.

[0090] Next, the presenting unit 172 determines whether the sutra recitation has ended (step S309). If the sutra recitation has not ended (step S309-No), the second sutra recitation process returns to step S304, and the presenting unit 172 waits again until the output time arrives. If the sutra recitation has ended (step S309-Yes), the second sutra recitation process ends.

[0091] 13 is a flow diagram showing the flow of the skill evaluation process. The skill evaluation process is executed in step S105 of the sutra recitation assistance process.

[0092] First, the emotion determination unit 176 determines the emotion of the user (step S401). The emotion determination unit 176 determines the emotion of the user based on the pulse wave interval d(n) of the user calculated in step S207 of the first sutra recitation process or step S307 of the second sutra recitation process.

[0093] The emotion determination section 176 acquires the pulse wave interval d(n) included in each emotion determination period from the storage section 11. For each emotion determination period, the emotion determination section 176 calculates the maximum Lyapunov exponent λ, which is expressed by the following equation.

number

[0094] The maximum Lyapunov exponent is an index that indicates the degree to which the time change in pulse wave interval contains complex system fluctuations. The larger the value, the greater the complex system fluctuations, and the smaller the value, the smaller the complex system fluctuations. A human's pulse wave interval contains a certain degree of complex system fluctuation due to the function of the autonomic nervous system, but it is known that complex system fluctuations are lost when the autonomic nervous system function declines. Therefore, the smaller the value of the maximum Lyapunov exponent, the more the function of the autonomic nervous system declines, and it is estimated that the person is experiencing stress factors, i.e., negative emotions.

[0095] The emotion determination unit 176 determines the emotion of the user during the emotion determination period by comparing the maximum Lyapunov exponent λ with a predetermined threshold. For example, if the maximum Lyapunov exponent λ is greater than +0.0150, the emotion determination unit 176 determines that the user has a high positive emotion that is suitable for chanting, and if the maximum Lyapunov exponent λ is equal to or less than +0.0150, the emotion determination unit 176 determines that the user has a low positive emotion or a negative emotion that is not suitable for chanting.

[0096] Next, the skill evaluation unit 177 evaluates the skill of the user's sutra chanting based on a combination of the evaluation of the pitch, the evaluation of the emotion, and the judgment of the feeling (step S402). The skill evaluation unit 177 evaluates the pitch, the emotion, and the judgment of the feeling for the entire period from the start to the end of the sutra chanting.

[0097] The proficiency evaluation unit 177 calculates the ratio of the sum of the lengths of the recitation periods or evaluation periods in which the pitch fluctuation is evaluated to be small to the sum of the lengths of all the recitation periods or evaluation periods as an evaluation of the pitch for the entire period. The proficiency evaluation unit 177 also calculates the ratio of the sum of the lengths of the recitation periods or evaluation periods in which the emotion is evaluated to be small to the sum of the lengths of all the recitation periods or evaluation periods as an evaluation of the emotion for the entire period. The proficiency evaluation unit 177 also calculates the ratio of the sum of the lengths of the emotion determination periods in which the user is determined to have a high positive emotion suitable for chanting to the sum of the lengths of all the emotion determination periods as an evaluation of the emotion for the entire period.

[0098] The skill evaluation unit 177 evaluates the user's proficiency in chanting based on a combination of results obtained by comparing the pitch evaluation, emotion evaluation, and emotion judgment for the entire period with predetermined thresholds. For example, if the pitch evaluation, emotion evaluation, and emotion judgment are all 50% or higher, the skill evaluation unit 177 evaluates that the user's proficiency is appropriate for the current settings. If the pitch evaluation, emotion evaluation, and emotion judgment are all less than 50%, the skill evaluation unit 177 evaluates that the user's proficiency is low for the current settings. If the emotion evaluation is 50% or higher and the pitch evaluation and emotion judgment are less than 50%, the skill evaluation unit 177 evaluates that the user's proficiency is very low for the current settings. This is because it is thought that the user has given up on matching the pitch because they are unable to recite sutras appropriately, making it difficult for them to feel emotions. If the evaluation of the pitch is 50% or more and the judgment of the emotion is less than 50%, the skill evaluation unit 177 evaluates that the user's skill level is too high for the current setting. This is because it is considered that the user is able to recite at an appropriate pitch without concentrating on chanting.

[0099] Next, the proficiency output unit 178 outputs the proficiency evaluation (step S403). The proficiency output unit 178 generates display data for a proficiency evaluation screen G4 that includes the pitch evaluation, emotion evaluation, and feeling judgment for the entire period calculated in step S401 in an item-by-item evaluation display area G41, and the proficiency evaluated in step S401 in a proficiency display area G42. The proficiency output unit 178 supplies the generated display data to the display unit 15. This completes the proficiency evaluation process.

[0100] As explained above, the sutra recitation assistance device 1 evaluates the pitch of the recitation voice and the user's emotion for each character segment of the sutra, and outputs the pitch evaluation and the emotion evaluation each time the recitation of a character segment is completed. This enables the sutra recitation assistance device 1 to present the evaluation of the recitation voice and emotion of the user reciting in real time.

[0101] Furthermore, the sutra recitation assistance device 1 accepts the user's selection of a sect and, depending on the sect selection, indicates the speed of the sutra recitation or the duration of the sutra recitation by indicating the timing of the user's breathing or by outputting the sound of a mokugyo at predetermined intervals. This makes it possible for the sutra recitation assistance device 1 to assist the user in reciting sutras regardless of the user's sect.

[0102] Furthermore, the sutra recitation assistance device 1 displays the pitch evaluation and the emotional evaluation using a single flame color and shape, thereby enabling the user reciting sutras to easily grasp both the evaluation of the recitation voice and the evaluation of the emotional evaluation.

[0103] Furthermore, the sutra recitation assistance device 1 judges the user's emotions and evaluates the user's level of chanting skill based on a combination of pitch evaluation, emotional evaluation, and emotion judgment. This enables the user to select a sutra passage and recitation speed that is appropriate for the user's level of skill.

[0104] In the above description, the receiving unit 171 receives the selection of a sect in step S101 of the sutra chanting support processing, but this is not a limited example. For example, the receiving unit 171 does not have to receive the selection of a sect. Furthermore, the receiving unit 171 may receive the selection of any type of sutra instead of the selection of a sect. In sutra chanting as training to improve mental health, it is also conceivable that a sutra different from one's own sect will be recited. By having the receiving unit 171 receive the selection of the type of sutra, it is possible to meet the needs of users who wish to recite a variety of sutras.

[0105] In the above description, in step S203 of the first sutra recitation process and step S303 of the second sutra recitation process, the presenting unit 172 starts acquiring data of moving images of the user, but this is not limited to such an example. The presenting unit 172 may acquire data of multiple still images of the user taken consecutively instead of moving images. Multiple still images of the user taken consecutively are another example of continuously obtained images.

[0106] In the above description, in step S206 of the first sutra recitation process and step S306 of the second sutra recitation process, the pitch evaluation unit 173 determines the amount of pitch fluctuation based on the difference between the maximum and minimum values ​​of the fundamental frequency during the recitation period, but this is not limited to this example. For example, instead of the fundamental frequency, the peak frequency of the spectral envelope of the audio signal may be used. Furthermore, instead of the difference between the maximum and minimum values, the standard deviation may be used.

[0107] In the above description, the output unit 175 displays the pitch and emotional evaluations using the color and shape of a flame in step S208 of the first sutra recitation process and step S308 of the second sutra recitation process. However, this is not a limited example. For example, the output unit 175 may display an arbitrary object other than a flame in the evaluation display area G23 or G33 and display the pitch and emotional evaluations using the color and shape of the object. The output unit 175 may also display the pitch and emotional evaluations using a combination of two different display modes other than color and shape. For example, the output unit 175 may display the pitch and emotional evaluations using a combination of the basic shape of a flame and changes in the shape over time. The output unit 175 may display the pitch and emotional evaluations using the hue and brightness of the color of the flame. Alternatively, the output unit 175 may display the pitch and emotional evaluations using two different display modes in a single display area. The output unit 175 may also output the pitch and emotional evaluations using audio.

[0108] In the above description, the emotion determination unit 176 determines the emotion of the user in step S401 of the proficiency evaluation process after the sutra chanting is completed, but this is not limited to this example. The emotion determination unit 176 may determine the emotion of the user during an emotion determination period each time the emotion determination period has elapsed since the start of sutra chanting. In this case, the output unit 175 may output the emotion determination by displaying it on the first sutra chanting support screen each time the user's emotion is determined. This allows the user to grasp the emotion determination in real time.

[0109] Each function of the sutra recitation assistance device 1 may be realized by multiple devices. For example, the processing performed by the processing unit 17 may be executed by a server that communicates with the sutra recitation assistance device 1, and the sutra recitation assistance device 1 may receive the processing results from the server. Also, the user may be photographed by an external imaging device such as a surveillance camera or an IP (Internet Protocol) camera, and the sutra recitation assistance device 1 may acquire moving or still images of the user by communicating with the external imaging device. In this case, the sutra recitation assistance device 1 may not have the imaging unit 12.

[0110] It should be understood by those skilled in the art that various changes, substitutions, and alterations can be made to the present invention without departing from the scope of the present invention. For example, the above-described embodiments and modifications may be implemented in appropriate combination within the scope of the present invention. [Explanation of symbols]

[0111] 1. Sutra recitation support device 171 Reception Department 172 Presentation section 173 Pitch evaluation section 174 Emotional Assessment Unit 175 Output section 176 Emotion Judgment Department 177 Proficiency Evaluation Department 178 Proficiency Output Unit

Claims

1. An imaging unit that captures an image of a user; a presentation unit that presents the speed at which the user should recite the sutra or the period during which the user should recite the sutra by means of an image or sound; a pitch evaluation unit that evaluates the pitch of the recitation voice when the user recites the sutra for each character segment of the sutra during the presentation of the speed or the period; an emotion evaluation unit that evaluates the emotion of the user who is reciting the sutra for each character segment while the speed or the period is being presented, based on images including areas corresponding to exposed skin of the human body of the user who is reciting the sutra, which are continuously obtained by the imaging unit; an output unit that outputs the pitch evaluation and the emotion evaluation each time the character segment is read aloud; A sutra recitation assistance device comprising:

2. The system further includes a reception unit that receives a user's selection of a denomination, the presenting unit presents the speed at which the user should recite the sutra or the period during which the sutra should be recited by the user, by presenting the timing for taking a breath or by outputting the sound of a mokugyo at predetermined intervals, in accordance with the selection of the sect; The sutra recitation assistance device according to claim 1.

3. the output unit displays the pitch evaluation and the emotion evaluation in a single display area using different display modes. The sutra recitation assistance device according to claim 1.

4. the output unit displays the pitch evaluation and the emotion evaluation using a single color and shape of a flame. The sutra recitation assistance device according to claim 3.

5. an emotion determination unit that determines an emotion based on a maximum Lyapunov exponent indicating a degree of fluctuation in pulse wave intervals of the user who is reciting the sutra; a skill evaluation unit that evaluates the skill of the user's sutra chanting based on a combination of the evaluation of the pitch, the evaluation of the emotion, and the determination of the feeling; A skill output unit that outputs the skill evaluation, The sutra recitation assistance device according to claim 1.

6. A sutra recitation assistance method executed by a sutra recitation assistance device, The speed at which the user should recite the sutra or the period during which the user should recite the sutra is displayed by image or sound; During the presentation of the speed or the period, an evaluation is made of the pitch of the recitation voice when the user recites the sutra for each character segment of the sutra; During the presentation of the speed or the period, the emotion of the user is evaluated for each character segment based on images including areas corresponding to exposed skin of the user reciting the sutra, the images being continuously obtained by an imaging unit; outputting the pitch evaluation and the emotion evaluation each time the character segment is read aloud; A sutra recitation support method comprising:

7. The speed at which the user should recite the sutra or the period during which the user should recite the sutra is displayed by image or sound; During the presentation of the speed or the period, an evaluation is made of the pitch of the recitation voice when the user recites the sutra for each character segment of the sutra; During the presentation of the speed or the period, the emotion of the user is evaluated for each character segment based on images including areas corresponding to exposed skin of the user reciting the sutra, the images being continuously obtained by an imaging unit; outputting the pitch evaluation and the emotion evaluation each time the character segment is read aloud; A program that causes a computer to execute the following:

Images