Acoustic device
A microphone with a motion sensor enables new singing experiences, addressing social distancing challenges by enhancing social interactions and economic revitalization through enriched social lives.
Patent Information
- Application Number
- JP2023072882
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-27
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2043-04-27
AI Technical Summary
Conventional performance methods and devices face challenges in providing a new way to perform, especially due to social distancing measures that disrupt healthy social interactions and relationships, necessitating a solution to enrich social lives and revitalize the economy.
A microphone equipped with a motion sensor that detects user movements, transmitting this information to a control device which selects and outputs corresponding audio data based on the detected movements, allowing for a new singing experience.
Enhances social interactions and health through new singing experiences, contributing to a positive outlook and economic revitalization by increasing opportunities to engage in social activities.
Smart Images

Figure 2024158045000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an acoustic device, a trained model, a method, and a program. [Background technology]
[0002] Conventionally, a performance device is known that is composed of a microphone that picks up a person's voice and converts the audio signal into an electrical signal, an acoustic converter that converts the electrical signal converted by the microphone into vibrations and outputs a sound audible to the human ear, a playback device that plays back music data, and a video output device that outputs images that accompany the music on a television, projector, monitor, etc. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Japanese Patent Application Publication No. 04-191882 Summary of the Invention [Problem to be solved by the invention]
[0004] Meanwhile, in conventional performance methods and performance devices such as that disclosed in Patent Document 1, audio output means is mainly used for converting the voice sung by a singer into an electrical signal and outputting it.
[0005] The present disclosure provides a new performance method in which a singer's voice data is pre-programmed through machine learning, a motion sensor reacts, for example, to shaking a microphone, a control device receives information from the sensor, and the corresponding voice data is played back to sing.
[0006] Recent temporary measures such as maintaining social distance as a preventative measure against viral infections are unwittingly causing distortions in people's healthy minds, and the fundamental issue that this disclosure aims to solve is to prevent the perpetuation of a state in which adverse trends that go beyond the above-mentioned measures make it difficult to build relationships or enrich social lives, to enable people to live healthy, happy, and positive lives, and to revitalize the economy and industry. [Means for solving the problem]
[0007] In order to solve the above problems, the device is characterized by comprising a microphone including a motion sensor that detects the movement of the microphone held by the user and transmits information about the detected content to a control device, a control unit that analyzes the information received from the motion sensor, selects corresponding audio data from a database or ROM, and transmits it to an output unit, and an output unit that outputs the audio data in a form that corresponds to the lyrics of the song being played by the control unit. Effect of the Invention
[0008] According to the present disclosure, a new singing experience can provide an opportunity for exercise to people who are concerned about their health due to the penetration of remote culture. Furthermore, the increase in opportunities to go out based on the present disclosure can also contribute to preventing the weakening of human relationships. [Brief description of the drawings]
[0009] [Figure 1] 1 is a diagram showing the concept of an audio device according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram showing an example in which the audio device according to the present embodiment is used. [Diagram 3] FIG. 1 is a diagram showing an example in which the audio device according to the present embodiment is used. [Figure 4] FIG. 2 is a diagram illustrating an example of a configuration of a microphone according to the present embodiment. [Diagram 5] 1A to 1C are diagrams illustrating an example of singing expression in this embodiment. [Figure 6]FIG. 2 is a diagram illustrating an example of a configuration of a control device according to the present embodiment. [Figure 7] FIG. 1 is a diagram illustrating an example of a method for machine learning of speech in this embodiment. [Figure 8] FIG. 2 is a diagram showing an example of a flowchart relating to the processing of a program for realizing singing expression in this embodiment. [Figure 9] FIG. 2 is a diagram showing an example of the configuration of a loudspeaker in the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The following embodiments do not limit the scope of the disclosed invention, and all of the combinations of features described in the embodiments are not necessarily essential to the solution of the present invention.
[0011] FIG. 1 is a diagram showing the concept of an audio device 14 in this embodiment, and includes a microphone 100 that detects or acquires the movements of a user 10 using a motion sensor, converts them into numerical values, and transmits the movement information of the user 10 to a control device 200 as a digital signal, the control device 200 that receives the movement information of the user 10 detected or acquired by the motion sensor and transmits it to an output unit 112, a loudspeaker 300 that plays audio data of the user 10 corresponding to the movements of the user 10, an audio converter 400 that plays music, and a display unit 500 that outputs images and lyrics corresponding to the music being played.
[0012] FIG. 2 is a diagram showing an example in which the audio device 14 according to this embodiment is used.
[0013] 2, user 10 holds microphone 100 and expresses the pitch of the sound by shaking it vertically. In this example, a program preliminarily learns the voice information of user 100 through machine learning to acquire feature information of user 100, which is then stored as data in database 250. Based on the feature information, control device 200 selects voice information of user 10 corresponding to the lyrics of the music played by control device 200, transmits it to loudspeaker 300, converts the electrical signal into vibration, and outputs it as the voice of user 10. Deep learning is a well-known technology that can be used as an example of machine learning.
[0014] The user 10 performs the action of singing by watching the display unit 500 that displays images and lyrics corresponding to the music played by the control device 200 and shaking the microphone 100 in time with the lyrics.
[0015] The acoustic transducer 400 outputs sound by converting the electrical signals of the music played and transmitted by the control device 200 into vibrations.
[0016] 3 is a diagram showing an example of a case where the user 10 shakes the microphone 100 sideways. In this embodiment, shaking the microphone 100 sideways is an action used when singing continuously. Note that continuous singing can also be achieved by shaking the microphone back and forth.
[0017] FIG. 4 is a diagram showing an example of the configuration of the microphone 100 in this embodiment.
[0018] The motion sensor 111 detects the movement of the user 10 shaking the microphone 100. As an example of the sensor, an acceleration sensor or a gyro sensor is used to detect the speed and angular velocity of shaking the microphone 100, converts the acquired information into an electrical signal, and transmits it wirelessly to the control device 200.
[0019] The output unit 112 receives motion information of the user 10 detected or acquired by the motion sensor 111 as an electrical signal from the control device 200, converts the received electrical signal into vibration, and outputs it.
[0020] FIG. 5 is a diagram showing an example of singing expression in this embodiment.
[0021] 101a shows an example of a playing method in which the pitch of the sound corresponds to the vertical shaking of the microphone 100. When the user 10 holds the microphone 100 high, the gyro sensor detects the movement and recognizes it as a high-pitched sound by calculating the angular velocity. The lower the height of the microphone 100, the lower the sound is recognized.
[0022] 102b shows an example of a playing method in which the strength of the sound corresponds to the speed at which the microphone 100 is shook. The acceleration sensor detects the speed at which the user 10 shakes the microphone 100, and recognizes that the faster the shaking speed, the stronger the sound, and vice versa.
[0023] FIG. 6 is a diagram showing an example of the configuration of the control device 200 in this embodiment.
[0024] The control device 200 includes a processor 210, a ROM 220, a RAM 230, a sensor interface 240, a database 250, a display interface 260, and an output interface 270. These components are communicatively connected to each other via a bus 288.
[0025] The processor 210 is composed of at least one integrated circuit. For example, the processor 210 is composed of a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a combination of these, and performs calculations required to control the control device 200. The processor 210 reads data from the RAM 230, the ROM 230, or the database 250, performs calculations, writes the results to the RAM 230, selects a song from the database 250 to play music data, transmits song information to the acoustic transducer 400, synthesizes machine-learned voice data of the user 10 based on information transmitted from the motion sensor 111, and transmits the synthesized voice data to the output unit 112.
[0026] The ROM 220 includes a program for machine learning audio data, a video processing system, a sound system, an operating system for the control device, and a database management system. It also stores music information, a machine learning program, and audio data.
[0027] Database 250 stores music information, a program for machine learning audio data, and audio data. The music information includes the song title, singer, lyricist, composer, release year, genre, and audio source data and lyric data corresponding to each song.
[0028] RAM 230 temporarily stores information necessary for reading music data, executing programs for playing music, programs for machine learning the voice data of user 10, programs for transmitting synthesized voice data of user 10 to output unit 112, and programs for controlling control device 200.
[0029] The sensor interface 240 relays data transmission between the processor 210 and the motion sensor 111 .
[0030] The display interface 260 relays data transmission between the processor 210 and the display unit 500 .
[0031] The output interface 270 relays data transmission between the processor 210 and the output unit 112 .
[0032] FIG. 7 is a diagram showing an example of a method for performing machine learning on the voice of the user 10 in this embodiment.
[0033] In the input layer, the voice of the user 10 is recorded by a microphone 100 or other method, or pre-prepared voice data of the user 10 is read.
[0034] In the intermediate layer, the speech information of the user 10 read in the input layer is analyzed.
[0035] In the output layer, characteristic information such as the pitch, strength, and interval of the user's 10 voice analyzed in the intermediate layer is acquired and stored as data in the ROM 220 or database 250.
[0036] In addition, deep learning is used as an example of a machine learning method in this embodiment, and in order to make the voice information of the user 10 learned by the neural network 222 even closer to the actual voice of the user 10, an additional layer is prepared, and learning is performed again including a part of the trained model previously trained in the previous neural network 222 and the additional layer, so that more accurate feature information of the user 10 can be obtained. Fine-tuning is a well-known technique. In addition, a voice information dataset for machine learning is used to achieve higher accuracy.
[0037] FIG. 8 shows an example of a flowchart in which the program in this embodiment detects the movement of the microphone 100 by the user 10 using the trained model based on the feature information of the user 10, selects and synthesizes voice data of the user 10 corresponding to the detected movement information, and outputs the synthesized voice of the user 10 to the output unit 112.
[0038] In step S100, the motion sensor 111 detects the movement of the microphone 100 held by the user 10, and the detected movement information is transmitted as an electrical signal to the control device 200. The received information is analyzed.
[0039] In step S200, audio data to be reproduced is selected based on the information analyzed in step S100, and synthesis processing such as adjustment of the pitch, dynamics, and interval of the sound is performed.
[0040] In step S300, the synthetic voice data to be reproduced selected in step S200 is transmitted to output unit 112 via output interface 270.
[0041] In step S400, the synthetic voice data to be reproduced that was transmitted to the output unit 112 in step S300 is converted into vibrations, so that the output unit 112 outputs the synthetic voice of the user 10 in a form corresponding to the lyrics of the music being reproduced.
[0042] FIG. 9 is a diagram showing an example of the configuration of the loudspeaker 300 in this embodiment.
[0043] Signal converter 310 receives voice information of user 10 processed by control device 200, converts it into vibrations, and outputs it from output unit 112 as a sound that can be heard by humans. [Industrial Applicability]
[0044] By making it possible to play using a microphone equipped with a motion sensor, new singing expressions can be created, increasing the variety of music, it can also be used for health purposes, and by increasing opportunities to go outside, it can help reduce the thinning of human relationships. [Explanation of symbols]
[0045] 10 users 14 Sound equipment 100 Microphones 200 Control device 300 loudspeaker 400 Acoustic Transducer 500 Display 111 Motion Sensor 112 Output section 101a Singing expression method 1 102b Singing expression method 2 210 Processor 220 ROM 230 RAM 240 Sensor Interface 250 Databases 260 Display Interface 270 Output Interface 288 Bus 222 Neural Networks 233 Flowchart for audio output Analyze information from the S100 motion sensor S200 Select the audio data to be played S300: Transmitting selected audio data to the output unit S400 Playing audio data
Claims
1. A microphone including a motion sensor that detects the movement of the microphone held by a user and transmits information about the detected movement to a control device; a control unit that analyzes information received from the motion sensor by machine learning, selects corresponding voice data from a database or a ROM, and transmits the selected voice data to an output unit; an output unit that outputs the audio data in a form corresponding to the lyrics of the music played by the control unit; An acoustic device comprising:
2. 2. The audio device according to claim 1, wherein the motion sensor recognizes the pitch, strength and interval of a sound in correspondence with an angle and speed of the microphone.
3. A trained model for causing a computer to function to output a quantified value of a voice similar to the user's voice based on voice information, It consists of a neural network, The neural network is composed of an input layer for inputting the voice information, an intermediate layer for analyzing the input voice, and an output layer for acquiring feature information such as pitch, strength, and interval of the analyzed voice, The learning is performed through the output layer, the intermediate layer, and the output layer, If necessary, an additional layer is added, and weighting calculations and learning are performed again based on the trained model, including the additional layer and a part of the trained model. A trained model for causing a computer to output a quantified value of a voice that is close to the user's voice from an output layer or additional layer of the neural network.
4. Analyzing information received from the motion sensor using the trained model; a step of selecting voice data to be reproduced from the database or ROM based on the analyzed information, adjusting the pitch, dynamics, and interval of the sound as necessary, and performing a voice synthesis process; transmitting voice data or synthetic voice data to an output unit; outputting the voice data or the synthetic voice data; The method includes:
5. A program for causing a computer to execute the method according to claim 4.
Citation Information
Patent Citations
JP191882A
