Display device and display method
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- YAMAHA CORP
- Filing Date
- 2025-01-22
- Publication Date
- 2026-08-03
Smart Images
Figure 2026125323000001_ABST
Abstract
Description
Technical Field
[0006] , , ,
[0005] , , , ,
[0001] The present disclosure relates to a display device and a display method.
Background Art
[0002] When a performer who plays a musical instrument practices alone, it is useful to have a device that supports the practice. As such a practice support device, for example, a practice device that supports based on the muscle activity of the performer's face is known (see Japanese Patent Application Laid-Open No. 2017-146584).
[0003] The practice device can output practice support information corresponding to muscle activity data and performance sound data, and a composite image obtained by combining a practice support image and a face image is displayed on the display device of the practice support device. With this configuration, by also using the performance sound as a variable, it is possible to handle conditions that cannot be handled only by muscle activity, such as conditions related to performance techniques, and it is said that it is possible to provide more diverse practice support information.
Prior Art Documents
Patent Documents
[0007] (1) A display device according to one aspect of the present disclosure is a display device for a performer who plays a musical instrument, and includes a display unit that displays the linked state of two or more sensing values, which are selected from body sensing values representing the movements of the performer's body and instrument sensing values representing the characteristics of the sound emitted by the instrument.
[0008] (2) The display device described in (1) may include a body sensing unit that acquires one or more body sensing values and a musical instrument sensing unit that acquires one or more musical instrument sensing values.
[0009] (3) In the display device of (1) or (2) above, the sensing value displayed on the display unit may include both the body sensing value and the instrument sensing value.
[0010] (4) In any of the display devices described in (1) to (3) above, the body sensing value is preferably the time variation value of the performer's lung expansion and contraction.
[0011] (5) In any of the display devices described in (1) to (4) above, the body sensing value is preferably the time variation value of the oral pressure of the performer.
[0012] (6) In the display device of (4) above, the time variation value of the expansion and contraction of the performer's lungs is measured by a strain sensor element, and the strain sensor element is preferably a thread-like or strip-like shape that expands and contracts in the longitudinal direction, and is positioned so that the longitudinal direction is along the chest circumference or waist circumference of the performer.
[0013] (7) In any of the display devices described in (1) to (6) above, the instrument is a wind instrument having a mouthpiece, and the instrument sensing value is a time variation value of the pressure inside the mouthpiece, a time variation value of the reed distortion, or a time variation value of the loudness of the sound produced, or a combination thereof.
[0014] (8) In any of the display devices described in (1) to (7) above, the sensing value displayed on the display unit is a time-varying value, and the display on the display unit is a graph represented on a common time axis.
[0015] (9) In any of the display devices described in (1) to (7) above, the sensing values displayed on the display unit are two or three, and two of the sensing values are plotted as dots on a plane represented by orthogonal XY axes and displayed.
[0016] (10) In the display device of (9) above, there are three sensing values to be displayed on the display unit, and the magnitude of the third sensing value, which is not plotted by a dot on a plane represented by orthogonal XY axes, is represented by the size of the dot.
[0017] (11) In the display device of (9) or (10) above, the sensing value displayed on the display unit is a time-varying value, and the display of the dot is maintained for a certain period of time after it is displayed.
[0018] (12) The display unit of any of (9) to (11) above has training data of a musical piece to be played by the performer, the training data has sensing values corresponding to the sensing values of the performer to be displayed on the display unit, and the sensing values corresponding to the training data are plotted as dots on the plane and displayed.
[0019] (13) In the display device described in (12), the display unit may emit identification information corresponding to the distance between the performer's dot and the teacher data dot on the plane.
[0020] (14) A display method according to another aspect of the present disclosure is a display method for showing a performance state to a performer playing a musical instrument, and displays a linked state of two or more sensing values among a body sensing value representing the movement of the performer's body and an instrument sensing value representing the characteristics of the sound emitted by the musical instrument.
Effect of the Invention
[0021] The display device and the display method display the linked state of two or more sensing values among the body sensing value and the instrument sensing value. Therefore, it is easy for the performer to understand how the performer's way of using the body affects the sound production of the musical instrument. Thus, it more surely leads to an improvement in performance technique.
Brief Description of the Drawings
[0022] [Figure 1] FIG. 1 is a configuration diagram showing a display device according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a schematic diagram (partial enlarged view of FIG. 1) showing a state where the sensor unit of FIG. 1 is attached to a living body. [Figure 3] FIG. 3 is a schematic plan view (outer view) showing the sensor unit of FIG. 2. [Figure 4] FIG. 4 is a schematic side cross-sectional view showing the mouthpiece of the musical instrument of FIG. 1. [Figure 5] FIG. 5 is an image diagram showing a first display form displayed on the display unit of FIG. 1. [Figure 6] FIG. 6 is an image diagram showing a second display form displayed on the display unit of FIG. 1. [Figure 7] FIG. 7 is an image diagram showing a second display form displayed on the display unit with different time variation values from FIG. 6. [Figure 8] FIG. 8 is an image diagram showing a second display form showing a linked state different from that of FIG. 7. [Figure 9] FIG. 9 is an image diagram showing a third display form displayed on the display unit of FIG. 1. [Modes for carrying out the invention]
[0023] The embodiments of this disclosure will be described in detail below, with reference to the drawings as appropriate.
[0024] The display device 1 shown in Figure 1 is a display device for performer X playing musical instrument M. The display device 1 comprises a body sensing unit 10, a musical instrument sensing unit 20, and a display unit 30.
[0025] In the display device 1 shown in Figure 1, the instrument M is exemplified as a saxophone, a woodwind instrument. However, it may be other woodwind instruments such as a clarinet, or other types of instruments such as acoustic instruments or brass instruments. As instrument M, a reed instrument that is easily affected by bodily sensing values such as breathing technique and oral pressure during performance is preferred. The following explanation will use the saxophone as an example, but this does not mean that instrument M in the display device 1 is limited to a saxophone.
[0026] <Body Sensing Unit> The body sensing unit 10 acquires one or more body sensing values that represent the body movements of performer X. In the display device 1 shown in Figure 1, the body sensing values are the time variation values of performer X's lung expansion and contraction and the time variation values of performer X's oral pressure. Since these body sensing values tend to affect the sound production of instrument M, it becomes easier for performer X to understand how their body movements affect the sound production of instrument M.
[0027] (Measurement of the time variation of lung expansion and contraction) The time variation of lung expansion and contraction of performer X can be measured by the sensor unit 11 shown in Figure 2. In the display device 1 shown in Figure 1, two sensor units 11 are used; one measures the time variation of lung expansion and contraction in the chest, and the other measures the time variation of lung expansion and contraction in the abdomen.
[0028] Each sensor unit 11 comprises a pair of thread-like or strip-shaped strain sensor elements 11a that expand and contract in the longitudinal direction, and a holding member 11b connected to this pair of strain sensor elements 11a that determines the positional relationship of these strain sensor elements with respect to the biological surface. "Strip-shaped" refers to a long, elongated shape with a width greater than its thickness, and includes configurations where the thickness and width differ in parts.
[0029] As shown in Figure 3, the pair of strain sensor elements 11a are arranged so that their longitudinal direction aligns with the chest and waist circumference of the performer. Furthermore, the pair of strain sensor elements 11 may be positioned to correspond to the left and right lungs, respectively.
[0030] The sensor unit 11 is configured to detect the expansion and contraction of the biological surface, which changes in response to the movement of the living organism, by measuring the resistance values at both ends of the strain sensor element 11a. An example of such a strain sensor element 11a is a CNT strain sensor using carbon nanotubes (hereinafter also referred to as "CNT").
[0031] The strain sensor elements 11a are arranged in a straight line and fixed to the holding member 11b at both ends. The holding member 11b can be composed of, for example, a stretchable fabric 11c and an adhesive portion 11d and a reinforcing member 11e laminated on the fabric 11c. The fabric 11c is configured to cover the surface of a living organism. The reinforcing member 11e is positioned at both ends of the strain sensor elements 11a in a plan view and fixes both ends of the strain sensor elements 11a via the adhesive portion 11d.
[0032] The sensor unit 11 has a pair of strain sensor elements 11a that expand and contract longitudinally in response to the expansion and contraction of the biological surface, making it less likely to compress the biological surface of the subject X. Therefore, the sensor unit 11a is less likely to cause discomfort or unease to the subject X, and can detect the natural movement of the biological surface of the subject X. Accordingly, by using the sensor unit 11, the time variation values of lung expansion and contraction can be measured easily and accurately.
[0033] (Measurement of the time variation of intraoral pressure) In the display device 1, the instrument M is a wind instrument (saxophone) having a mouthpiece M1, and the time variation value of the oral pressure of the performer X can be measured by an oral pressure sensor 12 incorporated into the mouthpiece M1, as shown in Figure 4.
[0034] As shown in Figure 4, the oral pressure sensor 12 comprises a space 12b provided from an opening 12a at the tip of the beak M11 to the boundary between the beak M11 and the barrel M12, and a pressure gauge body 12c provided at the end of the space 12b on the opposite side of the opening 12a.
[0035] With this configuration, when performer X blows air into the mouthpiece M1 from the tip opening M13 to play instrument M, the opening 12a is located inside performer X's oral cavity. Since the other end of the space 12b extending from opening 12a is closed by the pressure gauge body 12c, the value measured by the pressure gauge body 12c can be considered equal to the oral cavity pressure. Therefore, the time variation of the value measured by the pressure gauge body 12c represents the time variation of the oral cavity pressure of performer X.
[0036] <Instrument Sensing Department> The instrument sensing unit 20 acquires one or more instrument sensing values that represent the characteristics of the sound emitted by instrument M. In the display device 1 shown in Figure 1, the instrument sensing values are the time variation of the pressure inside the mouthpiece M1, the time variation of the reed distortion, and the time variation of the loudness of the played sound. These instrument sensing values are easily affected by how the performer X uses their body, making it easier for the performer X to understand how different body movements affect the sound produced by instrument M.
[0037] (Measurement of the time variation of pressure inside the mouthpiece) The time variation of the pressure inside the mouthpiece M1 can be measured by the pressure sensor 21 shown in Figure 4.
[0038] The pressure inside the mouthpiece M1 is the pressure in the flow path M15 through which air blown in from the tip opening M13 flows out to the shank M14, and the pressure sensor 21 is located within the flow path M15. Note that the pressure inside the mouthpiece M1 is the pressure of the air flowing out to the shank M14 and is different from the pressure measured by the intraoral pressure sensor 12.
[0039] (Time-dependent variation of lead distortion) The time variation of lead strain can be measured using the lead strain sensor 22 shown in Figure 4.
[0040] The reed strain sensor 22 is attached to the back surface (the side opposite to the mouthpiece M1) of the reed M2, which is attached to the mouthpiece M1, and detects vibrations of the reed M2. For example, a piezoresistive acceleration sensor or a capacitive acceleration sensor can be used as the reed strain sensor 22. Alternatively, a distance sensor attached to the mouthpiece M1 that measures the distance to the reed M2 may be used as the reed strain sensor 22.
[0041] (Temporal variation in the loudness of the sound played) The time variation in the loudness of the performance sound can be measured using a sound-collecting microphone (not shown).
[0042] The aforementioned sound-collecting microphone may be attached to the instrument M, but it may also be located outside the instrument M, for example, on the display unit 30.
[0043] <Display section> The display unit 30 displays the linked status of two or more sensing values from among the body sensing values and the instrument sensing values.
[0044] The display unit 30 can consist of a display for showing the linked status of two or more sensing values, an interface for receiving instructions from performer X, a CPU for controlling the operation of the display unit 30, and memory for storing control programs and data.
[0045] A display method performed using the display unit 30 is another aspect of this disclosure. That is, this display method is a method of showing the performance status to a performer playing a musical instrument, and displays the linked state of two or more sensing values, which are body sensing values representing the movement of the performer's body and instrument sensing values representing the characteristics of the sound emitted by the instrument. Several display forms that can be displayed on the display unit 30 are described below.
[0046] (1st display form) In the first display mode, the sensing values displayed on the display unit 30 are time-varying values, and as shown in Figure 5, the display on the display unit 30 is a graph represented on a common time axis. When represented on a common time axis in this way, it is easy to grasp the interconnected state of each sensing value.
[0047] Figure 5 displays the time variation values of the pressure inside the mouthpiece M1, the time variation values of the reed distortion, the time variation values of the volume of the sound being played, the time variation values of the expansion and contraction of performer X's lungs, and the time variation values of the oral pressure of performer X. Note that there are two graphs for the time variation values of performer X's lungs, as measurements are taken for both the chest and abdomen.
[0048] In the display configuration shown in Figure 5, all time-varying values are displayed, but the performer X may select which time-varying values to display. Furthermore, the display order may also be configured to be changeable by the performer X.
[0049] Furthermore, a start / stop button for recording may be provided, for example, so that performer X can define the time range for measuring each time variation value. The displayed time variation values may also be provided with functions such as zooming in and out at any point, and playback going back in time.
[0050] (Second display form) In the second display mode, there are three sensing values displayed on the display unit 30. As shown in Figure 6, two of these sensing values are plotted as dots D on a plane represented by orthogonal XY axes, and the magnitude of the third sensing value, which is not plotted as dots D on the plane represented by orthogonal XY axes, is represented by the size of dots D. In Figure 6, the dots D are plotted based on the time variation value of the pressure inside the mouthpiece M1 on the X axis and the time variation value of the reed strain on the Y axis, and the size of dots D represents the oral pressure of the performer X. Figure 6 is just one example, and the sensing values assigned to the X axis, Y axis, and dots D are not limited to these combinations, and it is preferable that the performer X be able to select any configuration.
[0051] Note that there may be two sensing values displayed on the display unit 30. In this case, the two sensing values are plotted as dots D on a plane represented by orthogonal XY axes, and the size of dot D is constant.
[0052] This configuration makes it easier to understand the interconnectedness of two or three sensing values.
[0053] In the sensing value displayed on the display unit 30, which is a time-varying value, it is desirable that the dot D remains displayed for a certain period of time after it is displayed. That is, as shown in Figure 6, the time variation of dot D remains as history R. This makes it easier to grasp the time change in the linked state of the sensing value. In this case, it is preferable that the history R remains as an afterimage display. Here, "afterimage display" means that displays other than the latest dot D are displayed with a different contrast from the latest dot D so that the visibility of the latest dot D is enhanced. For example, it means that displays other than the latest dot D are displayed in a lighter color or a different color with lower brightness than the latest dot D.
[0054] In Figures 7 and 8, there are two sensing values displayed on the display unit 30. The X-axis plots dots based on the time variation of lung expansion and contraction of performer X on the abdominal side, and the Y-axis plots dots based on the time variation of lung expansion and contraction of performer X on the thoracic side.
[0055] When diaphragmatic breathing is performed, the time variation of the lung expansion and contraction of performer X on the X-axis, i.e., the abdominal side, is large, as shown in Figure 7. On the other hand, when thoracic breathing is performed, the time variation of the lung expansion and contraction of performer X on the Y-axis, i.e., the thoracic side, is large, as shown in Figure 8. By displaying the correlation between the time variation of lung expansion and contraction of performer X on the abdominal and thoracic sides in this way, it is possible to determine whether performer X is performing diaphragmatic or thoracic breathing.
[0056] (3rd display form) In the third display mode, the display unit 30 has training data for a song performed by performer X, and the training data has sensing values corresponding to the sensing values of performer X to be displayed on the display unit 30, and the sensing values corresponding to the training data are plotted on the plane as dots D0 and displayed.
[0057] Figure 9 illustrates the display of training data for the second display mode shown in Figure 6. The training data plotted in Figure 9 includes the time variation value of the pressure inside the mouthpiece M1 corresponding to the sensing value of performer X (X axis), the time variation value of the reed strain (Y axis), and the oral pressure of performer X (size of dot D0), with the dot D0 of the training data plotted on the same XY plane.
[0058] The dot D0 in the training data should also be maintained for a certain period of time after being displayed. In other words, the time variation of dot D0 in the training data is recorded as history R0.
[0059] Plotting the training data in this way makes it easy to compare with the training data, leading to a more reliable improvement in performance technique.
[0060] Furthermore, the display unit 30 may emit identification information corresponding to the distance on the plane between the dot D representing performer X and the dot D0 of the training data. By emitting identification information corresponding to the distance in this way, comparison with the training data can be made even easier.
[0061] The aforementioned identification information can include, for example, changes in color, such as the color of dot D representing performer X changing from blue to yellow and then to red as the distance increases, i.e., as performer X's performance deviates from the training data, or changes in the blinking speed of dot D.
[0062] (Other display formats) The first to third display modes described above are merely examples, and the display modes are not limited to these. For example, various display modes can be adopted, such as displaying an animation of the actual jaw movement based on sensing values, showing the oral pressure of performer X as a change in the size of the ripples in that animation, or displaying the time variation of the pressure inside mouthpiece M1 as a bar graph.
[0063] Furthermore, the first to third display modes are not limited to being displayed exclusively, and do not prevent multiple display modes from being displayed simultaneously. In addition, they may be displayed in combination with the other display modes mentioned above.
[0064] [Other embodiments] The embodiments described above do not limit the configuration of the present disclosure. Accordingly, the embodiments may omit, substitute, or add components of each part of the embodiments based on the description herein and common technical knowledge, and all such omissions should be interpreted as falling within the scope of the present disclosure.
[0065] In the above embodiment, a case was described in which the display device includes a body sensing unit and a musical instrument sensing unit, but either one of these may be included. For example, when the linked state of only two or more musical instrument sensing values is displayed on the display unit, the body sensing unit can be omitted.
[0066] However, it is preferable that the display device includes both a body sensing unit and an instrument sensing unit. In this case, the sensing values displayed on the display unit of the display device include both body sensing values and instrument sensing values. By including both body sensing values and instrument sensing values in the sensing values displayed on the display unit in this way, the performer can more directly recognize the relationship between how the body is used and the sound produced by the instrument.
[0067] In the above embodiment, the case was described in which the body sensing values are both the time variation of the performer's lung expansion and contraction and the time variation of the performer's oral pressure, but it may also be only one of them. In addition, other body sensing values may be included as body sensing values in addition to the time variation of the performer's lung expansion and contraction and the time variation of the performer's oral pressure, or in place of one or both of them. Examples of such other body sensing values include heart rate (electrocardiogram), electroencephalogram, and blood oxygen saturation (SpO2).
[0068] In the above embodiment, the instrument sensing values were described as the time variation of the pressure inside the mouthpiece, the time variation of the reed distortion, and the time variation of the loudness of the played sound. However, these may be one or a combination of any two of these values. In addition, other instrument sensing values may be included as instrument sensing values, in addition to or instead of the aforementioned time variation values. Examples of such other instrument sensing values include the pitch of the played sound.
[0069] In the above embodiment, a case was described in which two sensor units, one in the chest and one in the abdomen, each have a pair of strain sensors. However, the number of strain sensors in a sensor unit is not limited to one pair; for example, it may have one strain sensor. In this case, the one strain sensor may be positioned to straddle the left and right lungs. It is not prohibited for the two sensor units in the chest and abdomen to have different numbers of strain sensors.
[0070] In the above embodiment, a case was described in which two sensor units are used, one in the chest and one in the abdomen, but the sensor unit 11 may be either one or the other. [Industrial applicability]
[0071] As described above, the display device and display method according to one embodiment of this disclosure make it easier for performers to understand how their body movements affect the sound production of their instruments, thus leading to a more reliable improvement in their playing technique. Therefore, the display device and display method can be suitably used, for example, for practicing playing musical instruments. [Explanation of Symbols]
[0072] 1 Display device 10 Body Sensing Unit 11 Sensor Unit 11a Strain sensor element 11b Retaining member 11c fabric 11d Adhesive part 11e Reinforcement member 12 Intraoral pressure sensor 12a opening 12b Space 12c pressure gauge body 20. Instrument Sensing Section 21 Pressure Sensor 22 Lead Strain Sensor 30 Display section M Musical Instruments M1 Mouthpiece M11 Beak M12 barrel M13 Tip Opening M14 Shank M15 channel M2 Lead X Performer D, D0 dot R, R0 history
Claims
1. A display device for musicians who play musical instruments, A display device comprising a display unit that displays the linked state of two or more sensing values, which are body sensing values representing the movements of the performer's body and instrument sensing values representing the characteristics of the sound emitted by the instrument.
2. A body sensing unit that acquires one or more of the aforementioned body sensing values, A musical instrument sensing unit that acquires one or more of the aforementioned musical instrument sensing values. The display device according to claim 1, comprising:
3. The display device according to claim 1, wherein the sensing values displayed on the display unit include both the body sensing value and the instrument sensing value.
4. The display device according to claim 1, wherein the body sensing value is the time variation value of the expansion and contraction of the performer's lungs.
5. The display device according to claim 1 or claim 4, wherein the body sensing value is the time variation value of the oral pressure of the performer.
6. The time variation of the performer's lung expansion and contraction is measured by a strain sensor element. The display device according to claim 4, wherein the strain sensor element is in the shape of a thread or strip that expands and contracts in the longitudinal direction, and the longitudinal direction is arranged to follow the chest circumference or waist circumference of the performer.
7. The aforementioned instrument is a wind instrument having a mouthpiece, The display device according to claim 1, wherein the instrument sensing value is a time variation value of the pressure inside the mouthpiece, a time variation value of the reed distortion, or a time variation value of the loudness of the sound produced, or a combination thereof.
8. The sensing value displayed on the display unit is a time-varying value. The display device according to claim 1, wherein the display unit displays a graph represented on a common time axis.
9. The sensing values displayed on the display unit are two or three. The display device according to claim 1, wherein two of the sensing values are plotted as dots on a plane represented by orthogonal X and Y axes.
10. The sensing values displayed on the aforementioned display unit are three in number. The display device according to claim 9, wherein the size of the dot is represented by the size of the dot, and the magnitude of the third sensing value among the sensing values that is not plotted as a dot on a plane represented by orthogonal XY axes.
11. The sensing value displayed on the display unit is a time-varying value. The display device according to claim 9 or claim 10, wherein the dot is maintained for a certain period of time after it is displayed.
12. The display unit has training data for the musical piece to be performed by the performer, The aforementioned training data has sensing values that correspond to the sensing values of the performer displayed on the display unit. The display device according to claim 9 or claim 10, wherein the sensing values corresponding to the training data are plotted as dots on the plane.
13. The display device according to claim 12, wherein the display unit emits identification information corresponding to the distance between the performer's dot and the training data dot on the plane.
14. A method of indicating the playing status to a musician playing a musical instrument, A display method for displaying the linked state of two or more sensing values, which are body sensing values representing the movements of the performer's body and instrument sensing values representing the characteristics of the sound emitted by the instrument.