Playback control method, playback control system, and program

Abstract

The playback control system includes: a state detection unit, which detects a first state in which an object is separated from an operating surface at a specified distance, and a second state in which the object is in contact with the operating surface; and a playback control unit, which starts playing the sound at a first moment when the first state is detected, continues playing the sound from the first moment until a third moment later than the second moment when the second state is detected, and changes the characteristic amount of the sound during a first period from the first moment to the second moment.

Description

Technical Field

[0001] The present disclosure relates to a technology for controlling sound.

[0002] This application claims priority based on Japanese Patent Application No. 2020-050817, filed on March 23, 2020, the disclosure of which is incorporated herein in its entirety. Background Art

[0003] For example, various technologies have been proposed for changing the characteristic values ​​of sounds played in response to user instructions. For example, Patent Document 1 discloses an electronic musical instrument that changes the pitch of sounds played in response to user-pressed keys in response to user operation of a pitch bend wheel (pitch bending).

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2017-161699 Summary of the Invention

[0007] Problems to be solved by the invention

[0008] However, in conventional configurations, users are required to operate the pitch bend wheel independently of instructing sound production by pressing a button. This results in a significant burden on users in terms of the operations required to produce sound. In light of this, one aspect of the present disclosure aims to reduce the burden of user instructions associated with sound production.

[0009] Means for solving problems

[0010] A playback control method involved in one embodiment of the present invention is executed by a computer, and includes: detecting a first state in which an object is separated from an operating surface at a specified distance, and a second state in which the object is in contact with the operating surface, starting the playback of a sound at a first moment when the first state is detected, continuing the playback of the sound from the first moment until a third moment later than the second moment when the second state is detected, and controlling the change of a characteristic quantity of the sound during a first period from the first moment to the second moment.

[0011] A playback control method according to one embodiment of the present disclosure is executed by a computer and includes: playing a sound while an object is in contact with an operating surface, and changing a characteristic value of the sound at a speed corresponding to the speed of movement of the object as the object moves away from the operating surface.

[0012] A playback control system involved in one embodiment of the present invention includes: a state detection unit, which detects a first state in which an object is separated from an operating surface at a specified distance, and a second state in which the object is in contact with the operating surface; and a playback control unit, which starts playing the sound at a first moment when the first state is detected, continues playing the sound from the first moment until a third moment later than the second moment when the second state is detected, and changes the characteristic quantity of the sound within a first period from the first moment to the second moment.

[0013] A program involved in one embodiment of the present invention causes a computer to execute: detecting a first state in which an object is separated from an operating surface at a specified distance, and a second state in which the object is in contact with the operating surface, starting the playback of a sound at a first moment when the first state is detected, continuing the playback of the sound from the first moment until a third moment later than the second moment when the second state is detected, and controlling the change in a characteristic quantity of the sound during a first period from the first moment to the second moment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a block diagram illustrating the structure of a playback control system.

[0015] Figure 2 is a schematic diagram illustrating the structure of a detection unit.

[0016] Figure 3 This is a block diagram illustrating the functional structure of a control system.

[0017] Figure 4 This is an explanatory diagram related to the state of the user's hand.

[0018] Figure 5 It is a flowchart illustrating a specific procedure of the control process.

[0019] Figure 6 This is an explanatory diagram regarding the state of the hand in the third embodiment. DETAILED DESCRIPTION

[0020] A: First embodiment

[0021] Figure 1This is a block diagram illustrating the structure of the playback control system 100 involved in the first embodiment of the present disclosure. The playback control system 100 is a computer system that plays sounds corresponding to user operations (hereinafter referred to as "target sounds"). The playback control system 100 includes a control system 1 and multiple detection units 2. The multiple detection units 2 detect the user's operations. The control system 1 plays the target sounds corresponding to the operations detected by the detection units 2. The target sounds played by the control system 1 are, for example, the sounds of musical instruments such as keyboard instruments. Among them, sounds such as singing sounds or speaking sounds can also be played as target sounds.

[0022] The control system 1 includes a control device 10, a storage device 11, and a sound playback device 13. The control system 1 is implemented by an information terminal such as a smartphone, tablet terminal, or personal computer. The control system 1 can be implemented by a single device or by multiple devices that are configured separately from each other.

[0023] The control device 10 is a single or multiple processors that control each element of the control system 1. Specifically, the control device 10 is composed of one or more processors, such as a CPU (Central Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit). The control device 10 generates an acoustic signal X representing a waveform of a target sound corresponding to a user's operation.

[0024] The sound emitting device 13 plays the target sound represented by the sound signal X generated by the control device 10. The sound emitting device 13 is, for example, a speaker or a headphone. The D / A converter that converts the sound signal X from digital to analog and the amplifier that amplifies the sound signal X are omitted for convenience. Figure 1 In the example shown, the sound emitting device 13 is mounted on the control system 1. In another example, the sound emitting device 13 may be connected to the control system 1 by wire or wirelessly as a separate unit from the control system 1.

[0025] The storage device 11 is a single or multiple memories that store programs executed by the control device 10 and various data used by the control device 10. The storage device 11 is composed of, for example, a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of multiple recording media. Alternatively, the storage device 11 may be provided separately from the control system 1 (e.g., cloud storage), with the control device 10 performing writing to and reading from the storage device 11 via a communication network such as a mobile communication network or the Internet. In other words, the storage device 11 may be omitted from the control system 1.

[0026] The playback control system 100 includes a plurality of detection units 2 corresponding to different pitches (hereinafter referred to as "standard pitches") Ps. Each of the plurality of detection units 2 is an operating element for the user to instruct the playback of a target sound of the standard pitch Ps corresponding to the detection unit 2. The user operates the detection unit 2 corresponding to the desired standard pitch Ps among the plurality of detection units 2, thereby playing the target sound of the standard pitch Ps. Each detection unit 2 constitutes, for example, a key of a keyboard instrument. That is, the arrangement of the plurality of detection units 2 constitutes a keyboard, and the playback control system 100 is implemented as a keyboard instrument.

[0027] Figure 2 2 is a schematic diagram illustrating the structure of an arbitrary detection unit 2. The detection unit 2 includes a container 20, a first detector 21, and a second detector 22. Figure 2 The housing 20 is a hollow structure that accommodates the first detector 21 and the second detector 22. Specifically, the housing 20 includes a box portion 20a and a light-transmitting portion 20b. The box portion 20a is a box-shaped structure having an internal space with an opening at the top. The light-transmitting portion 20b is a plate-shaped member that closes the opening of the box portion 20a. The light-transmitting portion 20b transmits light in a wavelength band that can be detected by the first detector 21. The light-transmitting portion 20b has an operating surface (striking surface) F, which is the surface of the light-transmitting portion 20b on the opposite side of the surface opposite to the box portion 20a. The user can bring the user's own hand H close to or away from the operating surface F and strike the operating surface F with the hand H. The user's hand H is an example of an "object".

[0028] The first detector 21 is an optical sensor that detects the state of the user's hand H. The first detector 21 is arranged near the midpoint (center) of the bottom surface of the box portion 20a. Specifically, a distance measuring sensor that measures the distance between the subject and the light-receiving surface is used as the first detector 21. For example, the first detector 21 receives the reflected light from the hand H that has passed through the light-transmitting portion 20b, thereby generating a detection signal Q1 that represents the position of the hand H in a direction perpendicular to the operating surface F (specifically, the distance from the light-receiving surface to the hand H) in a time series. The detection signal Q1 is sent to the control system 1 via wired communication or wireless communication. The light detected by the first detector 21 is not limited to visible light. The first detector 21 can also detect non-visible light such as infrared light.

[0029] The second detector 22 is a sensor for detecting contact of the hand H with the operating surface F. For example, a sound pickup device that collects ambient sound is used as the second detector 22. The second detector 22 collects the striking sound generated when the user's hand H strikes the operating surface F. The second detector 22 generates a detection signal Q2 representing the ambient sound including the striking sound. The detection signal Q2 is transmitted to the control system 1 via wired or wireless communication. The second detector 22 may also be disposed outside the housing 20.

[0030] Figure 3 1 is a block diagram illustrating a functional configuration of the control system 1. The control device 10 of the control system 1 implements a plurality of functions (a state detection unit 30 and a playback control unit 31) by executing a program stored in the storage device 11.

[0031] The state detection unit 30 detects the state of the user's hand H according to the detection results (detection signal Q1 and detection signal Q2) of each of the plurality of detection units 2. Specifically, the state detection unit 30 detects the first state or the second state as the state of the hand H. Figure 2 As illustrated, the first state is a state where the hand H is separated from the operation surface F of one detection unit 2 by a predetermined distance (hereinafter referred to as a "reference value") Dref. The second state is a state where the hand H is in contact with the operation surface F.

[0032] Figure 4 This is an explanatory diagram related to the state of the hand H. The state detection unit 30 detects that the hand H is in the first state by analyzing the detection signal Q1 generated by the first detector 21. Specifically, the state detection unit 30 estimates the distance D between the operation surface F and the hand H by analyzing the detection signal Q1. The estimation of the distance D is repeated at a predetermined period. That is, data representing the distance D in a time series is generated. Known techniques are arbitrarily used in the estimation of the distance D. Figure 4The figure shows the change in distance D over time. When the distance D between the operation surface F and the hand H matches the reference value Dref, the state detection unit 30 determines that the hand H is in the first state. Alternatively, the state detection unit 30 may determine that the hand H is in the first state when the distance D is within a predetermined allowable range including the reference value Dref. The reference value Dref is a fixed value set in advance. However, the reference value Dref may also be changed in response to user instructions.

[0033] Furthermore, the state detection unit 30 detects that the hand H is in the second state by analyzing the detection signal Q2 generated by the second detector 22. Specifically, the state detection unit 30 estimates the volume V of the sound represented by the detection signal Q2. The estimation of the volume V is repeated at a predetermined period. In other words, data representing the volume V in a time series is generated. The estimation of the volume V can be arbitrarily adopted by a known technique. Figure 4 , the figure shows the temporal changes in the volume V. Striking the operation surface F generates a striking sound, causing the volume V to increase rapidly. When the volume V exceeds a predetermined value (hereinafter referred to as the "reference value") Vref (i.e., when the striking sound is detected), the state detection unit 30 determines that the hand H is in the second state. The reference value Vref is a fixed value set in advance. However, the reference value Vref can also be changed in response to user instructions.

[0034] The user can instruct playback of the target sound of the desired standard pitch Ps by bringing his hand H close to the operating surface F of one of the detection units 2 that corresponds to the desired standard pitch Ps. The user's hand H sequentially enters the first state and the second state during a series of processes of approaching the operating surface F of one of the detection units 2. Specifically, at a specific moment t1 (hereinafter referred to as the "first moment") during the process of hand H approaching the operating surface F, hand H enters the first state, and at a moment t2 after the first moment t1 (hereinafter referred to as the "second moment"), hand H enters the second state.

[0035] The first moment t1 and the second moment t2 are arranged one after the other on the time axis. Figure 4 In FIG, the time (hereinafter referred to as the "third time") t3 after the second time t2 is shown. The third time t3 is the time after a predetermined time length has passed from the second time t2. Figure 4, a first period T1 and a second period T2 are shown. The first period T1 is the period from the first time t1 to the second time t2, and the second period T2 is the period from the second time t2 to the third time t3. The length of the first period T1 (the interval between the first time t1 and the second time t2) varies according to the speed at which the user moves the hand H. Furthermore, the length of the second period T2 (the interval between the second time t2 and the third time t3) can also be changed according to, for example, instructions from the user.

[0036] Figure 3 The playback control unit 31 controls the sound emitting device 13 to play the target sound corresponding to each detection unit 2, depending on the position of the hand H relative to the operation surface F of each detection unit 2. The target sound of the standard pitch Ps corresponding to the detection unit 2 near which the user's hand H is approaching is played. Specifically, the playback control unit 31 generates an acoustic signal X representing the target sound. For example, the storage device 11 stores a plurality of waveform data representing the waveforms of sounds of different standard pitches Ps. The playback control unit 31 reads the waveform data of the standard pitch Ps corresponding to the detection unit 2 near which the user's hand H is approaching from the storage device 11 and processes the waveform data to generate the acoustic signal X. The target sound is played by supplying the acoustic signal X to the sound emitting device 13.

[0037] like Figure 4 As illustrated, when a user's hand H approaches the operating surface F of one of the plurality of detection units 2, the sound generation control unit 31 generates an acoustic signal X to start the generation of the target sound at a first time t1 and to continue the generation of the target sound from the first time t1 until a third time t3 after a second time t2. In other words, the target sound is continuously generated from the first time t1 before the hand H contacts the operating surface F until the third time t3 after the contact.

[0038] The playback control unit 31 varies the pitch P of the target sound over time during a first period T1 from a first time t1 to a second time t2. The playback control unit 31 controls the temporal variation of the pitch P (i.e., the trajectory of the pitch P's change relative to the time axis) during the first period T1. Specifically, during the first period T1, the playback control unit 31 varies the pitch P of the target sound from a first pitch P1 to a standard pitch Ps along a straight line or a curve. The first pitch P1 is a pitch lower than the standard pitch Ps by a predetermined value. Therefore, the first pitch P1 varies for each detection unit 2. Specifically, the playback control unit 31 varies the pitch P of the target sound throughout the first period T1 so that the pitch P of the target sound reaches the first pitch P1 at the first time t1 and reaches the standard pitch Ps at the second time t2. The variation in pitch P during the first period T1 corresponds to a pitch bend of the target sound. The pitch P is an example of a "feature quantity" of the target sound. The standard pitch Ps is an example of a "target value."

[0039] To achieve the change in pitch P described above, the playback control unit 31 changes the pitch P of the target sound during the first period T1 at a speed corresponding to the speed of movement of the user's hand H (hereinafter referred to as "movement speed"). The movement speed is the amount of change per unit time of the distance D estimated by the state detection unit 30. The state detection unit 30 estimates the movement speed by analyzing the detection signal (image signal) Q1. Specifically, the playback control unit 31 changes the pitch P during the first period T1 so that the greater the movement speed, the faster the pitch P changes. With the above configuration, the user can adjust the speed of change of the pitch P during the first period T1 according to the movement speed of the hand H. In the first embodiment, the first pitch P1 and the standard pitch Ps are predetermined. The playback control unit 31 controls the trajectory (speed of change) of the pitch P from the first pitch P1 to the standard pitch Ps during the first period T1 according to the movement speed of the user's hand H. The relationship between the moving speed and the changing speed of the pitch P in the first period T1 may be changed in accordance with an instruction from the user.

[0040] The sound emission control unit 31 maintains the pitch P of the target sound at the standard pitch Ps during the second period T2 from the second time t2 to the third time t3. Specifically, the pitch P of the target sound is fixed at the standard pitch Ps throughout the second period T2.

[0041] Figure 5 This is a flowchart illustrating the specific steps of a process (hereinafter referred to as "control process") Sa executed by the control device 10. For example, the control process Sa is executed in parallel or sequentially for each of the multiple detection units 2. The control process Sa is repeated in a cycle that is sufficiently shorter than the cycle of the user's hand H approaching and separating from the operation surface F.

[0042] When the control process Sa is started, the state detection unit 30 detects the state of the user's hand H by analyzing the detection signal Q1 and the detection signal Q2 supplied from the detection unit 2.

[0043] (Sa1). The playback control unit 31 determines whether the state detection unit 30 detects that the hand H is in the first state (Sa2). If the first state is detected (Sa2: Yes), the playback control unit 31 causes the sound emitting device 13 to start playing the target sound of the first pitch P1 corresponding to the standard pitch Ps of the detection unit 2 (Sa3). If the first state is not detected (Sa2: No), the playback control unit 31 ends the control process Sa.

[0044] If the playback of the target sound starts, the playback control unit 31 changes the pitch P of the target sound toward the standard pitch Ps (Sa4). Specifically, the control device 10 makes the pitch P of the target sound approach the standard pitch Ps by an amount of change corresponding to the movement speed of the user's hand H. The playback control unit 31 determines whether the state detection unit 30 detects that the hand H is in the second state (Sa5). If the second state is not detected (Sa5: No), the playback control unit 31 transfers the processing to step Sa4. That is, within the first period T1 until the second moment t2 when the second state is detected, the pitch P of the target sound changes over time toward the standard pitch Ps. Through the above processing, at the second moment t2 when the second state is detected, the pitch P of the target sound reaches the standard pitch Ps.

[0045] If the second state is detected (Sa5: Yes), the playback control unit 31 maintains the target sound pitch P at the standard pitch Ps (Sa6). The playback control unit 31 then determines whether the third time t3 has been reached (Sa7). Before the third time t3 is reached, the target sound pitch P is maintained at the standard pitch Ps (Sa7: No). That is, the target sound pitch P is maintained at the standard pitch Ps during the second period T2. If the third time t3 is reached (Sa7: Yes), the playback control unit 31 stops playback of the target sound (Sa8).

[0046] As described above, in the first embodiment, the target sound is continuously played from first time t1 when the hand H enters the first state until third time t3 after second time t2 when the hand H enters the second state. Meanwhile, the pitch P of the target sound changes during the first period T1 between first time t1 and second time t2. Thus, the user can change the pitch P of the target sound during the first period T1 simply by moving the hand H close to the operation surface F. This reduces the burden on the user compared to a configuration in which the user must separately instruct the playback of the target sound and the change in pitch P.

[0047] Furthermore, in the first embodiment, since the pitch P of the target sound reaches the standard pitch Ps at the second time t2 when the user's hand H enters the second state of contact with the operation surface F, the user can easily indicate the time at which the pitch P reaches the standard pitch Ps. Furthermore, during the second period T2 from the second time t2 to the third time t3, the pitch P of the target sound is maintained at the standard pitch Ps. This also provides the advantage of allowing the user to easily indicate the playback of the target sound at the standard pitch Ps.

[0048] B: Second embodiment

[0049] In each embodiment described below, elements having the same functions as those in the first embodiment are denoted by the same reference numerals as those used in the description of the first embodiment, and detailed descriptions thereof are omitted as appropriate.

[0050] The playback control system 100 of the second embodiment detects a state (hereinafter referred to as the "third state") in which the hand H that has touched the operation surface F begins to separate from the operation surface F. Specifically, the state detection unit 30 detects a state in which the distance D between the operation surface F and the hand H increases from zero as the third state.

[0051] In the first embodiment, the target sound playback is stopped at the third time t3, which is the time when a predetermined time length has elapsed from the second time t2. In the second embodiment, the target sound playback is stopped at the third time t3, which is the time when the state detection unit 30 detects the third state. That is, while the user's hand H is in contact with the operation surface F, the target sound playback at the standard pitch Ps is maintained, and when the hand H is removed from the operation surface F, the target sound playback is stopped.

[0052] The second embodiment also achieves the same effects as those of the first embodiment. In addition, the second embodiment has the advantage that the user can easily instruct to stop the reproduction of the target sound by removing the hand H from the operation surface F.

[0053] C: Third embodiment

[0054] Figure 6 3 is an explanatory diagram of the reproduction of the target sound in the third embodiment. The state detection unit 30 in the third embodiment detects the third state in which the hand H is separated from the operation surface F, similarly to the second embodiment.

[0055] exist Figure 6 , a time (hereinafter referred to as "fourth time") t4 after the time t3 at which the third state is detected has passed. The fourth time t4 is a time after a predetermined time length has passed from the third time t3. Figure 6The third period T3 is the period from the third time t3 to the fourth time t4. The third period T3 corresponds to the process of the hand H moving away from the operation surface F (the process of the distance D increasing). The length of the third period T3 (the interval between the third time t3 and the fourth time t4) can also be changed according to instructions from the user, for example.

[0056] Similar to the first embodiment, the playback control unit 31 of the third embodiment maintains the pitch P of the target sound, which begins playback at the first time t1, at the standard pitch Ps during the second period T2, and then varies the pitch P of the target sound over time during the third period T3. Specifically, during the third period T3, the playback control unit 31 varies the pitch P of the target sound from the standard pitch Ps to a second pitch P2 along a straight line or a curve. The second pitch P2 is a pitch that is lower than the standard pitch Ps by a predetermined value. Thus, the second pitch P2 varies for each detection unit 2. Specifically, the playback control unit 31 varies the pitch P of the target sound throughout the third period T3 so that the pitch P of the target sound, which was at the standard pitch Ps at the third time t3, reaches the second pitch P2 at the fourth time t4. Whether the first pitch P1 and the second pitch P2 are the same or different, and which pitch is higher (lower), can be arbitrarily set.

[0057] In order to achieve the change in pitch P during the third period T3 described above, the playback control unit 31 changes the pitch P of the target sound during the third period T3 at a speed corresponding to the moving speed of the hand H. Specifically, the playback control unit 31 changes the pitch P during the third period T3 in such a manner that the greater the moving speed, the greater the speed of the change in pitch P. According to the above structure, the user can adjust the speed of the change in pitch P during the third period T3 according to the moving speed of the hand H. The relationship between the moving speed during the third period T3 and the speed of the change in pitch P can also be changed according to instructions from the user.

[0058] D: Modification

[0059] Specific variations of the above-described embodiments are exemplified below. Two or more embodiments arbitrarily selected from the following examples may be combined as appropriate within the scope of non-inconsistency.

[0060] (1) In each of the above embodiments, the case where the first detector 21 is a distance measuring sensor is exemplified, but the type of the first detector 21 is not limited to the above examples. For example, an image sensor that photographs an image of the user's hand H may be used as the first detector 21. In this case, the state detection unit 30 estimates the distance D by analyzing the image of the hand H photographed by the first detector 21, and detects the first state corresponding to the distance D. In addition, an infrared sensor that emits and receives infrared light may be used as the first detector 21. In this case, the state detection unit 30 estimates the distance D based on the light intensity of the infrared light reflected on the surface of the hand H. In addition, the position where the first detector 21 is set is arbitrary. For example, the first detector 21 may also photograph the image of the hand H from the side.

[0061] (2) In each of the above embodiments, the second state is detected by analyzing the detection signal Q2 representing a sound including a striking sound, but the structure and method for detecting the contact of the hand H with the operating surface F are not limited to the above examples. For example, the contact of the hand H with the operating surface F (i.e., the second state) can also be detected by analyzing the detection signal Q1 generated by the first detector 21. For example, when the distance D determined based on the detection signal Q1 reaches zero, the state detection unit 30 determines that the user's hand H is in the second state. In the structure that uses the detection signal Q1 when detecting the second state, the second detector 22 is omitted. In addition, a contact sensor (e.g., an electrostatic capacitance sensor) that detects the contact of the hand H with the operating surface F (translucent portion 20b), a vibration sensor that detects the vibration of the operating surface F (translucent portion 20b), or a pressure sensor that detects the pressure applied from the hand H to the operating surface F can also be used as the second detector 22.

[0062] (3) In each of the above embodiments, a structure in which the user's hand H contacts the operating surface F is illustrated, but the object in contact with the operating surface F is not limited to the hand H. For example, the user may also strike the operating surface F with a striking member such as a stick for a percussion instrument. Based on the above illustrations, it can be understood that the object in contact with the operating surface F includes both a part of the user's body (typically the hand H) and the striking member operated by the user. In the structure in which the striking member strikes the operating surface F, the first detector 21 or the second detector 22 may also be mounted on the striking member.

[0063] (4) The structure of the housing 20 of the detection unit 2 is arbitrary. Furthermore, the structure of housing the first detector 21 and the second detector 22 in the housing 20 is not essential. In other words, as long as the detection unit 2 includes an operating surface F that is contacted by an object such as a user's hand H, the specific structure of the housing 20 and its presence or absence are not particularly limited.

[0064] (5) In each of the above embodiments, the pitch P of the target sound is changed throughout the entire first period T1. However, the pitch P of the target sound may be changed during a portion of the first period T1, while maintaining the pitch P during the remaining period. That is, the pitch P of the target sound may reach the standard pitch Ps before reaching the second time t2. As can be understood from the above description, "changing the pitch P during the first period T1" means changing the pitch P during a portion or the entire first period T1. Similarly, "changing the pitch P during the third period T3" means changing the pitch P during a portion or the entire third period T3.

[0065] (6) In each of the above embodiments, the pitch P of the target sound is changed at a speed corresponding to the speed of movement of the user's hand H, but the method of linking the pitch P with the movement of the user's hand H is not limited to the above examples. For example, the pitch P of the target sound may be changed in accordance with the distance D of the hand H relative to the operation surface F. The playback control unit 31 changes the pitch P of the target sound over time, for example, in such a way that the pitch P increases as the distance D decreases, and the pitch P reaches the standard pitch Ps at the moment when the distance D becomes zero. During the first period T1, the pitch P may be lowered as the distance D increases.

[0066] Furthermore, the playback control unit 31 may also change the pitch P of the target sound in accordance with the direction in which the user's hand H is moving (hereinafter referred to as the "movement direction"). The state detection unit 30 determines the movement direction of the hand H relative to the operating surface F by analyzing the detection signal (image signal) Q1. The playback control unit 31 controls the relationship between the change in pitch P with respect to the time axis (i.e., the trajectory of pitch P with respect to the passage of time) in accordance with the movement direction of the hand H. Specifically, when the angle of the movement direction with respect to the operating surface F is within a first range, the playback control unit 31 changes the pitch P along a first trajectory with respect to the passage of time. On the other hand, when the angle of the movement direction with respect to the operating surface F is within a second range, the playback control unit 31 changes the pitch P along a second trajectory with respect to the passage of time, which is different from the first trajectory. For example, when the angle of the movement direction with respect to the operating surface F is greater than a predetermined threshold, the playback control unit 31 changes the pitch P abruptly with respect to the passage of time (the first trajectory). On the other hand, when the angle of the movement direction with respect to the operating surface F is less than the threshold, the playback control unit 31 changes the pitch P slowly with respect to the passage of time (the second trajectory).

[0067] It can be understood from the above examples that the playback control unit 31 changes the pitch P of the target sound according to the parameters related to the position of the user's hand H. The movement speed, distance D, and movement direction are specific examples of parameters related to the position of the hand H. In addition, the position of the user's hand H itself is also used as a parameter. For example, in addition to the position of the hand H in the direction perpendicular to the operation surface F (such as the distance D), the position of the hand H in the plane parallel to the operation surface F is also included in the parameters related to the position of the hand H. The parameters related to the position of the hand H are determined, for example, by analyzing the detection signal (image signal) Q1, as illustrated in the above embodiments.

[0068] (7) In the above embodiments, the first pitch P1 is shown as being lower than the standard pitch Ps. However, the first pitch P1 may be higher than the standard pitch Ps. That is, within the first period T1, the pitch P of the target sound may be lowered from the first pitch P1 to the standard pitch Ps over time. Furthermore, the first pitch P1 may be set in accordance with an instruction from the user.

[0069] While the third embodiment illustrates a configuration in which the second pitch P2 is lower than the standard pitch Ps, a configuration in which the second pitch P2 is higher than the standard pitch Ps is also contemplated. Specifically, during the third period T3, the target sound pitch P increases over time from the second pitch P2 to the standard pitch Ps. Furthermore, the second pitch P2 can also be set in response to user instructions.

[0070] (8) In each of the above embodiments, the pitch P of the target sound is controlled in accordance with the state of the hand H, but the characteristic quantity of the target sound controlled by the playback control unit 31 is not limited to the pitch P. For example, the volume of the target sound may be controlled in accordance with the state of the user's hand H. In addition, the timbre of the target sound may be controlled in accordance with the state of the user's hand H. For example, the playback control unit 31 generates an acoustic signal X of the target sound that represents a timbre intermediate between the first and second timbres by mixing first waveform data representing a sound of a first timbre with second waveform data representing a sound of a second timbre. The playback control unit 31 changes the mixing ratio of the first waveform data and the second waveform data in accordance with the state of the user's hand H, similar to the pitch P in each of the above embodiments. According to the above configuration, for example, within the first period T1 or the third period T3, the timbre of the target sound can be made to approach the other of the first and second timbres.

[0071] (9) In each of the above embodiments, a structure in which the user's hand H actually contacts the operating surface F is illustrated, but for example, the following structure may be adopted: the user contacts the virtual operating surface F using haptics technology that uses tactile feedback. In this case, the user contacts the operating surface F set in the virtual space by operating a pseudo-hand that exists in the virtual space. By utilizing a vibrating body that vibrates when contacting the operating surface F in the virtual space, the user feels as if actually contacting the operating surface F. Based on the above description, it can be understood that the operating surface F can also be a virtual surface in the virtual space. Similarly, the object that contacts the operating surface F (such as the hand H) can also be a virtual object in the virtual space.

[0072] (10) The functions of the playback control system 100 illustrated above (especially the functions of the control system 1) are realized by the cooperation between the single or multiple processors constituting the control device 10 and the program stored in the storage device 11 as described above. The program involved in the present disclosure can be provided in a form stored in a recording medium that can be read by a computer and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and optical recording media (optical discs) such as CD-ROMs are appropriate examples, but it can also be any known form of recording medium such as a semiconductor recording medium or a magnetic recording medium. The non-transitory recording medium includes any recording medium other than a transitory propagating signal, and does not exclude volatile recording media. In addition, in a structure in which a distribution device distributes a program via a communication network, the storage device storing the program in the distribution device is equivalent to the non-transitory recording medium.

[0073] E: Notes

[0074] According to the above-exemplified embodiment, for example, the following configurations can be understood.

[0075] The playback control method involved in one method (method 1) of the present disclosure is executed by a computer, and includes: detecting a first state in which an object is separated from an operating surface at a specified distance, and a second state in which the object is in contact with the operating surface, starting the playback of a sound at a first moment when the first state is detected, continuing the playback of the sound from the first moment until a third moment later than the second moment when the second state is detected, and controlling the change of a characteristic quantity of the sound during a first period from the first moment to the second moment.

[0076] According to the above method, the user simply moves an object close to the operating surface, and the sound is continuously played from the first moment the object enters the first state until a third moment after the second moment it enters the second state. The characteristic quantity of the sound changes during the first period between the first moment and the second moment. This reduces the user's burden compared to a configuration where the user must separately instruct the sound playback and characteristic quantity changes. The characteristic quantity of the sound may also be, for example, pitch, volume, or timbre.

[0077] In a specific example of Method 1 (Method 2), the control includes changing the characteristic value of the sound during the first period so that the characteristic value of the sound reaches a target value at the second moment. According to the above method, the characteristic value of the sound reaches the target value at the second moment when the object enters the second state of contact with the operating surface. This has the advantage of making it easy for the user to indicate the moment when the characteristic value of the sound reaches the target value.

[0078] In a specific example of aspect 2 (aspect 3), the playback control method further includes maintaining the characteristic value of the sound at the target value during a second period from the second time point to the third time point. According to the above aspect, since the characteristic value of the sound is maintained at the target value during the second period from the second time point to the third time point, the user can easily instruct playback of the sound having the characteristic value at the target value.

[0079] In any specific example of any of Methods 1 to 3 (Method 4), the control includes: varying the characteristic quantity of the sound during the first period according to a parameter related to the position of the object. According to the above method, the user can adjust the trajectory of the variation of the characteristic quantity during the first period according to the position of the object. Examples of parameters related to the position of the object include the speed of the object's movement (Method 5), the distance of the object from the operating surface (Method 6), or the direction of the object's movement (Method 7).

[0080] In any specific example of any one of aspects 1 to 7 (aspect 8), the feature value of the sound is the pitch of the sound. With the above configuration, it is possible to control the change (pitch bend) in pitch at the beginning (within the first period) of the start sound playback.

[0081] A playback control method according to one embodiment of the present invention (method 9) is executed by a computer and includes: playing a sound while an object is in contact with an operating surface, and changing a characteristic value of the sound at a speed corresponding to the speed at which the object moves while the object is moving away from the operating surface.

[0082] A playback control system involved in one embodiment of the present invention (embodiment 10) includes: a state detection unit, which detects a first state in which an object is separated from an operating surface at a prescribed distance, and a second state in which the object is in contact with the operating surface; and a playback control unit, which starts playing the sound at a first moment when the first state is detected, continues playing the sound from the first moment until a third moment later than the second moment when the second state is detected, and changes the characteristic quantity of the sound within a first period from the first moment to the second moment.

[0083] A program involved in one embodiment of the present invention (embodiment 11) causes a computer to execute: detecting a first state in which an object is separated from an operating surface at a specified distance, and a second state in which the object is in contact with the operating surface, starting the playback of a sound at a first moment when the first state is detected, continuing the playback of the sound from the first moment until a third moment later than the second moment when the second state is detected, and controlling the change in the characteristic quantity of the sound during a first period from the first moment to the second moment.

[0084] Industrial applicability

[0085] The present disclosure can also be applied to a playback control method, a playback control system, and a program.

[0086] Label Description

[0087] 100...Playback Control System

[0088] 1...Control System

[0089] 10...Control device

[0090] 11... Storage device

[0091] 13...Playing device

[0092] 2……Detection unit

[0093] 20...container

[0094] 20a……Box

[0095] 20b...light-transmitting part

[0096] 21……First Detector

[0097] 22...Second detector

[0098] 30……Status detection unit

[0099] 31...Playback Control Unit

Claims

1. A playback control method, executed by a computer, comprising: detecting a first state in which an object is separated from an operating surface by a predetermined distance and a second state in which the object is in contact with the operating surface; Starting to play the sound at the first moment when the first state is detected; continuously playing the sound from the first moment to a third moment later than the second moment when the second state is detected; as well as During a first period from the first time to the second time, a change in the feature amount of the sound is controlled.

2. The playback control method according to claim 1, The control includes: The feature value of the sound is changed during the first period so that the feature value of the sound reaches a target value at the second time.

3. The playback control method according to claim 2, further comprising: During a second period from the second time point to the third time point, the feature value of the sound is maintained at the target value.

4. The playback control method according to any one of claims 1 to 3, The control includes: During the first period, the feature amount of the sound is changed according to a parameter related to the position of the object.

5. The playback control method according to claim 4, The parameter is the speed at which the object moves.

6. The playback control method according to claim 4, The parameter is the distance of the object relative to the operating surface.

7. The playback control method according to claim 4, The parameter is the direction of movement of the object.

8. The playback control method according to any one of claims 1 to 3, The characteristic amount of the sound is the height of the sound.

9. The playback control method according to any one of claims 1 to 3, While the object is moving away from the operation surface, the characteristic amount of the sound is changed at a speed corresponding to the speed at which the object moves.

10. A playback control system comprising: a state detection unit that detects a first state in which an object is separated from an operation surface by a predetermined distance and a second state in which the object is in contact with the operation surface; and The playback control unit starts playing the sound at a first moment when the first state is detected, continues playing the sound from the first moment to a third moment later than the second moment when the second state is detected, and changes the characteristic amount of the sound during a first period from the first moment to the second moment.

11. A program that causes a computer to: detecting a first state in which an object is separated from an operating surface by a predetermined distance and a second state in which the object is in contact with the operating surface; Starting to play the sound at the first moment when the first state is detected; continuously playing the sound from the first moment to a third moment later than the second moment when the second state is detected; as well as During a first period from the first time to the second time, a change in the feature amount of the sound is controlled.

Images