System, attachment device and communication method for interacting with stringed instrument

Abstract

A system (10) for interacting with a stringed instrument, comprising: a motion processor for receiving bow motion data relating to motion of a bow when the bow interacts with the stringed instrument; the sound processor is used for receiving acquired sound data of interaction between the stringed musical instrument and the bow; and the motion processor and the sound processor are respectively used for processing the fiddlestick motion data and the sound data so as to provide one or more music teaching, performance or game functions.

Description

Technical Field

[0001] This disclosure relates to a system for interacting with stringed musical instruments and an attachment device for a stringed instrument bow. This disclosure also relates to a communication method between the attachment device for the stringed instrument bow and a control console. Background Technology

[0002] Stringed instruments produce sound through one or more taut and vibrating strings. The vibration of the strings is typically transmitted to the air through the instrument's body. The sound output of a stringed instrument depends on the length, tension, and thickness of the strings. For example, longer strings produce lower pitches than shorter strings, tighter strings produce higher pitches than looser strings, and thicker strings produce lower pitches than thinner strings.

[0003] Playing the violin requires the coordinated use of both hands. The player uses the fingertips of their left hand to press the strings against the fingerboard, shortening the vibrating portion of the string and changing the pitch. By placing the fingers in specific positions, the player can produce different notes and play melodies. The right hand holds the bow, which typically consists of a pernambuco bow stick, an attachable part called a frog, and horsehair. The player applies force to the bow and pulls it across the strings, creating friction and causing the strings to vibrate. Right-hand techniques include controlling the speed, pressure, and direction of the bow to produce a variety of nuanced tonal variations and bowing techniques such as legato, staccato, and spiccato / spiking.

[0004] To play string instruments accurately, a player must possess both the ability to clearly distinguish pitch and the ability to physically produce notes in a consistent manner. A string player's ear needs to be well-trained to differentiate between correct and incorrect notes. Furthermore, string players require extensive practice to develop muscle memory, ensuring that the timing and placement of each finger placement are precise. Summary of the Invention

[0005] According to a first aspect of this disclosure, a system for interacting with a stringed instrument is provided. The system includes: A motion processor is used to receive bow motion data related to the movement of the bow when the bow interacts with the stringed instrument; A sound processor, configured to receive acquired sound data from the interaction between the stringed instrument and the bow; and The motion processor and the sound processor are each further configured to process the bow motion data and the sound data to provide one or more music teaching, performance, or gaming functions.

[0006] In one embodiment of the first aspect, the motion processor is configured to: determine the Euler angles of the bow based on received bow motion data, the Euler angles representing the rotation of the bow relative to a known reference orientation.

[0007] In one embodiment of the first aspect, the motion processor is further configured to: determine the bow posture represented by quaternions based on the determined Euler angles, or determine the bow trajectory by using a gait tracking process.

[0008] In one embodiment of the first aspect, the motion processor is configured to: determine at least one of the velocity and acceleration of the bow based on the received bow motion data.

[0009] In one embodiment of the first aspect, the motion processor is further configured to: determine the tilt angle of the bow based on the received bow motion data.

[0010] In one embodiment of the first aspect, the motion processor is further configured to: determine the bow trajectory based on the bow motion data received within a predetermined time period, which includes at least two measurement data.

[0011] In one embodiment of the first aspect, the bow motion data is received from a motion sensing unit, which is capable of moving with the bow when the bow interacts with the stringed instrument.

[0012] In one embodiment of the first aspect, the sound data is acquired by a sound sensing unit, which is used to acquire sound data related to the interaction between the bow and the stringed instrument.

[0013] In one embodiment of the first aspect, the motion processor and the sound processor are used to synchronize the received bow motion data and the sound data, thereby reducing noise related to the delay between the received bow motion data and the sound data.

[0014] In one embodiment of the first aspect, the bow motion data and the sound data are each processed by the motion processor and the sound processor, such that information related to the processed data is presented to the user at least in an auditory or visual manner.

[0015] In one embodiment of the first aspect, the auditory or visual presentation forms at least a part of the music teaching, performance, or game function.

[0016] In one embodiment of the first aspect, the motion sensing unit and the sound sensing unit are configured to be embedded in the horsehair reservoir of the bow of the stringed instrument when in use.

[0017] In one embodiment of the first aspect, the data transmission of the bow motion data and the sound data is performed using a Bluetooth Low Energy (BLE) device, which facilitates the transmission of the sound data and the bow motion data to the motion processor and the sound processor.

[0018] In one embodiment of the first aspect, the Bluetooth Low Energy device includes a Generic Attribute Profile (GATT) for defining the format of the sound data and the bow motion data in the Bluetooth Low Energy service, thereby reducing noise associated with Bluetooth Low Energy latency between the bow motion data and the sound data arriving at the console.

[0019] In one embodiment of the first aspect, the general attribute profile defines a single feature associated with both the sound data and the bow motion data.

[0020] According to a second aspect of this disclosure, an attachment device for a stringed instrument bow is provided. The attachment device includes: A motion sensor for detecting the movement of the bow when interacting with the stringed instrument; A sound detection device is used to collect the sound of the stringed instrument interacting with the bow; and A control module is used to connect the two sensors and transmit the data to an external device.

[0021] In one embodiment of the second aspect, the motion sensor, the sound detection device, and the control module are used to form part of the horsehair cushion of the bow of the stringed instrument.

[0022] In one embodiment of the second aspect, the attachment device further includes: a horsehair housing for holding the motion sensor, the sound detection device, and the control module, the horsehair housing being at least partially hollow to provide one or more accommodating portions for the motion sensor, the sound detection device, and the control module.

[0023] In one embodiment of the second aspect, the horsehair storage housing includes a first compartment for forming a conventional horsehair storage tenon and a second compartment for forming a receiving portion for the motion sensor, the sound detection device, and the control module, the second compartment being spatially isolated from the first compartment.

[0024] In one embodiment of the second aspect, the second compartment is sandwiched between the first compartment and the attachment portion to the conventional bow.

[0025] In one embodiment of the second aspect, the dimensions of the motion sensor, the sound detection device, and the control module are set to be adapted to the cross-sectional area of ​​the second compartment.

[0026] In one embodiment of the second aspect, the net mass of the hollow horsehair storage shell, the motion sensor, the sound detection device, and the control module is comparable to that of a conventional horsehair storage shell.

[0027] In one embodiment of the second aspect, the center of gravity of the hollow horsehair warehouse shell, the motion sensor, the sound detection device, and the control module is similar to that of a conventional horsehair warehouse.

[0028] In one embodiment of the second aspect, the motion sensor, the sound detection device, and the control module are positioned and tilted toward at least one shell wall of the horsehair storage housing, and the horsehair storage housing further includes a counterweight for balancing the mass of the electronic module.

[0029] In one embodiment of the second aspect, the attachment device further includes a battery compartment for providing power to the motion sensor, the sound detection device, the haptic feedback module, and the control module, the battery compartment serving as a counterweight to maintain the stability of the bow.

[0030] According to a third aspect of this disclosure, a method for communication between an attachment device for a stringed instrument bow and a control console is provided. The method includes: The bow motion data is transmitted from the attachment to the console via a first feature of the service, wherein the bow motion data is related to the movement of the bow when the bow interacts with a stringed instrument; The second feature of the service transmits sound data from the attachment to the console, wherein the sound data is related to the sound of the interaction between the stringed instrument and the bow; Mapping the bow motion data and the sound data; and Based on the mapped data, information related to one or more music teaching, performance, or game functions is presented to the user. The first feature and the second feature are transmitted through a synchronization channel, such that the service is processed when the transmission of both the first feature and the second feature is completed.

[0031] In one embodiment of the third aspect, data transmission is performed using a Bluetooth Low Energy device, which facilitates the transmission of the sound data and the bow movement data to the console.

[0032] In one embodiment of the third aspect, the Bluetooth Low Energy device includes a General Attribute Profile (GATT) for defining the format of the sound data and the bow motion data in the Bluetooth Low Energy service, thereby reducing noise associated with Bluetooth Low Energy latency between the bow motion data and the sound data arriving at the console.

[0033] In one embodiment of the third aspect, the general attribute profile defines a single feature associated with both the sound data and the bow motion data.

[0034] In one embodiment of the third aspect, the single feature data includes the following specifications: 2 bytes - timestamp; 2 bytes - Sensor 1 - X-axis; 2 bytes - Sensor 1 - Y-axis; 2 bytes - Sensor 1 - Z-axis; 2 bytes - Sensor 2 - X-axis; 2 bytes - Sensor 2 - Y-axis; 2 bytes - Sensor 2 - Z-axis; 2 bytes - sensor 3 - X-axis; 2 bytes - sensor 3 - Y-axis; 2 bytes - sensor 3 - Z axis.

[0035] In one embodiment of the third aspect, the single feature data includes the following specifications: 19 bytes - Audio Adaptive Differential Pulse Code Modulation (ADPCM); 5 bytes - Audio sync. Attached Figure Description

[0036] Specific embodiments of this disclosure will now be described by way of example with reference to the accompanying drawings.

[0037] Figure 1 A system for interacting with a stringed instrument according to an embodiment of the present disclosure is shown; Figure 2 A system for interacting with a stringed instrument according to another embodiment of the present disclosure is shown, wherein the system operates within or with a personal device (e.g., a mobile communication device); Figure 3 A stringed instrument interaction method according to an embodiment of the present disclosure is shown, wherein the method is composed of... Figure 1 or Figure 2 The system execution; Figure 4A The startup screen displayed on the mobile device's screen is shown. Figure 4B This shows the user profile interface displayed on a mobile device screen; Figure 4C This shows a friend's profile interface displayed on a mobile device screen; Figure 4D The startup screen displayed on the mobile device's screen is shown. Figure 4E The game launch screen displayed on a mobile device's screen is shown. Figure 4F The market launch screen displayed on a mobile device screen is shown. Figure 5 This is a schematic block diagram illustrating how data is transferred from the attachment device to the central processing unit for data processing; Figure 6A This is a schematic diagram illustrating the structure of a standard Bluetooth Low Energy (BLE) profile in communication between a client and a server. Figure 6B For illustrative purposes, it shows the structure of a Bluetooth Low Energy (BLE) profile in communication between a client and a server according to an example of this disclosure; Figure 7A A bow of a stringed instrument with an attachment device is shown according to an embodiment of the present disclosure; Figure 7B This is a perspective view showing a portion of the bow with an attachment device according to an embodiment of the present disclosure; Figure 7C for Figure 7B A cross-sectional view of the bow with attachment device depicted in the image; Figure 8A This is a perspective view showing a stringed instrument attachment device according to an embodiment of the present disclosure, wherein the attachment device is detached from the bow. Figure 8B It shows Figure 8A The attachment device shown is a two-part housing assembly without a circuit board. Figure 8C It shows Figure 8B The fastening structure between the upper and lower housing components of the attached device shown; Figure 8D It shows Figure 8B The fastening structure between the upper and lower housing components of the attached device shown; Figure 8E It shows Figure 8A Another perspective view of the attachment device shown; Figure 8F for Figure 8E The attached device shown is a cross-sectional view without a circuit board; Figure 9A for Figure 8A A cross-sectional view of the attachment device shown; Figure 9B for Figure 9A A further cross-sectional view of the attachment device shown; Figure 9C for Figure 9A The cross-sectional view of the attachment device shown illustrates the internal arrangement as viewed from the rear end of the bow horsehair storage unit. Figure 10A The horsehair cassette of a standard violin bow is shown; and Figure 10B A bow tail holder according to an embodiment of the present disclosure is shown. Detailed Implementation

[0038] This disclosure relates to a string instrument interaction system or a system for interacting with string instruments. The system, also referred to as a string instrument interaction system, allows an instrumentalist to communicate with the system using a personal communication device and an attachment for detecting sound or bow movement. The user can then send various types of information to the system, such as bow movement data and sound data. The system is used to process data related to the instrumentalist's playing, practice, or performance. The system is also used to present this data in the form of music teaching, performance, or game functions, allowing the instrumentalist to practice independently with minimal instructor guidance or to perform pieces under the system's guidance. In some embodiments, the system can also record, capture, or transmit the instrumentalist's performance as needed. Recording the performer's performance is very useful for playback, review, or comparison with previous or other performances, thereby aiding in learning and improvement.

[0039] The system also allows users to interact with the system or other users regarding specific practice areas or user preferences. It also provides entertainment features. The advantage of these features is that they transform tedious practice into a more engaging platform, offering gamification to the learning and improvement of an instrument. Furthermore, if a player's posture or pitch is incorrect, it can be immediately detected and automatically corrected or adjusted with minimal teacher guidance. This reduces contact time with human instructors, making practice and learning more efficient and enjoyable. Additionally, users can record their performances or practice sessions for progress comparison, enhancing learning effectiveness. These advantages lower the cost of learning an instrument while encouraging new learners to be more interested and engaged in the learning process.

[0040] refer to Figure 1An embodiment of this disclosure is illustrated. This embodiment provides a system for interacting with a stringed instrument. The system includes: a motion processor for receiving bow motion data related to bow motion when the bow interacts with the stringed instrument; and a sound processor for receiving acquired sound data of the interaction between the stringed instrument and the bow. The motion processor and the sound processor are each further configured to process the motion data and the sound data to provide one or more music teaching, performance, or gaming functions.

[0041] This embodiment also provides an attachment device for a stringed instrument bow. The attachment device includes: a motion sensor for detecting the movement of the bow when interacting with a stringed instrument; a sound detection device for collecting the sound from the interaction between the stringed instrument and the bow; and a control module for connecting the two sensors and transmitting the data to an external device.

[0042] Preferably, the music teaching, performance, or game function includes a practice program. This program contains one or more exercises for the user to perform, and the practice program is displayed on a monitor.

[0043] In one exemplary embodiment, the system can be integrated as an application (APP) ecosystem to provide one or more music teaching functions. For example, the ecosystem is used to: (i) record, store, and analyze data related to string instrument playing; (ii) provide real-time feedback through gamified methods adapted to specific student levels / ages; (iii) integrate all these elements into traditional teaching routines to support the work of teachers (e.g., student practice tracking, objective fact indicators, etc.) and students (e.g., real-time feedback, milestone-based approaches); and (iv) modify ensemble performance through time-aligned data analysis and feedback.

[0044] In another exemplary embodiment, the ecosystem may also provide one or more performance functions. For example, posture data extracted from the system may be used in musical performances to enable real-time control of sound or visual events (e.g., digital sound processing in a visual programming language such as MaxMSP or other similar software, or audiovisual mapping in a visual programming environment such as TouchDesigner or other similar software).

[0045] In one example, the processor is used to provide one or more gaming features. For instance, the processor is used to generate and present game exercises on the display so that a user can follow instructions and perform exercises within the exercise program. Alternatively, the processor can also be programmed to present one or more entertainment features on the display as an extension of the gaming features.

[0046] like Figure 1The diagram illustrates a string instrument interaction system 10 and a user 12 (e.g., a string instrument player) interacting with the system 10. The string instrument interaction system 10 includes a computing device 100. Figure 1 As shown, the computing device 100 includes appropriate components necessary for receiving, storing, and executing appropriate computer instructions. These components may include: a processing unit 102, which includes a central processing unit (CPU), a math processing unit (Math Processor), a graphics processing unit (GPU), or a tensor processing unit (TPU) for tensor or multidimensional array computation or manipulation operations; read-only memory (ROM) 104; random access memory (RAM) 106; input / output (I / O) devices, such as a disk drive 108; and a user interface 110 (such as a keyboard or touchscreen). The computing device may also include: other input devices 116, such as an Ethernet port, a USB port, etc.; a display 112, such as a liquid crystal display, a light-emitting display, or any other suitable display; and a communication link (i.e., a communication interface) 114.

[0047] In this exemplary embodiment, processor 102 is used to receive motion and sound data from an external sensing unit. For example, motion and sound data related to user 12 playing a stringed instrument can be recorded by a sensing unit located on the stringed instrument or its components. Processor 102 can be a single processor to provide combined functionality of a motion data processor and a sound data processor.

[0048] The computing device 100 may include instructions that can be stored in ROM 104, RAM 106, or disk drive 108 and executed by processing unit 102. One or more communication interfaces (i.e., one or more communication links) 114 may be provided, which can connect to one or more computing devices, such as servers, personal computers, terminals, wireless or handheld computing devices, Internet of Things (IoT) devices, smart devices, and edge computing devices, in various ways. At least one of the communication links may be connected to an external computing network via a telephone line or other type of communication link. The communication interface 114 is used to allow data communication over any suitable communication network using any suitable protocol (e.g., Wi-Fi, Bluetooth, 4G, 5G, or any other suitable communication protocol).

[0049] In this exemplary embodiment, an IoT device or smart device may be exemplified as attachment device 130. This attachment device is suitable for attachment to, embedding in, or near a stringed instrument played by user 12. As some examples show, the attachment device is implemented as a replacement bow hair holder and is used for embedding or otherwise attaching to the bow. The bow hair holder may be retrofitted to an existing bow or simply be part of a new bow made for a stringed instrument player. Attachment device 130 includes suitable components necessary for sensing, acquiring, and sending / receiving appropriate computer data, electronic signals, or computational instructions. These components may include motion sensor 132 (e.g., an inertial measurement unit (IMU) sensor), sound detection device 134 (e.g., a microphone), and a control module 136 that connects to and controls motion sensor 132 and sound detection device 134. Attachment device 130 may also include a communication module 138. This module is used to transmit signals acquired by motion sensor 132 and sound detection device 134 to user interface 110 or other parts or components of system 10.

[0050] The computing device 100 may include storage devices such as disk drive 108, which may include solid-state drives, hard disk drives, optical drives, tape drives, or remote or cloud-based storage devices. The computing device 100 may use a single disk drive or multiple disk drives, or remote storage services. The computing device 100 may also have a suitable operating system residing on its disk drive or ROM.

[0051] The computing device 100 may also include one or more cameras to capture multiple images or capture video streams. The camera may be a webcam or other suitable camera. The camera may be mounted on the display 112. The camera may also be a separate unit capable of being coupled to one of the input devices 116. The camera is used to communicate electronically with the processor 102. The processor 102 is used to receive recorded images or recorded video from the camera. The processor 102 is also used to process the images or video from the camera. The camera is capable of capturing images or video from the user 12.

[0052] Advantageously, in some exemplary embodiments, the computing device 100 may load a pose estimation or pose detection process. In these examples, the pose estimation process is used to capture the user's motion using a camera, and the processor 102 is accordingly used to apply the pose estimation process or motion capture process to identify the user's motion, posture, or action, including the movement of their body, limb positions, and torso and head / neck positions. The user 12's actions may be processed by the processor 102 along with motion and sound data related to playing a stringed instrument to generate one or more signal outputs (e.g., to animate a virtual avatar in a virtual world and graphically display it on the display 112), and to compare them to obtain the correct body posture or the correct way to hold the instrument.

[0053] The computing device 100 can also provide the computing power required to operate or interface with machine learning networks (e.g., neural networks), thereby providing various functions and outputs. Neural networks can be implemented locally, or accessed or partially accessed via a server or cloud-based service. Machine learning networks can also be untrained, partially trained, or fully trained, and / or can be retrained, modified, or updated over time.

[0054] The computing device 100 includes one or more databases storing various data used by the processor 102. In the illustrated embodiment, the computing device 100 includes a user database 120. The user database includes information about users, such as name, date of birth, age, and address. Database 120 can be created during registration. Users can register via an app on their computing device. Database 120 can be stored on a cloud service and can be accessible. The computing device 100 may also include a profile database 122 storing profile information about users (i.e., profile data). This profile data may include any one or more of the following: learner level, teacher level, practice duration, skill analysis, repertoire (library), achievements. The computing device 100 also includes an entertainment database 124. The entertainment database 124 is used to store various games, user-related exercises and practice programs, sheet music or other memory-stimulating games, or any other audio / video entertainment data. Optionally, one or more of these databases may be stored on a remote server accessible by the device 100.

[0055] The computing device 100 includes a software application (i.e., an APP) stored in a memory unit (e.g., ROM 104, RAM 106, or another memory unit). The software application includes computer-readable and executable instructions. The processor is configured to execute the instructions to cause the processor to perform one or more functions defined in the instructions. The application can control the processor to provide one or more music teaching, performance, or gaming functions.

[0056] In one example, computing device 100 may be a user computing device. Computing device 100 may be a tablet computer, smartphone, laptop, or other personal computing device. An application is installed on computing device 100 and, once executed, controls the functions of computing device 100.

[0057] In one embodiment, computing device 100 may be used to communicate with one or more servers 20 via communication interface 114. In one example, computing device 100 is used to communicate with a virtual world server (e.g., a metaspace server) to allow user access to the metaspace. Processor 102 may be used to generate and animate virtual avatars and send this information to the virtual world server, enabling the virtual avatars to be animated and controllable within the virtual world. The virtual avatar may be a virtual representation of the user, or a character selected by or associated with the user. The computing device allows the selection or creation of customized virtual avatars and allows users to interact with the virtual world through these avatars. A camera is used to capture the user's movements. Processor 102 is used to identify the user's movements by applying a pose estimation process or a motion capture process. The user's movements are used to animate the virtual avatars in the virtual world.

[0058] In this example embodiment, the computing device can be implemented by any computing architecture, including: a portable computer, a tablet computer, a standalone personal computer (PC), a smart device, an IoT device, an edge computing device, a client / server architecture, a "dumb" terminal / host architecture, a cloud computing-based architecture, or any other suitable architecture. The computing device can be appropriately programmed to implement this disclosure. The computing device can execute an application program (APP) to perform various functions defined by the application program.

[0059] In an alternative embodiment, Figure 1 The string instrument interaction system 10 can be modified, adapted, or implemented to operate with mobile communication devices, preferably connected to a server or cloud-based system for data storage or processing. For example, the string instrument interaction system 10a may include a computing device 200, which is substantially identical in setup to the computing device 100, except that the computing device 200 is implemented on a mobile device 210 having a user interface 212. The mobile device 210 may include a processing unit 202, ROM 204, RAM 206, input / output devices 208, a display 212, and a communication interface (e.g., BLE communication 214). The remaining components in the string instrument interaction system 10a can be integrated with… Figure 1 The stringed instrument interaction system 10 shown is the same as or similar to that shown.

[0060] In one exemplary embodiment, the stringed instrument can be implemented by any stringed instrument, including real or virtual instruments, such as a Chinese erhu, violin, electric violin, cello, viola, or other instruments that are not necessarily physical devices. Therefore, it can include physical instruments or physical simulations of instruments, as well as virtual instruments, air guitars, or virtual violins graphically presented on a portable computer (e.g., a tablet), wherein the interaction between the stringed instrument and the bow is not necessarily physical contact; for example, it can be interacted solely through gestures.

[0061] Alternatively, the attachment device 130 mentioned above can be implemented as a wearable device. For example, the wearable device could be a bracelet that can be worn on a user's arm or wrist.

[0062] Figure 3 A block diagram of an example method 300 for interacting with a stringed instrument is shown. Method 300 illustrates functions that can be performed by computing device 100 / 200. An application is executed and method 300 begins. The application contains computer-readable instructions that are executed by a processor and cause computing device 100 / 200 to perform method 300. The method includes step 302. Step 302 includes: identifying a user. In one example, the user is identified through a login process. In this example, a login interface is presented on display 112 / 212 when the application is launched by a musician or user. The user logs in using user-specific credentials. Processor 102 / 202 authorizes the user by verifying the credentials stored in user database 120. This process can be done locally, or processor 102 / 202 can access a server when performing the authorization process. Alternatively, the user can be identified by biometric scanning or facial scanning using a camera.

[0063] After a user logs in, the application controls the processor 102 / 202 to display an initial login screen, a startup screen, or a main screen. The method includes performing one or more functions. The processor 102 / 202 is used to perform one or more functions. Step 304 includes generating a profile screen. Optionally, the user can also interact with one or more virtual buttons, which will display further information. After identifying the user, the method can proceed to step 306. Step 306 includes presenting the user with further information, such as practice time, skill analysis, and a repertoire library.

[0064] After the user interacts with one or more virtual buttons through user interface 110, the method may proceed to step 308. Step 308 includes generating a performance launch interface. In one example, the performance functionality includes a virtual online rehearsal function. Step 310 includes conveying the performance interface to the user after the user interacts with one or more virtual buttons. For example, the processor may be used to convey a string ensemble rehearsal function to present sheet music to the user on a display. The performance interface may also allow the user to practice together and interact with other string ensemble members.

[0065] After the user interacts with one or more virtual buttons through user interface 110, the method may proceed to step 312. Step 312 includes generating a game launch screen. In one example, the game functionality includes a musical instrument playing function. The processor is used to transmit the musical instrument playing function to the display so that it is presented to the user on the display. Step 314 includes conveying the game interface to the user after the user interacts with one or more virtual buttons. Step 316 includes capturing bow movements and the performance of the instrument player. Step 318 includes generating and animateing the user's output.

[0066] After the user interacts with one or more virtual buttons through user interface 110, the method may proceed to step 320. Step 320 includes generating a market launch interface. In one example, the market functionality includes online shopping capabilities.

[0067] Figures 4A to 4F An example implementation of a string instrument interaction system is shown. In this example, the computing device 200 of the string instrument interaction system 10 is a mobile phone with an integrated IMU sensor. The user executes a software application to access the system 10.

[0068] refer to Figure 4A After the user completes the login process, the startup interface (i.e., the main interface) will be displayed on the monitor 212. The startup interface 400 includes multiple interactive virtual buttons 402, 404, 406, and 408, each allowing the user to activate various functions. The profile button 402 allows the user to view their detailed profile. The performance button 404 allows access to functions related to string instrument performance. The game button 406 allows the user to select one or more game functions provided by the string instrument interaction system. The marketplace 408 allows the user to select one or more items for online purchase.

[0069] refer to Figure 4BThe user's profile interface 410 is displayed on monitor 212. The user profile interface 410 can be displayed in response to the user selecting the profile button 402. The user profile interface 410 can contain various information. For example, the user profile interface 410 can display a user profile picture 412. The user profile interface 410 can also include a virtual settings button 413, allowing the user to customize application settings. Other information such as learner level and teacher level can also be displayed next to the user profile. In the upper right corner of the user profile, a pearl-shaped graphic representation 414 is provided, representing the points earned by the performer. Advantageously, the points earned by the user can later be used to purchase rewards, items, or services in the marketplace or exchange.

[0070] Additional virtual buttons are also provided so users can access further information. The Practice Duration button 416 allows users to view trends in their practice time. The Skill Analysis button 418 allows users to view their skill attributes and focus on specific areas needing further improvement. The Repertoire Library button 420 allows users to select one or more sheet music for violin performance. Other user-related information (e.g., user achievements) can also be displayed in buttons 422, 424, 426, and 428.

[0071] refer to Figure 4C The display 212 shows the profile interface 430 of the user's friends. The profile interface 430 can be displayed in response to the user selecting a friend's name from the friend list displayed on the user's profile interface 410. Similarly, the profile interface 430 can display various information of another user on the display 212.

[0072] refer to Figure 4D The display also shows a game launch interface 450, which can be displayed on the monitor 212 in response to the user selecting the game button 406. The game launch interface 450 displays a drop-down menu containing several interactive virtual buttons 452, 454, 456, and 458. The bow tilt button 452 allows the user to activate the bow tilt game. The bow side tilt button 454 allows the user to activate the bow side tilt game. The flying bow button 456 allows the user to activate the flying bow game for practicing spiccato techniques. The sheet music button 458 allows the user to activate the music game function, enabling analysis of the user's bow movement and sound data for further evaluation.

[0073] The following will be referenced Figure 4EThe game launch screen 470 shown depicts a bow tilt game. In response to the user activating the bow tilt button 452, the game launch screen 470 appears on the display 212. During gameplay, the display 212 will display a note 472 on the game launch screen 470, instructing the user to play the equivalent note on the stringed instrument. The system 200 then records and processes the bow movement and the pitch played by the instrumentalist. The processor generates and animates a virtual representation of the bow tilt on the game launch screen 470. The processor also displays how much the pitch played by the instrumentalist differs from a reference pitch.

[0074] The game launch screen 470 may also include a semitone tuner 480 for displaying the pitch deviation from a reference value. A pointer 482 rotates relative to a reference point 484 to indicate the pitch deviation from the reference value. For example, clockwise rotation of pointer 482 indicates the pitch is too high (sharp) and should be lowered. Conversely, counter-clockwise rotation of pointer 482 indicates the pitch is too low (flat) and should be raised. Optionally, an auditory cue indicating this deviation may also be presented to the user.

[0075] For example, a bow tilting game might display an A-flat note 472 on the game launch screen 470. If the chromatic tuner 480 detects that the user has input a B-flat note, it will display a deviation of 22.5 degrees from the reference point 484 on the game launch screen 470.

[0076] refer to Figure 4F The display 212 shows the user teacher's profile interface 490. This profile interface 490 can be displayed in response to the user selecting the market button 408. The teacher profile interface 490 can contain various information. For example, the user profile interface 490 can display a teacher profile picture 492. Other information such as ratings and language proficiency can also be displayed in the teacher profile and below the teacher profile picture 492. In the lower right corner of the teacher profile, a pearl-shaped graphic representation 494 is also provided, representing the points required to interact with the teacher.

[0077] Although not strictly necessary, the embodiments described with reference to the accompanying drawings can be implemented as an application programming interface (API) or as a series of libraries for developers to use, or can be included in another software application, such as a terminal or personal computer operating system or a portable computing device operating system. Typically, since program modules include routines, programs, objects, components, and data files that assist in performing specific functions, those skilled in the art will understand that the functionality of a software application can be distributed across many routines, objects, or components to achieve the same functionality required herein.

[0078] refer to Figure 5Another embodiment of this disclosure is also shown. This embodiment provides a communication method between an attachment device for a stringed instrument bow and a control console. The method includes: transmitting bow motion data from the attachment device to the control console via a first feature of the service, wherein the bow motion data is related to the movement of the bow when the bow interacts with the stringed instrument; transmitting sound data from the attachment device to the control console via a second feature of the service, wherein the sound data is related to the sound of the interaction between the stringed instrument and the bow; mapping the bow motion data and the sound data; and presenting information to a user related to one or more music teaching, performance, or game functions based on the mapped data. The first and second features are transmitted via a synchronous channel, such that the service is processed when the transmission of both the first and second features is complete.

[0079] Figure 5 An explanatory block diagram is shown, depicting the processing of string instrument playing-related data received from the motion sensor 312 of the attached device 130 by the processor 102 / 202 of the system 100 / 200. Data communication between the attached device 130 and the mobile device 210 is achieved via BLE transmission. For example, the attached device 130 can be a BLE client, and the mobile device 210 can be a Bluetooth Low Energy server. Raw data recorded by the attached device 130 can be transmitted to the mobile device 210 via a Bluetooth Low Energy profile.

[0080] In this example embodiment, motion sensor 132 may be embodied as IMU sensor 510, which uses a combination of accelerometer 520, gyroscope 530 and magnetometer 540 to measure the motion of the bow on three axes (x, y and z).

[0081] Accelerometer 520 may be a triaxial accelerometer that provides simultaneous measurements in three orthogonal directions to analyze all vibrations experienced by the structure. Each unit contains three independent sensing elements oriented perpendicular to each other. Gyroscope 530 may be a triaxial gyroscope that measures rotational speed on three axes (pitch, roll, and yaw). Magnetometer 540 may be a triaxial magnetometer that measures magnetic field strength in three dimensions (along the X, Y, and Z axes). Combined measurements can provide a vector indicating the magnetic field strength and direction.

[0082] Therefore, the IMU data captured by a single IMU sensor (Inertial Measurement Unit sensor) 510 will consist of nine data points, including three sets of raw (x, y, z) data from the accelerometer, gyroscope, and magnetometer. Each raw data point and timestamp is 2 bytes in size and is transmitted to the processor 102 / 202 via BLE. The IMU data is processed to extract acceleration, bow orientation, inclination, tilt, skew, bow lateral position, and velocity.

[0083] In one exemplary embodiment, data transmission is achieved via a BLE connection, and data is exported from the IMU sensor 510 to the processor 102 / 202 via a console (e.g., the OSC protocol as shown in step 550).

[0084] After the raw data is transmitted to processors 102 / 202, various instructions are executed to analyze the data. As shown in step 560, processors 102 / 202 convert the initial pose of the bow hair holder 130 (which may be attached, embedded, or otherwise joined or positioned near the bow, instrument, or player) in the raw data obtained from IMU sensor 510 into an initial pose represented by an Euler angle matrix. The Euler angle representation defines the rotation of the bow relative to a known reference orientation based on the received raw data of bow motion. Then, as shown in step 562, processors 102 / 202 convert the initial pose represented by the orientation matrix into an initial pose represented by a quaternion, which is a unit vector in the direction of rotation and a rotation angle. The quaternion represents the orientation of the bow.

[0085] Typically, the accelerometer function of the IMU sensor 510 measures the acceleration of the bow. Acceleration is, in fact, the rate of change of an object's velocity. To determine the velocity, the processor 102 / 202 can be used to integrate the acceleration data over time. Therefore, the processor 102 / 202 can easily process and retrieve the bow velocity and bow acceleration in step 564.

[0086] The position of the bow relative to the strings is typically represented by three bow angles or parameters: tilt angle, slant angle, and lateral tilt angle. The tilt angle determines which string is being played, the slant angle indicates the degree to which the bow deviates from a perpendicular position to the string, and the lateral tilt angle represents the rotation about the long axis of the bow, i.e., the angle between the horsehair strap and the string. While the slant angle is unlikely to affect tone quality, the lateral tilt angle may have a slight impact compared to the tilt and slant angles. To perform a proper bowing motion, the bow path should remain perpendicular to the string throughout the entire movement.

[0087] In one exemplary embodiment, the angles of the bow relative to the stringed instrument (e.g., tilt angle, yaw angle, and yaw angle) are also measured by the IMU sensor 510 and further processed by the processor 102 / 202, as shown in step 566. Based on the bow velocity, bow acceleration, tilt angle, yaw angle, and yaw angle over a predetermined time period, the bow trajectory can also be processed by the processor 102 / 202, as shown in step 568.

[0088] In another example, gait tracking can be used to estimate or determine the trajectory of the bow. By using gait tracking or gait analysis, the player's movements (including movements of the player's anatomy such as the position or movement of their arms, hands, or shoulders) can be processed to determine the movement or trajectory of the bow.

[0089] In addition to motion data, a sound sensing unit (e.g., a microphone) is also provided. Figure 5 (Not shown in the image), which collects sound data related to the interaction between the bow and stringed instruments. The sound data is also transmitted to the same processor 102 / 202 or an additional sound processor in system 100 / 200.

[0090] In one exemplary embodiment, a novel sound feature extraction model is also provided for converting AC audio input into digital audio data. The audio is first recorded as an analog waveform by a sound detection device 134 of the attachment 130, and further converted into a digital format by a sound processor. Essentially, multiple sound features related to different aspects of sound (e.g., spectral envelope, energy, loudness, fundamental frequency, fast Fourier transform (FFT), etc.) are input into a statistical model to model the audio input in a high-resolution audio format with up to, for example, 25 different sound features.

[0091] After processing the audio input, the sound data can be used to calibrate the motion data collected by the motion processor, as shown in step 552. For example, when a user plays an open G string (e.g., without pressing the G string with the left hand fingers) and this is recorded by the sound detection device 134, the sound processor can generate a signal that helps determine the bow tilt angle.

[0092] Optionally, in some embodiments, the attachment device or the horsehair holder 130 may also include a haptic feedback module. In these embodiments, the haptic feedback module (e.g., an electrodynamic or vibratory device) can be controlled to provide haptic feedback in the form of sensory stimulation, vibration, motion, or oscillation. Haptic feedback is advantageous because it provides haptic feedback to the user during interaction with the instrument, thereby providing feedback or guidance in real time.

[0093] Traditionally, violins come in two adult sizes: the "4 / 4" full size (i.e., the standard size), measuring 23 to 23.5 inches long and 14 inches wide, and the "7 / 8" size, measuring 22.5 inches long and 13.5 inches wide. Seven sizes are available for children aged 3 to 12, each based on the player's arm length: 3 / 4, 1 / 2, 1 / 4, 1 / 8, 1 / 10, 1 / 16, and 1 / 32. While the string instrument interaction system 10 is pre-calibrated as the default setting to capture bow movement when played on a full-size violin, an additional custom calibration process can also be implemented within the string instrument interaction system 10 for further calibration to accommodate different string instruments and various sizes.

[0094] In an alternative example embodiment, the string instrument interaction system 10 is pre-tuned to determine acceleration, bow orientation, tilt angle, side tilt angle, yaw angle, bow lateral position, and velocity based on multiple reference parameters of a full-size violin. However, sound data processed by processor 102 / 202 can also be used to determine the type of string instrument played by the user.

[0095] For example, the acceleration and velocity required to play the same note on a full-size violin are different from those on a quarter-size violin. Based on IMU data and sound data, the motion processor 102 / 202 can identify the appropriate size of the violin and thus calibrate the processed data accordingly based on the corresponding reference parameters.

[0096] In another alternative embodiment, the motion processor 102 / 202 can also determine the type of stringed instrument used by the performer based on IMU data. For example, a key difference between a violin or viola and a cello lies in how the performer holds the instrument. A cello is played while seated, with the performer balancing the instrument between their knees and using a tailpiece on the floor. In contrast, violins and violas are placed below the user's chin. Therefore, the bridge orientation of a violin or viola and the bow orientation relative to the ground will be significantly different from that of a cello, which can be easily detected by processing the IMU data obtained by the magnetometer 540.

[0097] In another alternative embodiment, the motion processor can also determine the type of stringed instrument used by the performer based on IMU data and sound data. For example, although violins and violas look very similar, the strings of a violin are G (lowest), D, A, and E (highest), while the strings of a viola are C (lowest), G, D, and A (highest). The bow angle for playing the G, D, or A open strings on a violin is different from the bow angle for playing the G, D, or A open strings on a viola.

[0098] An example of the Bluetooth Low Energy profile used in communication between the attached device 130 and the mobile device 210 will now be described in further detail.

[0099] As the inventors discovered in their research and experiments, Bluetooth Low Energy profiles can be adapted to control communication between two devices. The advantage lies in the ability to design communication processes and protocols for specific purposes to address any particular use case concerns. In one example, a Bluetooth Low Energy profile could have a Generic Attribute Profile (GATT), which specifies the structure for transferring and exchanging profile data between two devices (server and client). The Generic Attribute Profile (GATT) is known to be built on top of the Attribute Protocol (ATT) and defines a framework for organizing resources in each transmission.

[0100] In some examples, the Generic Attribute Profile (GATT) defines two roles: the GATT server and the GATT client. The GATT server can store data transmitted via the Attribute Protocol and accept Attribute Protocol requests, commands, and acknowledgments from the GATT client. The GATT server can also send responses to requests and, when configured, asynchronously send indications and notifications to the GATT client when specific events occur on the GATT server. The Generic Attribute Profile (GATT) can also specify the format of the data contained on the GATT server. Therefore, modifications to the Generic Attribute Profile (GATT) can configure how information is exchanged between the client and the server.

[0101] Figure 6A This is a diagram illustrating the example format of the GATT Service Information Table 600 in a Bluetooth Low Energy connection, based on commonly used protocols. The GATT protocol layer defines a four-layer tree framework. The top layer of the hierarchy is Profile 610 (Level 0). GATT Service Information Table 600 defines basic elements such as Service 612 (Level 1) and Features 614, 616, and 618 (Level 2) used in Profile 610.

[0102] In this example, configuration file 610 may contain one or more services 612 required to implement the use case. A service 612 is a collection of data and related behaviors used to perform a specific function or feature of the device or a part of the device. There are two types of services: primary services and secondary services. Primary services are those that provide the primary functions of the device. Secondary services are those that provide auxiliary functions of the device and are referenced by at least one primary service on the device. Examples of services that may reside in a GATT table on the GATT server include bow motion data and sound data.

[0103] Specifically, service 612 consists of multiple features 614, 616, 618 or references to other services. Each feature 614, 616, 618 contains a single value 614b, 616b, 618b (level 3) used in the service, and attributes 614a, 616a, 618a. Optionally, each feature 614, 616, 618 may also contain configuration information about how to access the values ​​614b, 616b, 618b, and information about how to display or represent the values ​​614b, 616b, 618b. For example, each feature 614, 616, 618 may also include one or more descriptors 614c, 616c, 618c (level 3) that describe feature values ​​614b, 616b, 618b or allow the server to be configured for feature values ​​614b, 616b, 618b so that the values ​​614b, 616b, 618b can be understood by the user.

[0104] Advantageously, by utilizing the predefined structure of services 612, features 614, 616, 618, feature values ​​614b, 616b, 618b, and descriptors 614c, 616c, 618c, GATT clients not specific to configuration file 610 can still traverse the GATT server and display feature values ​​614b, 616b, 618b to the user.

[0105] like Figure 6A As shown, the GATT Service Information Table 600 includes a first Bluetooth feature 614, a second Bluetooth feature 616, and a third Bluetooth feature 618. In existing solutions, accelerometer, magnetometer, and gyroscope data captured by the IMU sensor 510 are transmitted in a single Bluetooth feature 614.

[0106] Audio signals are also encoded and presented using Differential Pulse-Code Modulation (DPCM). Typically, sampled analog signals are digitally represented using digital signals, while the differences between successive samples are calculated and quantized.

[0107] In existing solutions, ADPCM is commonly used for encoding audio signals. Compared to DPCM, ADPCM further modifies the quantization step size and reduces the data bandwidth required for a given signal-to-noise ratio. Furthermore, ADPCM Sync code correction is employed to reduce transcoding distortion. Therefore, ADPCM audio (8bit / 8kHz) and ADPCM sync are also transmitted in two corresponding Bluetooth features, 616 and 618.

[0108] The reading of these feature data 614, 616, and 618 is real-time. However, an unavoidable delay exists between receiving the Bluetooth feature 614 related to bow movement data and receiving the two Bluetooth features 616 and 618 related to audio data. Although this delay only occurs within microseconds, it can introduce latency and noise, thus affecting the accuracy of BLE transmission.

[0109] Figure 6B This is an illustration of an example format of a GATT service information table 620 transmitted between an attached device 130 and a mobile device 210 according to at least one embodiment of this disclosure. The top level of the hierarchy is a configuration file 630, which consists of one or more services 632 required to complete the use case. A service 632 consists of one or more features 634 or references to other services. Each feature 634 contains a value 634b and may contain an optional descriptor 634c for that value 634b.

[0110] and Figure 6A In contrast to the commonly used service 612 in the standard GATT service information table 600, all accelerometer, magnetometer, and gyroscope data, as well as audio data, are contained within a single Bluetooth feature 634 and transmitted as that single feature. Custom firmware according to an example of this disclosure transmits sensor features and audio features in a single feature 634. Transmitting all data together in a single Bluetooth feature 634 prevents the BLE receiver of the mobile device 210 from reading individual accelerometer, magnetometer, and gyroscope data, as well as audio data, in real time, and avoids synchronization problems, thereby achieving the lowest possible latency and significantly reducing noise in the BLE connection.

[0111] In one exemplary embodiment, a 44-byte data packet is sent in the following format:

[0112]

[0113] Although the audio format is compressed in the form of ADPCM in this exemplary embodiment, other alternative codecs or formats may also be used. For example, the audio format may be encoded using one or more DPCM variants.

[0114] Advantageously, the horsehair kimono is specially designed to replicate the feel and function of the traditional horsehair kimono. References will be made now. Figures 7A to 10B Detailed description as per this disclosure Figure 1 and Figure 2 The detailed structure of the attachment device 130 of an example embodiment is shown.

[0115] Figure 7AA violin bow 700 is depicted, primarily comprising a bow stick 710 made of wood or carbon fiber, used to hold violin horsehair 720, typically made of horsehair, in place. The horsehair 720 is used to glide across the strings of the stringed instrument, causing the strings to vibrate and produce sound. The bow stick 710 includes a bow tip 712 at one end, and the horsehair 720 is attached to the bow stick at the bow tip 712 via a bow tip patch 711. At the opposite end of the bow 700, a horsehair holder 730 is provided, housing a mechanism for tightening and loosening the horsehair 720.

[0116] refer to Figures 7B to 7C An example of an innovative horsehair bow attachment 730 with a specific shape and size is provided. The attachment 730 can be adapted to a conventional bow to achieve the same function, while embedding various electronic components of the attachment 130.

[0117] In this example embodiment, the horsehair holder 730 includes several small components that facilitate adjustment and reassembly of the horsehair 720. Specifically, the horsehair holder 730 includes a tongue 731 at its bottom that contacts and accommodates the horsehair 720. As shown in this example embodiment, the horsehair holder 730 includes a ferrule 732, a removable metal plate used to surround and protect the horsehair 720. Bundles 722 of the horsehair 720 pass through the ferrule 732 and are inserted into the horsehair holder. A curved portion extends across the ferrule 732 to form a throat 736, allowing the instrument player to place their thumb in or over the throat to firmly grip the bow 700. Optionally, a circular decorative eye 738 is provided on each side of the horsehair holder 730 for decorative purposes.

[0118] A metal bolt 740 is also provided at the bottom of the horse tail 730. The bolt 740 includes a bolt shank 742 that protrudes from the bottommost end of the bow shank 710, and a screw 744 that extends from the bolt shank along the hollow bow shank 710. The opposite end of the screw 744 is tightened through an eyelet 746, which further extends to form a shank 748 for fixing to the horse tail 730.

[0119] refer to Figures 8A to 8F A horsehair storage unit 730 is also provided, which is at least partially hollow housing 800, to provide one or more compartments for accommodating various electronic components. Preferably, the horsehair storage unit 730 is made of an upper housing member 810 and a lower housing member 820, which together form housing 800 to provide one or more compartments. The upper housing member 810 includes a bottom slide plate 812 extending longitudinally in a direction parallel to the horsehair 720, and further connected to a horsehair storage unit end plate 814 forming the rear surface of the horsehair storage unit 720. Below the bottom slide plate 812 is a reinforcing plate 816, which together form a gap to accommodate a portion of the lower housing member 820.

[0120] The lower shell member 820 is formed by a pair of sidewalls 822 and 824, which are connected to each other by a bottom wall 823. Grooves 826 matching the contour of the bottom slide plate 812 are provided on the upper edges of the sidewalls 822 and 824. A tab 828 extends further along the upper edge of the sidewall 822 to the other sidewall 824, allowing the tab 828 to slide into the gap formed by the bottom slide plate 812 and the reinforcing plate 816, and to be sandwiched between the bottom slide plate 812 and the reinforcing plate 816 to secure the upper and lower shell members 810 and 820.

[0121] Preferably, additional screw and thread arrangements are provided to further ensure the fastening between the upper and lower housing components 810, 820. For example, the bottom slide plate 812 may include a threaded hole 832, and the slot 826 may include a threaded hole 834. When the bottom slide plate 812 is positioned above the slot 826, the threaded holes 832, 834 overlap, allowing screws 836 to be screwed into the threaded holes 832, 834 to fasten the upper and lower housing components 810, 820.

[0122] Fastening devices are also provided for securing the bolt 740 to the tailstock 730. For example, the bottom slide plate 812 may include a threaded hole 842, the reinforcing plate 816 may include a threaded hole 844, and the tab 828 may include a threaded hole 846. When the bottom slide plate 812 is positioned above the slot 826, the threaded holes 842, 844, and 846 overlap, allowing the threads of the shank 748 to be screwed into the threaded holes 842, 844, and 846 to fasten the bolt 740 to the tailstock 730.

[0123] Figures 8E to 8F An example horsehair cassette 730 is depicted in another perspective view to show the bottom components in more detail. The horsehair cassette end plate 814 also extends further from the bottom edge to form a tab 818. The bottom wall 823 of the horsehair cassette also includes an exposed groove 826. The tab 818 is mounted on the groove 826 of the bottom wall 823 to provide a flush surface and form a slide 850. Optionally, screws 852 are also provided for securing the tab 818 to the groove 826.

[0124] refer to Figures 9A to 9C The internal structure of the bow lock 730 is shown. The bow lock 730 contains several compartments for accommodating various components. For example, motion and sound sensing functions are provided by a single PCB board 900, which integrates all bow motion data and sound data capture functions of the aforementioned attachment device 130.

[0125] For example, PCB 900 can be a custom IMU, embedding all necessary sensors (such as gyroscopes, accelerometers, magnetometers, force sensors (FSRs), BLE antennas, and microcontrollers) into the required size. PCB 900 can also implement a haptic feedback module, although the haptic feedback module can be placed elsewhere, but is controlled by circuitry on PCB 900.

[0126] The PCB board 900 may further include a USB port 910 for battery charging. The PCB board 900 also communicates electrically with the onboard battery 920.

[0127] The interior space of the horsehair storage 730 is divided into multiple compartments to accommodate various components. A horsehair storage tenon 930 is also provided on the bottom wall 823 of the horsehair storage, which is a rectangular shell or box extending from the bottom wall 823 of the horsehair storage, for accommodating the bundle of bow hair 722 that passes through the bow hoop 732 and is inserted into the horsehair storage 730.

[0128] To provide users with more tactile feedback, thereby replicating the feel of a traditional bow mortise and tenon, the components are arranged in a compact manner, providing a user experience similar to that of a traditional bow mortise and tenon. In this example embodiment, approximately one-quarter of the cross-sectional area of ​​the housing 800, cut along its longitudinal length, is occupied by a first compartment defined by the bow mortise and tenon 930. The remaining cross-sectional area of ​​the housing 800 is defined as a second compartment, which is spatially isolated from the first compartment. The shape and size of the PCB board 900 are designed to occupy the second compartment. For example, the PCB board 900 is formed in an L-shape, flush with the outer contour defined by the bow mortise and tenon 930, and completely occupies the second compartment.

[0129] Preferably, these components are packaged in a compact manner. For example... Figure 9C As shown in the cross-sectional view, the PCB board 900 is positioned adjacent to the side wall 824 of the horsehair warehouse and is as close to the side wall 824 as possible. This prevents the PCB board 900 from vibrating. On the other hand, a counterweight is provided adjacent to another side wall 822 of the horsehair warehouse, such that the weight of the PCB board 900 and the counterweight are symmetrical about the centerline AB. Optionally, the counterweight can be provided by the battery 920.

[0130] Final Reference Figures 10A to 10B According to an example embodiment of this disclosure, a comparison is shown between a conventional bow hair holder 1000 and a bow hair holder 1100 arranged to act as an attachment device.

[0131] The traditional bow horsehair cassette 1000 includes a solid horsehair cassette body 1030 made of wood. The body 1030 includes a tongue 1031 at the bottom through which horsehair is inserted into the cassette. A pair of Parisian eyes 1038 are provided on the side surface of the cassette body 1030.

[0132] Final Reference Figure 10B The image shows a 3D printable hollow horsehair barrel housing unit 1100 according to an example embodiment of the present disclosure. The interactive interface generated by the horsehair barrel housing unit 1100 is similar to or identical to a conventional bow horsehair barrel 1000 in terms of touch, appearance, and weight, and can be reassembled by any bow maker using conventional procedures.

[0133] The Mawei housing unit 1100 combines a metal cover with a range of other non-metallic materials. For example, the metallic materials can be brass, steel, aluminum, silver, or gold, while the non-metallic materials can be different types of polylactic acid (PLA), photosensitive resin, or wood.

[0134] Specifically, the novel bow hair holder attachment 1100 also includes a housing 1130, but it is formed by an upper housing member 1210 and a lower housing member 1220. The housing 1130 includes a tongue 1131 at its bottom through which horsehair can be installed within the bow hair holder 1100, in the same manner as the conventional bow hair holder 1000. The housing 1130 includes a pair of holes into which a pair of Paris eyelets can be attached. The bow hair holder 1100 also provides a compartment for accommodating a PCB board 1300, which is attached to the rear surface of the lower housing member 1220.

[0135] Typically, the balance point (i.e., center of gravity) of a traditional stringed instrument bow is located approximately one-third of the way from the end of the bow's breechblock. To ensure that the balance point of the stringed instrument bow, incorporating the bow breechblock attachment 1100, remains constant, the net mass of the hollow breechblock housing plus the electronic components, as well as the center of gravity of the entire bow breechblock attachment 1100, should be equivalent to the mass and center of gravity of a traditional breechblock.

[0136] Preferably, the dimensions of the attachment device 1100 can be set according to the horsehair barrel of various stringed instrument bows. For example, the dimensions of the attachment device 1100 can be 16 mm × 49 mm × 22 mm for modification to a traditional violin bow. For example, the dimensions of the attachment device 1100 can be 16 mm × 49 mm × 25 mm for modification to a traditional viola bow. For example, the dimensions of the attachment device 1100 can be 17 mm × 50 mm × 27 mm for modification to a traditional cello bow.

[0137] Advantageously, the novel bow horsehair library 1100 is designed differently from previous string instrument interfaces. It is not complicated or intrusive, and can be integrated into the daily performance practice of string musicians of any level without any retraining.

[0138] Interactive systems for string instruments may be particularly useful for string instrument learners who receive limited personal guidance from a tutor. The various features offered by interactive systems for string instruments can provide a partial self-learning platform and improve the quality of an instrument player's individual practice.

[0139] It should also be understood that when the methods and systems of this disclosure are implemented entirely or partially by a computing system, any suitable computing system architecture may be utilized. This will include stand-alone computers, network computers, and dedicated hardware devices. In the use of the terms "computing device" and "computing apparatus," these terms are intended to cover any suitable arrangement of computer hardware capable of realizing the described functions.

[0140] Those skilled in the art will understand that various variations and / or modifications can be made to the illustrated embodiments of this disclosure without departing from the spirit or scope of this disclosure as broadly described herein. Therefore, these embodiments should be considered illustrative rather than restrictive in all respects.

[0141] Unless otherwise stated, any references to prior art contained herein should not be construed as an admission that the information is common knowledge.

Claims

1. A system for interacting with stringed musical instruments, characterized in that, include: A motion processor is used to receive bow motion data related to the movement of the bow when the bow interacts with the stringed instrument; A sound processor, configured to receive acquired sound data from the interaction between the stringed instrument and the bow; and The motion processor and the sound processor are each further configured to process the bow motion data and the sound data to provide one or more music teaching, performance, or gaming functions.

2. The system according to claim 1, characterized in that, in, The motion processor is used to: determine the Euler angles of the bow based on the received bow motion data, wherein the Euler angles represent the rotation of the bow relative to a known reference orientation.

3. The system according to claim 2, characterized in that, in, The motion processor is also used for: Based on the determined Euler angles, determine the orientation of the bow, represented by a quaternion; or The trajectory of the bow is determined through gait tracking.

4. The system according to claim 1, characterized in that, in, The motion processor is used to: determine at least one of the velocity and acceleration of the bow based on the received bow motion data.

5. The system according to claim 1, characterized in that, in, The motion processor is also used to: determine the tilt angle of the bow based on the received bow motion data.

6. The system according to claim 1, characterized in that, in, The motion processor is also used to: determine the bow trajectory based on the bow motion data received within a predetermined time period, which has at least two measurement data.

7. The system according to claim 1, characterized in that, in, The bow motion data is received from the motion sensing unit. When the bow interacts with the stringed instrument, the motion sensing unit can move together with the bow.

8. The system according to claim 7, characterized in that, in, The sound data is collected by a sound sensing unit, which is used to collect sound data related to the interaction between the bow and the stringed instrument.

9. The system according to claim 1, characterized in that, in, The motion processor and the sound processor are used to synchronize the received bow motion data and the sound data, thereby reducing noise related to the delay between the received bow motion data and the sound data.

10. The system according to claim 1, characterized in that, in, The bow motion data and the sound data are each processed by the motion processor and the sound processor, respectively, so that information related to the processed data is presented to the user at least in an auditory or visual manner.

11. The system according to claim 10, characterized in that, in, The auditory or visual presentation forms at least a part of the music teaching, performance, or play function.

12. The system according to claim 8, characterized in that, in, The motion sensing unit and the sound sensing unit are used to be embedded in the horsehair reservoir of the bow of the stringed instrument during use.

13. The system according to claim 1, characterized in that, in, The data transmission of the bow motion data and the sound data is performed using a Bluetooth Low Energy device, which facilitates the transmission of the sound data and the bow motion data to the motion processor and the sound processor.

14. The system according to claim 13, characterized in that, in, The Bluetooth Low Energy device includes a general attribute profile that defines the format of the sound data and the bow motion data in the Bluetooth Low Energy service, thereby reducing noise associated with Bluetooth Low Energy latency between the bow motion data and the sound data and the console.

15. The system according to claim 14, characterized in that, in, The general attribute configuration file defines a single feature associated with both the sound data and the bow motion data.

16. An attachment device for a bow of a stringed instrument, characterized in that, include: A motion sensor for detecting the movement of the bow when interacting with the stringed instrument; A sound detection device for collecting the sound of the stringed instrument interacting with the bow; as well as A control module is used to connect the two sensors and transmit the data to an external device.

17. The attachment device according to claim 16, characterized in that, in, The motion sensor, the sound detection device, and the control module are used to form part of the horsehair cushion of the bow of the stringed instrument.

18. The attachment device according to claim 17, characterized in that, Also includes: A horsehair housing for holding the motion sensor, the sound detection device, and the control module, the horsehair housing being at least partially hollow to provide one or more accommodating portions for the motion sensor, the sound detection device, and the control module.

19. The attachment device according to claim 18, characterized in that, in, The horsehair storage shell includes a first compartment for forming a conventional horsehair storage tenon and a second compartment for forming a receiving portion for the motion sensor, the sound detection device, and the control module, the second compartment being spatially isolated from the first compartment.

20. The attachment device according to claim 19, characterized in that, in, The second compartment is sandwiched between the first compartment and the attachment portion to the conventional bow.

21. The attachment device according to claim 20, characterized in that, in, The dimensions of the motion sensor, the sound detection device, and the control module are set to be adapted to the cross-sectional area of ​​the second compartment.

22. The attachment device according to claim 16, characterized in that, in, The net mass of the hollow horsehair storage shell, the motion sensor, the sound detection device, and the control module is equivalent to the mass of the conventional horsehair storage replaced by the attachment device.

23. The attachment device according to claim 16, characterized in that, in, The center of gravity of the hollow horsehair storage shell, the motion sensor, the sound detection device, and the control module is approximately equal to the center of gravity of the conventional horsehair storage shell replaced by the attachment device.

24. The attachment device according to claim 16, characterized in that, in, The motion sensor, the sound detection device, and the control module are positioned and tilted toward at least one wall of the horsehair storage housing, and the horsehair storage housing also includes a counterweight for balancing the mass of the electronic module.

25. The attachment device according to claim 24, characterized in that, Also includes: A battery compartment for supplying power to the motion sensor, the sound detection device, the haptic feedback module, and the control module, the battery compartment acting as a counterweight to maintain the stability of the bow.

26. A communication method between an attachment device for a stringed instrument bow and a control console, characterized in that, include: The bow motion data is transmitted from the attachment to the console via a first feature of the service, wherein the bow motion data is related to the movement of the bow when the bow interacts with a stringed instrument; The second feature of the service transmits sound data from the attachment to the console, wherein the sound data is related to the sound of the interaction between the stringed instrument and the bow; Mapping the bow motion data and the sound data; and Based on the mapped data, information related to one or more music teaching, performance, or game functions is presented to the user. The first feature and the second feature are transmitted through a synchronization channel, such that the service is processed when the transmission of both the first feature and the second feature is completed.

27. The communication method according to claim 26, characterized in that, in, Data transmission is performed using a Bluetooth Low Energy device, which facilitates the transmission of the sound data and the bow movement data to the console.

28. The communication method according to claim 27, characterized in that, in, The Bluetooth Low Energy device includes a general attribute profile that defines the format of the sound data and the bow motion data in the Bluetooth Low Energy service, thereby reducing noise associated with Bluetooth Low Energy latency between the bow motion data and the sound data and the console.

29. The communication method according to claim 27, characterized in that, in, The general attribute configuration file defines a single feature associated with both the sound data and the bow motion data.

30. The communication method according to claim 29, characterized in that, in, Single-feature data includes the following specifications: 2 bytes - timestamp; 2 bytes - Sensor 1 - X-axis; 2 bytes - Sensor 1 - Y-axis; 2 bytes - Sensor 1 - Z-axis; 2 bytes - Sensor 2 - X-axis; 2 bytes - Sensor 2 - Y-axis; 2 bytes - Sensor 2 - Z-axis; 2 bytes - sensor 3 - X-axis; 2 bytes - sensor 3 - Y-axis; 2 bytes - sensor 3 - Z axis.

31. The communication method according to claim 29, characterized in that, in, Single-feature data includes the following specifications: 19 bytes - Audio Adaptive Differential Pulse Code Modulation; 5 bytes - audio synchronization.

Images