Intelligent violin, performance control method and multi-instrument synchronous teaching system

Abstract

The present application relates to the technical field of violin, and discloses an intelligent violin, a performance control method and a multi-instrument synchronous teaching system, which comprises a violin body, an intelligent bow and a master control terminal, the violin body is provided with a body and a neck connected to the body, the surface of the neck is provided with a touch sensing array, and the left-hand pressing string action is responded through the touch sensing array, since the touch signal only identifies the electrical change of the finger contact position, the pitch determination is independent of the pressing depth and the strength, and since the mapping relationship between the position and the pitch is fixedly calibrated by the circuit, the learner only needs to focus on the finger position accuracy to obtain stable pitch, thereby reducing the sound control threshold of the non-grade musical instrument, meanwhile, the intelligent bow is used to independently collect and output the bowing posture data, the master control terminal can perform real-time analysis on the touch original signal and the bowing posture data, so that the performer can make targeted adjustment according to the quantitative data, rather than relying on the instantaneous hand feeling, and the learning difficulty is reduced.

Description

Technical Field

[0001] This invention relates to the field of violin technology, and particularly to an intelligent violin, a performance control method, and a multi-instrument synchronous teaching system. Background Technology

[0002] The violin is a stringed instrument without fixed frets, requiring extremely high precision in its playing. Learners need to simultaneously control the left-hand fingering accuracy and various bowing parameters of the right hand. For beginners, middle-aged and elderly learners, and in group teaching settings, the physical strings of the traditional violin can easily cause fingertip strain. It also presents challenges in pitch control and bowing technique, resulting in a relatively high entry barrier and hindering widespread teaching.

[0003] Currently available silent and electroacoustic modified violins still generally retain the physical strings, bridge, and physical bow hair friction sound-generating structure. These devices can only achieve volume control and sound effect optimization, but cannot simplify the core operations of traditional playing, nor can they accurately collect and quantify data on bow speed, bow pressure, bow direction, and various bowing techniques, making it difficult to meet the data guidance needs of standardized teaching.

[0004] Currently available music teaching aids are limited in function, only providing basic assistance such as beat calibration and pitch alignment. They cannot integrate features like finger position touch recognition, bow movement motion sensing, dynamic score synchronization, accompaniment adaptation, and classroom ensemble data. Furthermore, the lack of a unified clock calibration and delay compensation mechanism for multi-device collaboration easily leads to issues such as voice part misalignment and accompaniment asynchrony, severely impacting the effectiveness of intelligent and ensemble violin teaching. Summary of the Invention

[0005] The main objective of this invention is to provide an intelligent violin that addresses the shortcomings of the prior art and solves the problems of difficulty in simplifying the core operations of traditional violin playing, as well as the difficulty in controlling pitch and the instability of bowing techniques.

[0006] To achieve the above objectives, the present invention proposes a smart violin, comprising: The instrument has a body and a neck connected to the body. The surface of the neck is provided with a touch sensor array, which is used to respond to the left hand's string pressing action and output the original touch signal. A smart bow is used to collect and output data on the right hand's bowing posture. The main control terminal is installed on the instrument body. The touch sensor array and the smart bow are both communicatively connected to the main control terminal. The main control terminal is used to receive raw touch signals and bowing posture data, and generate standardized performance event data packets.

[0007] Optionally, the main control terminal includes: A touch acquisition chip is electrically connected to the touch sensing array and is used to acquire the original touch signals of the touch sensing array; The first main control unit is electrically connected to the touch acquisition chip and is used to receive the original touch signal and perform preprocessing. An audio output module, electrically connected to the first main control unit, is used to receive audio signals and output audible sound waves. The first wireless communication module is electrically connected to the first main control unit; The first power module, the touch acquisition chip, the first main control unit, the audio output module and the first wireless communication module are all electrically connected to the first power module.

[0008] Optionally, the touch-sensing array is arranged along the surface of the neck, and the touch-sensing array includes: Four virtual string zones are arranged along the Y-axis, with the four virtual string zones spaced apart, and the four virtual string zones are respectively mapped to the positions of the G string, D string, A string, and E string; Multiple pitch segments are arranged along the X-axis, each pitch segment is mapped to a fixed semitone pitch range, and four virtual string areas and multiple pitch segments constitute a matrix sensing unit, wherein the Y-axis direction is perpendicular to the X-axis direction.

[0009] Optionally, the touch sensing array is a capacitive touch sensor, a pressure film sensor, or an optical touch sensor. Each of the touch sensing arrays is configured to simultaneously acquire three types of raw touch signals, including touch center coordinates, touch contact area, and touch pressure value. Wherein, the touch center coordinates C(t) = (x(t), y(t)), x(t) is the coordinate value along the X-axis direction, and y(t) is the coordinate value along the Y-axis direction.

[0010] Optionally, the audio output module includes: The built-in amplifier speaker is embedded in the body of the instrument for external audio output; A headphone jack is embedded in the body of the instrument and is electrically connected to the first main control unit for connecting headphones. A line output interface is embedded in the body of the instrument and is electrically connected to the first main control unit for connecting external audio equipment.

[0011] Optionally, the smart bow includes: The grip is equipped with a six-axis inertial attitude sensor and a first pressure sensor. The six-axis inertial attitude sensor is used to collect raw data of three-axis acceleration and raw data of three-axis angular velocity. The first pressure sensor is used to collect the pressing stress applied by the right hand fingers to output multi-point pressing signals. The bow is equipped with a second pressure sensor, which is used to collect the deformation stress value of the bow. The bow motion detection unit is electrically connected to the six-axis inertial attitude sensor, the first pressure sensor and the second pressure sensor, receives raw data of acceleration, angular velocity and pressure, and outputs bow direction, bow speed, bow pressure, bow change mark and acceleration change rate. The second wireless communication module is electrically connected to the bow motion detection unit and is used to upload motion data in real time. The second power module is electrically connected to the six-axis inertial attitude sensor, the first pressure sensor, the second pressure sensor, the bow motion detection unit, and the second wireless communication module.

[0012] Optionally, the main control terminal further includes: The second main control unit is used to fuse and process the original touch signal and bow posture data; The audio rendering unit is electrically connected to the second main control unit and is used to generate digital audio signals of violin timbre or MIDI control commands based on the performance event. The third wireless communication module is electrically connected to the second main control unit; The third power supply module is used to supply power to the second main control unit, the audio rendering unit, and the third wireless communication module.

[0013] This application also provides a method for controlling the playing of a smart violin as described above, including the following steps: Real-time acquisition of raw multi-point touch signals from the left hand on the touch sensor array; The smart bow collects real-time data on the right hand's bowing posture, which includes at least bow speed, bow pressure, and direction of movement. The main control terminal receives the original touch signal and bow posture data, and aligns the original touch signal and bow posture data along the time axis according to the acquisition time. The original touch signal, aligned with the timeline, is encapsulated with the bowing posture data to generate a standardized performance event data packet carrying a global timing identifier.

[0014] Optionally, it also includes a preprocessing step of pitch resolution of the original touch signal, specifically including the following steps: Match the corresponding virtual string area based on the Y-axis coordinate value in the touch center coordinate system; Match the corresponding phoneme segment based on the X-axis coordinate value in the touch center coordinate system; Map the matched virtual string regions and pitch segments to the corresponding standard pitches; When multi-finger touch is detected, a valid pitch is locked from multiple touch points according to a preset rule, which is either the highest pitch priority rule or the lowest pitch priority rule.

[0015] Optionally, the preprocessing process further includes identifying slides and shifting actions, specifically including the following steps: Continuously track the longitudinal displacement Δx and lateral displacement Δy of the touch point; When the longitudinal displacement Δx and the lateral displacement Δy satisfy |Δx|≥(2.5×|Δy|), the absolute value of the longitudinal displacement Δx is greater than 50% of the semitone interval, and the continuous movement time of the touch point reaches 80ms, it is determined to be a shift or slide action.

[0016] In addition, this application also provides a multi-instrument synchronous teaching system, including: The aforementioned intelligent violin; The ensemble master control terminal is used to assign vocal parts to at least one of the following instruments: smart violin, smart guitar, smart electric wind instrument, smart electronic drum, smart keyboard, or bass. The synchronization module is used to enable multiple instruments to play or record performance events according to a unified timeline; The teaching error correction module is used to generate single-track scores, full-track scores, or ensemble synchronous scores based on standard scores and real-time performance events.

[0017] Beneficial effects: The intelligent violin proposed in this invention includes a violin body, an intelligent bow, and a main control terminal. The violin body has a body and a neck connected to the body. The surface of the neck is provided with a touch sensor array, which is used to respond to the left hand's string pressing action and output the original touch signal. The intelligent bow is used to collect and output the bowing posture data of the right hand. The main control terminal is installed on the violin body. Both the touch sensor array and the intelligent bow are communicatively connected to the main control terminal. The main control terminal is used to receive the original touch signal and the bowing posture data, and generate standardized performance event data packets. This design uses a touch-sensitive array to respond to the left hand's string-pressing actions. Since the touch signal only recognizes electrical changes at the finger's contact position, pitch determination is independent of the pressing depth and force, reducing the risk of fingertip strain. At the same time, because the mapping relationship between position and pitch is fixed by the circuit, it is no longer affected by string tension, wear, or fluctuations in pressing force. This allows learners to obtain stable pitch by focusing only on the accuracy of finger placement, lowering the threshold for intonation control unique to fretless instruments. In addition, the use of a smart bow to independently collect and output bowing posture data allows the main control terminal to analyze the original touch signal and bowing posture data in real time, enabling performers to make targeted adjustments based on quantitative data rather than relying on instantaneous feel, thus reducing the learning difficulty. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0019] Figure 1 This is a three-dimensional structural diagram of the intelligent violin disclosed in this application; Figure 2 This is the front view of the smart violin disclosed in this application; Figure 3 This is a three-dimensional structural diagram of the intelligent bow disclosed in this application; Figure 4 This is one of the structural block diagrams of the intelligent violin disclosed in this application; Figure 5 This is the second structural block diagram of the intelligent violin disclosed in this application; Figure 6 This is the third structural block diagram of the intelligent violin disclosed in this application; Figure 7 This is a flowchart of the performance control method disclosed in this application.

[0020] Explanation of icon numbers: 10. Body; 11. Body; 12. Neck; 121. Touch sensor array; 20. Smart bow; 21. Grip; 22. Bow stick.

[0021] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0024] In this application, unless otherwise expressly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0025] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the word "and / or" throughout the text means including three parallel solutions; for example, "A and / or B" includes solution A, solution B, or a solution that simultaneously satisfies A and B. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of a person skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0026] The violin is a stringed instrument without fixed frets, requiring extremely high precision in its playing. Learners need to simultaneously control the left-hand fingering accuracy and various bowing parameters of the right hand. For beginners, middle-aged and elderly learners, and in group teaching settings, the physical strings of the traditional violin can easily cause fingertip strain. It also presents challenges in pitch control and bowing technique, resulting in a relatively high entry barrier and hindering widespread teaching.

[0027] Currently available silent and electroacoustic modified violins still generally retain the physical strings, bridge, and physical bow hair friction sound-generating structure. These devices can only achieve volume control and sound effect optimization, but cannot simplify the core operations of traditional playing, nor can they accurately collect and quantify data on bow speed, bow pressure, bow direction, and various bowing techniques, making it difficult to meet the data guidance needs of standardized teaching.

[0028] Currently available music teaching aids are limited in function, only providing basic assistance such as beat calibration and pitch alignment. They cannot integrate features like finger position touch recognition, bow movement motion sensing, dynamic score synchronization, accompaniment adaptation, and classroom ensemble data. Furthermore, the lack of a unified clock calibration and delay compensation mechanism for multi-device collaboration easily leads to issues such as voice part misalignment and accompaniment asynchrony, severely impacting the effectiveness of intelligent and ensemble violin teaching.

[0029] Based on this, this embodiment provides a smart violin, see [link to documentation]. Figures 1-6 As shown, the intelligent violin includes a violin body 10, an intelligent bow 20, and a main control terminal. The violin body 10 has a body 11 and a neck 12 connected to the body 11. A touch sensor array 121 is provided on the surface of the neck 12. The touch sensor array 121 is used to respond to the left hand's string pressing action and output the original touch signal. The intelligent bow 20 is used to collect and output the bowing posture data of the right hand. The main control terminal is installed on the violin body 11. The touch sensor array 121 and the intelligent bow 20 are both communicatively connected to the main control terminal. The main control terminal is used to receive the original touch signal and the bowing posture data and generate standardized performance event data packets. This design, through the touch sensor array 121 responding to the left hand's string-pressing action, ensures that pitch determination is independent of the pressing depth and force, as the touch signal only recognizes electrical changes at the finger contact position, reducing the risk of fingertip strain. Simultaneously, since the mapping relationship between position and pitch is fixed by the circuit, it is no longer affected by string tension, wear, or fluctuations in pressing force. This allows learners to obtain stable pitch simply by focusing on finger position accuracy, lowering the threshold for intonation control inherent in fretless instruments. Furthermore, the intelligent bow 20 independently collects and outputs bowing posture data, and the main control terminal can analyze the original touch signal and bowing posture data in real time, enabling performers to make targeted adjustments based on quantitative data rather than relying on instantaneous feel, thus reducing the learning difficulty.

[0030] Specifically, by adopting the aforementioned structure, the touch-sensing array 121 replaces the physical strings of a traditional violin, making pitch determination entirely dependent on the electrical changes at the finger's contact position, rather than the physical vibration frequency of the strings. Since the touch signal only recognizes the electrical changes at the finger's contact position, pitch determination is independent of the depth and force of the pressure applied, thus reducing the risk of fingertip strain for beginners caused by repeatedly pressing metal strings. Simultaneously, because the mapping relationship between position and pitch is fixed by the circuit, it is no longer affected by nonlinear factors such as changes in string tension, string wear, or fluctuations in pressing force. This allows learners to obtain stable pitch simply by focusing on accurate finger placement, effectively lowering the pitch control threshold inherent in fretless instruments. Furthermore, the intelligent bow 20 independently collects and outputs bowing posture data. The main control terminal analyzes the original touch signal and bowing posture data in real time, enabling performers to make targeted adjustments based on quantitative data, rather than relying on instantaneous feel, thereby reducing learning difficulty and improving practice efficiency.

[0031] In this embodiment, the overall shape of the violin body 10 is constructed to be a streamlined outline that is basically the same as that of a traditional violin, so as to conform to the ergonomic holding posture. The neck 12 extends from one end of the violin body 11, and its length and width can be set with reference to the size proportions of a standard violin, so as to accommodate the hand size range of learners of different ages.

[0032] The touch sensor array 121 is arranged on the upper surface of the neck 12, that is, the area covered by the natural curvature of the other four fingers when the player's left thumb supports the neck 12.

[0033] The touch sensing array 121 is arranged along the surface of the neck 12. The touch sensing array 121 includes four virtual string areas arranged along the Y-axis and multiple pitch segments arranged along the X-axis. The four virtual string areas are spaced apart and are respectively mapped to the pitches of the G string, D string, A string, and E string. Each pitch segment is mapped to a fixed semitone pitch range. The four virtual string areas and multiple pitch segments constitute a matrix sensing unit, wherein the Y-axis direction and the X-axis direction are perpendicular.

[0034] Furthermore, the touch-sensing array 121 is arranged in a matrix structure along the length and width directions of the neck 12. Specifically, the width direction of the neck 12 is defined as the virtual Y-axis, in which four spaced-apart and electrically independent virtual string zones are arranged, each mapping to one of the four strings of a traditional violin: the G, D, A, and E strings. The length direction of the neck 12 is defined as the virtual X-axis, in which multiple consecutive pitch segments are arranged, each pitch segment mapping to a fixed semitone pitch range. The four virtual string zones and the multiple pitch segments together spatially constitute a matrix sensing unit.

[0035] In the aforementioned matrix sensing unit, the Y-axis direction is geometrically perpendicular to the X-axis direction, together constructing a two-dimensional coordinate positioning system for the left hand's string-pressing position.

[0036] By constructing a two-dimensional coordinate matrix, the touch sensor array 121 can identify the absolute position of the left-hand fingers on the surface of the neck 12. Four independent virtual string zones are divided along the width direction (Y-axis), allowing the player's left-hand fingers to trigger the corresponding virtual string zone when moving laterally. The intervals between the four virtual string zones can be set with blanking zones or dead zones to avoid accidental triggering between adjacent string zones. Continuous pitch segments are divided along the length direction (X-axis), each corresponding to a fixed semitone pitch interval. The length of these intervals can be customized according to different teaching stages. For example, in the beginner teaching stage, the length of each pitch segment can be set to be greater than the standard semitone interval, increasing the allowable positioning error range of the fingers in the longitudinal direction, thereby reducing the precision requirements for pitch control. As the student's playing level improves, the pitch segments can be gradually compressed to the standard semitone interval, or even smaller, to meet the requirements of fine finger position control in higher positions. Through the above configuration, the touch sensor array 121 can not only output the position of a single touch point, but also simultaneously output the position information of multiple touch points in scenarios where multiple fingers touch simultaneously.

[0037] In one embodiment of the present invention, the touch sensing array 121 may be a capacitive touch sensor.

[0038] Capacitive touch sensors detect touch events by utilizing changes in the coupling capacitance formed between a human finger and sensing electrodes. They have advantages such as high sensitivity, fast response speed, low power consumption, and are unaffected by ambient light.

[0039] When a finger approaches or touches the sensing area on the surface of the neck 12, the equivalent capacitance of the sensing electrode to ground changes. By detecting this change in capacitance, it is possible to accurately determine whether a finger has touched the surface and the specific location of the touch. To achieve simultaneous multi-point detection, a projected capacitive touch method can be used. Driving electrodes and sensing electrodes are arranged along the X and Y axes, respectively. By scanning the capacitance change at the intersection of the driving line and the sensing line, the two-dimensional coordinates of all touch points can be calculated.

[0040] Alternatively, the touch sensing array 121 can also be implemented using a pressure film sensor. Based on the piezoresistive or piezoelectric effect, the pressure applied by a finger to the surface of the neck 12 can be converted into a change in resistance or charge. It can not only detect the touch position but also output an electrical signal related to the pressure applied, making it particularly suitable for detecting signal fluctuations caused by changes in force applied during vibrato by the left-hand fingers.

[0041] Alternatively, the touch sensing array 121 can also be implemented using an optical touch sensor. For example, an array of infrared light-emitting diodes and a photodetector array can be arranged on the surface of the neck 12 to determine the touch coordinates by detecting the position of the finger reflecting or blocking the infrared beam. Optical touch sensors are not affected by electrostatic interference and can work normally when wearing gloves, making them uniquely applicable in certain special teaching scenarios.

[0042] Regardless of the sensing principle used, each touch sensing array 121 is configured to synchronously acquire three types of raw touch signals: touch center coordinates C(t), touch contact area S(t), and touch pressure value P(t).

[0043] The touch center coordinates C(t) are expressed as a function of time t, C(t) = (x(t), y(t)), where x(t) is the coordinate value of the touch point along the X-axis and y(t) is the coordinate value of the touch point along the Y-axis. The touch contact area S(t) reflects the size of the physical contact area between the finger and the sensing area; this parameter is related to the degree of finger pressure and the amount of fingertip deformation. The touch pressure value P(t) reflects the magnitude of the normal force applied by the finger to the surface of the neck 12. These three types of raw signals, from different physical dimensions, jointly describe the complete state of the left-hand string-pressing action, providing data support for subsequent performance technique recognition.

[0044] In this embodiment, the main control terminal includes a touch acquisition chip, a first main control unit, an audio output module, a first wireless communication module, and a first power supply module. The touch acquisition chip is electrically connected to the output terminal of the touch sensing array 121 via a flexible circuit board or ribbon cable. It is used to acquire changes in the electrical parameters of each sensing node of the touch sensing array 121 at a predetermined scanning frequency, and converts the analog electrical signals into digital signals to obtain the raw touch signal, including touch position, area, and pressure. The sampling frequency of the touch acquisition chip can be set according to actual needs, for example, between 100Hz and 500Hz. A higher sampling frequency helps to capture rapid finger position changes and improves the temporal resolution of glissando and vibrato detection, but it also increases data throughput and power consumption.

[0045] In a preferred embodiment, the sampling frequency of the touch acquisition chip can be set to 200Hz, achieving a good balance between dynamic response and power consumption. The first main control unit is electrically connected to the touch acquisition chip and is used to receive the raw touch signal and perform preprocessing operations on it.

[0046] Preprocessing operations include, but are not limited to, digital filtering, baseline drift compensation, effective touch area segmentation, and multi-touch sequencing. Digital filtering can employ finite impulse response low-pass filters or moving average filters to suppress high-frequency noise components introduced by environmental electromagnetic interference or mechanical vibration. Baseline drift compensation is used to eliminate the slow drift of the capacitive sensing reference value caused by temperature or humidity changes, ensuring the long-term stability of touch detection. Effective touch area segmentation is used to extract the areas where effective touches actually exist from the entire sensing array, excluding proximity sensing caused by a hand approaching but not touching. Multi-touch sequencing is used to determine the tonic and consonant according to preset rules when multiple touch points are detected, providing a basis for subsequent harmony determination.

[0047] The audio output module is electrically connected to the first main control unit and is used to receive audio signals generated by the first main control unit and output audible sound waves. The audio output module can adopt a combination architecture of digital-to-analog converter and power amplifier to convert digital audio signals into analog voltage signals to drive the sound-generating element. The first wireless communication module is electrically connected to the first main control unit and is used to realize wireless data transmission between the main control terminal and the smart bow 20, as well as between the main control terminal and external teaching equipment. The first wireless communication module can support Bluetooth Low Energy protocol, Wi-Fi protocol, or 2.4GHz proprietary radio frequency protocol to realize data interaction. The first power supply module is used to provide operating power to the touch acquisition chip, the first main control unit, the audio output module, and the first wireless communication module. The first power supply module can use a rechargeable lithium-ion battery or a lithium polymer battery and is charged through a universal serial bus interface for convenient use.

[0048] In one implementation of this embodiment, the audio output module includes a built-in power amplifier speaker, a headphone jack, and a line output interface. The built-in power amplifier speaker is embedded in the front or side panel of the instrument body 11, and is used to output sound externally without the need for external audio devices.

[0049] The built-in amplifier speaker can be a full-range driver-coil speaker. The headphone jack is embedded in the body 11 and is electrically connected to the audio output of the first main control unit for connecting external wired headphones. When headphones are plugged in, the first main control unit can automatically detect and cut off the signal path of the built-in amplifier speaker to avoid interference caused by the speaker and headphones emitting sound simultaneously. This mode is suitable for silent practice or nighttime practice scenarios.

[0050] The line output interface is embedded in the surface of the violin body 11. It is electrically connected to the audio output terminal of the first main control unit, allowing connection to professional audio equipment, mixing consoles, or recording equipment via an audio signal cable. The line output interface can be unbalanced mono or stereo output, or a balanced XLR connector, to meet the higher requirements for signal quality and transmission distance in professional performances or recording settings. Through the combination of built-in speakers, headphone jacks, and the line output interface, the intelligent violin can flexibly adapt to various usage scenarios, including personal practice, classroom teaching, stage performances, and recording production, greatly expanding the instrument's applicability.

[0051] In this embodiment, the smart bow 20 is constructed to simulate the shape and feel of a traditional violin bow 22, but integrates a variety of micro sensors and signal processing circuits inside.

[0052] Specifically, the smart bow 20 includes a grip 21, a bow stick 22, a bow motion detection unit, a second wireless communication module, and a second power module. The grip 21 is located at one end of the bow stick 22 and incorporates a six-axis inertial attitude sensor and a first pressure sensor. The six-axis inertial attitude sensor is used to collect raw data of three-axis acceleration and three-axis angular velocity. The six-axis inertial attitude sensor can be an integrated microelectromechanical system inertial measurement unit, such as a chip containing a three-axis accelerometer and a three-axis gyroscope. The accelerometer can measure the linear acceleration vector of the bow stick 22 in three-dimensional space, and the gyroscope can measure the rotational angular velocity vector of the bow stick 22 around the three axes. By performing time integration and attitude calculation on the acceleration and angular velocity, information such as the tilt angle, direction of motion, speed of motion, and rotational attitude of the bow stick 22 in space can be obtained. The first pressure sensor is used to collect the pressing stress applied by the right hand fingers to the bow 22 in order to output multi-point pressing signals. The first pressure sensor can be a resistive thin-film pressure sensor, and its sensitive area covers the gripping positions of the index finger, middle finger, ring finger and little finger.

[0053] The bow shank 22 is equipped with a second pressure sensor, which is used to collect the deformation stress value generated by bending of the bow shank 22 during use. The bow motion detection unit is electrically connected to the six-axis inertial attitude sensor, the first pressure sensor and the second pressure sensor, and is used to receive raw data of acceleration, angular velocity and pressure.

[0054] The bow motion detection unit has a built-in microcontroller that extracts derived parameters such as bow direction, bow speed, bow pressure, bow change markers, and rate of change of acceleration through time-domain analysis. A second wireless communication module is electrically connected to the bow motion detection unit and is used to upload the aforementioned motion data to the main control terminal in real time in the form of data packets.

[0055] To ensure rapid and accurate capture of bowing movements, the data upload sampling frequency of the second wireless communication module is set to 200Hz, meaning one frame of data is sent every 5 milliseconds. A sampling frequency of 200Hz covers the main frequency components of finger and bow stick movements. According to the Nyquist sampling theorem, this sampling rate is sufficient to recover motion signals with a maximum frequency of 100Hz. Therefore, a sampling rate of 200Hz provides ample time resolution, ensuring that every bowing turn and speed change is completely recorded.

[0056] The second power module provides operating power to the six-axis inertial attitude sensor, the first pressure sensor, the second pressure sensor, the bow motion detection unit, and the second wireless communication module. The second power module can also use a miniature rechargeable lithium battery and be charged via wireless charging or a contact charging interface.

[0057] During the operation of the intelligent bow 20, the raw data of triaxial acceleration and triaxial angular velocity from the six-axis inertial attitude sensor are fused and processed by the bow motion detection unit to calculate the real-time motion direction of the bow rod 22.

[0058] The multi-point pressure signals output by the first pressure sensor can be used to identify the performer's bow grip and force distribution, for example, to determine whether the index finger has applied sufficient pressure to generate effective bow pressure. The deformation stress value output by the second pressure sensor reflects the degree of bending of the bow stick 22.

[0059] In this embodiment, the main control terminal also includes a second main control unit, an audio rendering unit, a third wireless communication module, and a third power supply module. The second main control unit is configured to fuse and process the raw touch signal from the touch acquisition chip and the bowing posture data from the smart bow 20. Its processing includes precisely synchronizing the two heterogeneous data sources on the time axis to determine the correspondence between the left hand's string pressing state and the right hand's bowing state at each moment, and then determining the pitch, dynamics, and bowing technique that should be produced at the current moment according to the preset performance logic.

[0060] The audio rendering unit is electrically connected to the second main control unit. It is used to call the built-in violin timbre sample library based on the performance event data generated by the second main control unit to generate digital audio signals with real violin timbre characteristics, or to generate control instructions that conform to the general MIDI standard for external sound source devices to further synthesize sound.

[0061] The audio rendering unit can use a digital signal processor or an ARM-based microprocessor. It stores multiple violin timbre samples with different sampling levels. It can select the most matching sample in real time based on the input velocity value and play it, and achieve a smooth transition of velocity changes through interpolation algorithms.

[0062] The third wireless communication module is electrically connected to the second main control unit and is used to realize data exchange between the main control terminal and external devices, such as sending standardized performance event data packets to the classroom host, teacher tablet, or cloud server. The third power supply module is used to supply power to the second main control unit, the audio rendering unit, and the third wireless communication module.

[0063] The second main control unit also has a built-in automatic calibration process. The automatic calibration process is designed to eliminate individual differences in the force of the bow grip, bowing habits and hand size of different users, so that the sensor data output of the smart bow 20 is normalized to a unified reference standard, thereby improving the accuracy and consistency of subsequent pitch, force and bowing technique judgment.

[0064] The automatic calibration process upon startup is executed in two consecutive stages.

[0065] The first stage is the baseline acquisition stage. During this stage, the user is prompted to maintain a seated posture, naturally hold the smart bow 20, and remain still for 2 seconds. During this period, the six-axis inertial attitude sensor, the first pressure sensor, and the second pressure sensor continuously output sensor data. The main control terminal collects this data as baseline noise samples. The baseline noise samples reflect the background noise level of the sensors under conditions of no effective motion excitation, including random fluctuations introduced by thermal noise from electronic components, quantization noise, and environmental mechanical vibration. After acquisition, the main control terminal performs statistical processing on the baseline noise samples, calculating the mean and standard deviation of each sensor channel.

[0066] The second stage is the standard motion acquisition stage, in which the user is prompted to perform 8 standard bowing movements. The standard bowing movement can be predefined as a smooth full or half bow reciprocating motion performed in the middle bow area at a moderate bow speed and moderate pressure. The main control terminal collects the sensor data during this stage as a standard motion sample, which reflects the signal amplitude range and variation pattern of the user under typical playing conditions.

[0067] Based on the baseline noise samples collected in the first phase, the main control terminal uses the mean of the baseline samples plus three standard deviations as the current minimum effective detection threshold for the user. This threshold setting ensures the confidence level of subsequent effective signal detection. Based on the standard motion samples collected in the second phase, the main control terminal extracts the 20th percentile of these samples as a personalized attack threshold adapted to the current user, and extracts the 95th percentile as a personalized maximum range threshold adapted to the current user. The 20th percentile corresponds to the dynamic level when the performer plays very softly, and the 95th percentile corresponds to the dynamic level when the performer plays loudly. Through these personalized threshold settings, different users can obtain response curves that match their own dynamic habits during performance. Soft playing is less likely to be ignored due to excessively high thresholds, and loud playing is less likely to lose dynamic range due to saturation distortion. This achieves adaptive matching of the instrument to the individual differences of different performers, improving the user-friendliness of the performance experience.

[0068] Furthermore, the personalized attack threshold specifically includes the effective attack determination threshold and the rest determination threshold.

[0069] The effective onset judgment condition is configured as an AND logic that simultaneously satisfies two conditions. The first condition is that the real-time bow speed is greater than 0.08 m / s for more than 30 ms. This condition is set based on physical considerations of the lower limit of the effective movement speed of the bow stick 22. When the bow speed is lower than this value, even if the bow stick 22 contacts the string, it cannot generate enough energy to excite the string to vibrate, which corresponds to ineffective friction without actual sound production in a traditional violin. The second condition is that the real-time bow pressure is greater than 0.12 N for more than 30 ms. The 0.12 N bow pressure threshold is determined based on experimental results of the minimum normal force required to establish effective contact between the bow hair and the string. Only when the above two conditions are simultaneously satisfied and remain stable for more than 30 ms will the main control terminal determine that a note initiation event has occurred and trigger the audio output module to produce sound. The 30 ms duration constraint can effectively avoid false note triggering caused by accidental instantaneous noise spikes or brief sensor disturbances. At the same time, this delay is controlled within the time resolution perceptible to human hearing and will not cause a noticeable delay to the real-time performance.

[0070] The rest determination condition is configured such that when the real-time bow speed is below 0.025 m / s or the real-time bow pressure is below 0.05 N, and this condition persists for more than 80 ms, the main control terminal determines that a note termination event has occurred. The 80 ms duration corresponds to the typical duration required for the reverberation of the sound to decay below the audible threshold after the bow hair leaves the string or the bowing stops in a traditional violin. A reasonable rest determination delay can prevent notes from being interrupted during soft passages or bow changes, making the musical phrases flow naturally and smoothly.

[0071] Furthermore, the second master control unit is configured to calculate the MIDI standard velocity value V according to a specific mapping algorithm. This mapping algorithm nonlinearly weights and combines three dimensions of parameters: normalized bow velocity, normalized bow pressure, and normalized acceleration, to simulate the complex coupling relationship between loudness and the aforementioned multiple physical variables in traditional violin playing. The specific calculation formula is as follows: V=clamp(round(1+70×S n +45×P n +12×A n ), 1, 127) Among them, S n The normalized bow velocity is calculated by dividing the real-time bow velocity by the 95th percentile maximum bow velocity of the calibration sample. When calibration is not performed, the default divisor is 0.55 m / s. The output value of the normalized bow velocity is limited to a closed interval between 0 and 1; it takes the value 1 when the real-time bow velocity exceeds the maximum range, and 0 when the real-time bow velocity is negative or zero. n The normalized bow pressure is calculated by dividing the real-time bow pressure by the 95th percentile maximum bow pressure of the calibration sample. When calibration is not performed, a divisor of 1.20 N is used by default. The output value of the normalized bow pressure is also limited to a closed interval between 0 and 1. A n The normalized acceleration is calculated by dividing the absolute value of the instantaneous acceleration change by a reference threshold of 1.50 m / s², and the output value is restricted to a closed interval between 0 and 1.

[0072] In the above formula, the weighting coefficients for bow speed (70), bow pressure (45), and acceleration (12) are optimized values ​​derived from a large amount of actual performance data through multiple linear regression analysis. These values ​​maximize the match between the calculated dynamic values ​​and the subjective loudness levels produced by professional performers under the same action conditions. The `round` function rounds the integer value to the nearest integer. The `clamp` function restricts the calculated value within the parentheses to a closed interval between 1 and 127. Any calculated value less than 1 is forced to 1, and any calculated value greater than 127 is forced to 127. 1 corresponds to the weakest dynamic in the MIDI standard, and 127 corresponds to the strongest dynamic. Through this mapping algorithm, subtle changes in the dynamics of the performer's right-hand bow stroke are precisely quantified into discrete dynamic levels. These dynamic values ​​not only drive the volume envelope and timbre brightness changes of the internal audio rendering engine but can also be transmitted to external sound sources via the MIDI protocol, achieving compatibility with standard digital music workstations.

[0073] The second main control unit is further configured to execute the automatic bowing technique recognition function. This function classifies bowing techniques by analyzing time series data of multiple kinematic parameters, such as bow speed, bow pressure, rate of change of acceleration, and posture angle, automatically distinguishing five professional bowing techniques: legato, bow change, staccato or accented bowing, spiccato, and vibrato. The specific discrimination logic is as follows: The criteria for legato bowing are: the current performance continuously meets the valid attack criteria, meaning that both bow speed and bow pressure are consistently higher than the valid attack threshold, the interval between two adjacent valid pitch events is less than 250ms, and there is no pause lasting longer than 80ms in between. Under these conditions, the main control terminal determines the current performance state as legato bowing, meaning that multiple pitches are played continuously in the same bow direction without any clear breaks in between. The legato bowing recognition result can be used to control the envelope generator in the audio rendering unit, ensuring that no new volume oscillations are triggered between adjacent notes, thus achieving the smooth timbre transition unique to legato bowing.

[0074] The bow change determination condition is: the angle of reversal of the bow stick 22's movement direction relative to the previous stable posture exceeds 120°, and the effective onset condition is reached again after a 60ms to 120ms action buffer dead zone. The reversal angle is calculated based on the projection angle of the velocity direction vector output by the six-axis inertial attitude sensor onto the horizontal plane. When the bow stick 22's movement direction reverses significantly, it means that the performer has actively changed the bowing direction. The 60ms to 120ms buffer dead zone corresponds to the physical transition time necessary for the bow stick 22 to decelerate, stop, and then accelerate in the opposite direction. Within this dead zone, even if the bow pressure and bow speed temporarily drop below the effective onset threshold, the system does not immediately determine a pause, but waits for the bow change action to be completed before re-triggering the onset, thus avoiding the note being erroneously interrupted at the moment of bow change.

[0075] The criteria for determining staccato or accented notes are: within a 100ms time window, the bow speed decreases by more than 40%, the rate of change of acceleration is greater than 15m / s³, and the bow pressure remains within the effective attack range. This parameter mode corresponds to the performer producing staccato or accented effects with sharp attack and rapid decay characteristics by instantaneously accelerating and suddenly decelerating the bow 22 while maintaining a large bow pressure.

[0076] The criteria for determining a spiccato bow are: the peak vertical acceleration is greater than 1.5 times the gravitational acceleration, i.e., the peak vertical acceleration is greater than 14.7 m / s², and the bow pressure is below the rest threshold for 30 ms to 120 ms before quickly recovering to above the effective onset threshold.

[0077] The criteria for vibrato are: the bow speed and direction produce a periodic, small-amplitude reciprocating motion of 6Hz to 12Hz, and the single reciprocating stroke does not exceed 15% of the full bow stroke. This parameter mode corresponds to the performer rapidly shaking their wrist to make the bow stick 22 reciprocate over a very short distance, producing a dense vibrato effect.

[0078] The standardized performance event data packet is designed to include the following data fields: unique device ID, part number, target pitch, calculated velocity value, start timestamp, finish timestamp, bowing type marker, local device clock value, and estimated global playback time. The unique device ID is used to uniquely identify each instrument in ensemble or multi-device network scenarios. The part number distinguishes the different part roles played by different instruments in the same ensemble piece. The target pitch is the standard pitch value after resolution by the touch-sensitive array 121, represented in MIDI note number format. The calculated velocity value is the standard MIDI velocity value obtained according to the aforementioned mapping algorithm. The start and finish timestamps mark the precise moments when the note begins and ends, respectively, with a time resolution of milliseconds or higher. The bowing type marker is the output of the aforementioned automatic bowing type recognition function, such as legato, bow change, staccato, spiccato, or vibrato markers. The local device clock value records the count value of the device's internal clock at the time the data packet is generated, used in the time synchronization algorithm to compensate for clock drift between devices. The estimated global playback time is calculated by the main control terminal based on the time synchronization algorithm and is used to indicate the target playback time on the global timeline when the audio rendering unit plays the data packet.

[0079] To further explain, the present invention also provides a method for controlling the playing of an intelligent violin, see [link to relevant documentation]. Figure 7 As shown, the performance control method includes the following steps: S1. Real-time acquisition of multi-point touch signals from the left hand on the touch sensor array 121; Real-time acquisition of bowing posture data of the right hand by the smart bow 20, which includes at least bow speed, bow pressure and direction of movement. S2. The main control terminal receives the original touch signal and bow posture data, and aligns the original touch signal and bow posture data along the time axis according to the acquisition time. S3. Encapsulate the original touch signal after time axis alignment with the bowing posture data to generate a standardized performance event data packet carrying a global timing identifier.

[0080] Specifically, the system acquires real-time multi-point touch signals from the left hand on the touch sensor array 121. This step is performed periodically by the touch acquisition chip according to a set sampling frequency. The smart bow 20 acquires real-time bowing posture data from the right hand. The bowing posture data includes at least bow speed, bow pressure, and direction of movement. This step is obtained by the bow motion detection unit through the fusion of a six-axis inertial attitude sensor and a pressure sensor. The main control terminal receives the original touch signals and bowing posture data, and aligns the original touch signals and bowing posture data along the time axis according to the acquisition time.

[0081] The specific implementation method of time axis alignment is as follows: the main control terminal adds a local clock tag of the receiving time to each frame of raw touch data and each frame of bow posture data. Then, according to the clock tag, the two data sequences are rearranged in chronological order so that the touch signal and motion signal collected at the same time are merged into the same processing cycle.

[0082] Finally, the original touch signal after time-axis alignment is encapsulated with the bowing posture data to generate a standardized performance event data packet carrying a global timing identifier. Through the above time-axis alignment process, the time offset introduced by the independent sampling of the touch sensing array 121 and the smart bow 20 and the different data transmission paths is eliminated, ensuring that pitch events and dynamic change events strictly correspond in time, and avoiding timbre distortion and rhythm errors caused by time misalignment.

[0083] The performance control method also includes a preprocessing step of pitch resolution of the original touch signal.

[0084] The preprocessing process specifically includes the following steps: S21. Match the corresponding virtual string zone based on the Y-axis coordinate value of the touch center coordinate system; match the corresponding pitch segment based on the X-axis coordinate value of the touch center coordinate system. S22. Map the matched virtual string regions and pitch segments to the corresponding standard pitches; S23. When multi-finger touch is detected, lock a valid pitch from multiple touch points according to preset rules. The preset rules are either the highest pitch priority rule or the lowest pitch priority rule.

[0085] Since the four virtual string zones are arranged along the Y-axis and each occupies a fixed Y-axis coordinate interval, the corresponding virtual string can be determined by judging which interval the Y-axis coordinate of the touch point falls into. When the touch point is exactly on the boundary line between two virtual string zones, it can be allocated using a preset boundary processing strategy. For example, the touch point on the boundary can be assigned to the virtual string zone with the smaller coordinate value, or it can be allocated probabilistically based on the distribution trend of adjacent touch points. Next, the corresponding pitch segment is matched according to the X-axis coordinate value of the touch center. Similarly, multiple pitch segments are arranged sequentially along the X-axis, and each pitch segment corresponds to a continuous X-axis coordinate interval. The corresponding semitone pitch interval can be determined by judging which interval the X-axis coordinate of the touch point falls into. Then, the matched virtual string zones and pitch segments are mapped to the corresponding standard pitches. This mapping relationship is pre-stored in the memory of the main control terminal in the form of a lookup table or calculation formula. For a traditional violin in standard tuning, the open G string is 196Hz, the D string is 293.7Hz, the A string is 440Hz, and the E string is 659.3Hz. The pitch of each string increases upwards according to the twelve-tone equal temperament system at different positions. Therefore, once the virtual string range and pitch segment are determined, the corresponding frequency values ​​or MIDI note numbers can be obtained by looking up a table or by calculation.

[0086] When multi-finger touch is detected, i.e., when the touch sensor array 121 outputs multiple touch point signals simultaneously, the main control terminal locks a valid pitch from the multiple touch points according to a preset rule. This preset rule can be either a highest pitch priority rule or a lowest pitch priority rule. The highest pitch priority rule is suitable for melodic performance scenarios, assuming the performer's primary intention is to play a higher note, while lower touch points are considered auxiliary or reserved fingers. The lowest pitch priority rule is suitable for harmonic or double-note performance scenarios and can be switched as needed in specific teaching modes. Through these multi-finger touch processing rules, the system can extract the performer's melodic intention and avoid pitch ambiguity caused by simultaneous multi-finger contact.

[0087] The above preprocessing process also includes steps to identify slides and shifting actions.

[0088] Glissando and shifting are highly expressive techniques in violin playing, and also key and challenging aspects of teaching. Accurate identification of these techniques helps the teaching system assess the continuity of the performance and the accuracy of the fingering.

[0089] The identification steps specifically include: S31. The main control terminal continuously tracks the longitudinal displacement Δx and lateral displacement Δy of the touch point.

[0090] S32. When the longitudinal displacement Δx and the lateral displacement Δy satisfy |Δx|≥(2.5×|Δy|), the absolute value of the longitudinal displacement Δx is greater than 50% of the semitone interval, and the continuous movement time of the touch point reaches 80ms, it is determined to be a shift or slide action.

[0091] Specifically, the longitudinal displacement Δx is the change in position of the touch point along the X-axis, i.e., the length of the neck 12, and the lateral displacement Δy is the change in position of the touch point along the Y-axis, i.e., the width of the neck 12. When the absolute values ​​of the longitudinal displacement Δx and the lateral displacement Δy satisfy the inequality |Δx|≥(2.5×|Δy|), and the absolute value of the longitudinal displacement Δx is greater than 50% of the semitone interval, and the touch point continues to move for 80ms, the main control terminal determines that the current action is a shift or a glissando.

[0092] Furthermore, the first main control unit is also configured to perform vibrato recognition processing on the raw touch signal. Vibrato is an important technique in violin playing, where the left-hand fingers roll or swing back and forth on the strings with minute amplitudes to modify the tone. Its recognition requires high temporal and spatial resolution. Specifically, the vibrato recognition processing is as follows: when the main control unit detects that the touch point moves periodically back and forth along the Y-axis or X-axis within the same pitch segment, and this back and forth movement does not cross the boundary of adjacent pitch segments, and the frequency of this back and forth movement is between 4Hz and 8Hz, and the pitch fluctuation caused by this movement is between 0.15 and 0.8 semitones, the main control terminal determines that the current action is a vibrato technique. The frequency range of 4Hz to 8Hz corresponds to the rate of typical vibrato action; too slow and it is considered slow vibrato or tremolo, too fast and it exceeds the range that the human hand can achieve. The pitch fluctuation range is controlled between 0.15 and 0.8 semitones, which eliminates noise signals caused by unconscious micro-tremors of the fingers and ensures that the vibrato effect has a sufficiently discernible depth of pitch change. The vibrato recognition result can be used as one of the indicators for evaluating performance expressiveness, and can also be used to control the modulation parameters in the audio rendering unit, so that the synthesized timbre automatically adds corresponding pitch and low-frequency oscillation effects during the vibrato action, further enhancing the realism of the performance.

[0093] During the preprocessing of the first main control unit, accidental touch filtering is also performed. Accidental touch filtering targets invalid touch events that may occur during performance, such as a slight touch of the side of a finger on the neck 12, foreign objects on the surface of the neck 12, or electromagnetic noise interference.

[0094] The specific logic for mis-touch filtering is as follows: When the touch contact area instantly increases by more than 60% of its original area, the first main control unit marks the touch point as invalid data and excludes it from the performance logic calculation. An abnormally large instantaneous increase in contact area usually corresponds to a finger making large-area contact with the sensing area using a non-fingertip portion of the finger. Such contact does not conform to the proper fingering hand shape and should not trigger a valid pitch. On the other hand, when the touch pressure value is lower than a preset minimum effective pressure threshold, for example, below 0.05N, the first main control unit marks the touch point as invalid data. Extremely low pressure touches usually originate from proximity sensing or very slight touches near the sensing area and do not have a clear intention to press the string. Through the above mis-touch filtering process, noise and mis-pronounced sounds caused by unexpected touches are effectively avoided, improving the cleanliness and accuracy of the performance pitch sequence, while reducing the computational burden of subsequent data fusion and bowing technique recognition steps.

[0095] Furthermore, the standardized performance event data packet can be divided into two outputs after generation.

[0096] The first output is fed locally into a digital signal processor for audio rendering, synthesizing violin timbres in real time and playing them through the audio output module. This output provides the performer with immediate auditory feedback, ensuring the real-time nature and interactivity of the performance.

[0097] The second output is uploaded to the teaching terminal via Bluetooth, Wi-Fi, or MIDI wireless protocols for ensemble synchronization and AI-powered teaching analysis. On the teaching terminal, standardized performance event data packets are parsed into teaching indicators such as visualized score trajectories, bowing accuracy scores, pitch deviation analysis, and rhythmic stability statistics, presented to teachers and students in a graphical interface. Simultaneously, in ensemble settings, the teaching terminal acts as a time reference source, broadcasting a global clock synchronization signal to all connected smart violins. Each smart violin then calibrates its local clock based on this synchronization signal.

[0098] In this embodiment, the smart violin also has an introductory teaching mode.

[0099] When the introductory teaching mode is activated, the main control terminal only opens a preset number of string areas from the four virtual string areas, and / or only opens a preset range of pitch segments from N position segments, to limit the available range and the area to be pressed. For example, in the most basic introductory stage, only the two virtual string areas corresponding to the A and E strings are open, and only the first five pitch segments in the first position are open, that is, some natural notes on the A and E strings. By actively reducing the available range and the area to be pressed, beginners do not have to face the pressure of choosing all possible pitches in the entire range, and can focus their attention on practicing a few basic pitches and simple melodic fragments, effectively avoiding learning anxiety caused by information overload. As the learner's skill level improves, the teaching system can be remotely controlled by an external terminal to gradually increase the number of open string areas and pitch segments until the entire range is fully opened, realizing a gradual, step-by-step teaching process. The activation and parameter configuration of this introductory teaching mode can be set through physical buttons on the main control terminal or through a wirelessly connected mobile application, which is simple and convenient to operate.

[0100] The second main control unit also performs a child-adaptive calibration function.

[0101] When the user is a child, the bow pressure threshold and / or bow speed threshold in the effective onset judgment threshold are automatically lowered by 10% to 20%. Children, due to their smaller hand strength and shorter arm length, often produce bow speed and pressure lower than the average adult level. Using a uniform onset threshold could easily lead to child players being unable to trigger sound with normal force, forcing them to consciously increase their force and thus developing unnatural playing habits. By lowering the onset threshold, children can trigger notes normally at their natural force level, maintaining a relaxed and natural playing posture.

[0102] When the user is an advanced user, the second main control unit raises the effective onset threshold to the 95th percentile of the calibration sample to meet the advanced user's higher requirements for dynamic range and touch sensitivity. The user type can be automatically determined by the amplitude distribution characteristics of the standard action samples collected in the calibration process, or it can be manually set by externally inputting the user's age or level information.

[0103] In this embodiment, the intelligent bow 20 distinguishes five types of professional bowing techniques based on bow direction, velocity, acceleration abrupt change, and pressure duration, as detailed below: Legato: The bow pressure remains constant and the pitch interval is less than 250ms, with no release gap of more than 80ms in between; Bow change: The bow direction is reversed at an angle greater than 120°, and the sound is triggered again after a 60ms~120ms dead zone buffer. Staccato / Accent: Bow speed decreases by 40% within 100ms, the rate of change of acceleration is greater than 15m / s³, and bow pressure remains within the effective range; Jumping bow: The peak vertical acceleration is greater than 1.5g, and the bow pressure briefly drops for 30ms to 120ms before recovering; Tremolo: The bow speed / direction reciprocates in a small, periodic manner at 6Hz~12Hz, with the amplitude of movement less than 15% of the full bow stroke.

[0104] Traditional violin playing involves the right hand holding the bow, employing two independent force mechanisms: a stable grip and downward string pressure. The index, middle, ring, and little fingers encircle and grip the bow stick 22, providing lateral clamping force to prevent the bow from slipping out of the hand—this is the basic grip. The first joint of the right index finger presses forward and downward against the bow stick 22, providing downward pressure perpendicular to the strings—this is the core force point for controlling bow pressure. The middle and ring fingers only provide auxiliary stability and bear almost no downward pressure load. This human force application habit is the basis for the zoned arrangement of the pressure sensors in the intelligent bow 20. It eliminates the need for the entire hand to press down simultaneously; only the index finger applies downward pressure, while the other fingers maintain a wrapping grip.

[0105] The first pressure sensor is used to identify the horizontal clamping force. Its function is to determine whether the user is holding the bow normally. It does not participate in the calculation of the virtual bow pressure P, but only serves as a judgment of the effective bow holding state.

[0106] The middle and ring fingers continuously and gently press against both sides of the bow rod 22, horizontally clamping the bow rod 22 to maintain a firm grip and prevent slippage; only the index finger applies force forward and downward to press against the second pressure sensor on the inner side of the bow rod 22, generating a vertical downward pressure signal.

[0107] When fluctuations in grip force introduce minor pressure noise, the six-axis inertial attitude sensor performs secondary differentiation based on the tilt posture of the bow 22, further eliminating interference. Specifically: When the bow stick 22 is parallel to the playing plane of the virtual neck 12, simulating the real bowing posture, the downward pressure of the index finger is decomposed into an effective component force perpendicular to the neck 12, which is included in the virtual bow pressure P. When the bow is raised vertically, laid flat on its side, and held solely by the hand, the finger pressure is a horizontal clamping force, and the vertical component of the force approaches zero. The algorithm directly sets the bow pressure to zero, and no sound is triggered. When the bow is held still and there is no lateral movement of the bow, even if the index finger applies slight force, the bow speed Vs is 0, and it will not be considered a valid start to the note, thus blocking false triggering from the underlying logic.

[0108] The working mechanism of the present invention will be fully explained below in conjunction with the actual human operation process.

[0109] In the basic gripping action, the user's middle and ring fingers support the bow stick 22 from below, while the little finger lightly touches the tail or side edge of the bow stick 22. The thumb and index finger work together to clamp and fix the bow stick 22 from both sides, maintaining a close and enveloping state between the fingers and the surface of the bow stick 22 throughout the process. During this stage, the first pressure sensor detects a stable horizontal clamping force signal, and the main control terminal determines that the smart bow 20 is in an effective gripping state. However, since there is no lateral bowing displacement (i.e., the bow speed Vs is zero) and the second pressure sensor does not receive an active downward pressure signal from the index finger, the system remains silent and does not produce any sound.

[0110] When the user performs a simulated string-pressing action, the index finger bends, and the first pad of the index finger presses forward against the sensing protrusion on the inside of the bow stick 22. The back of this protrusion faces the sensitive surface of the second pressure sensor. The index finger applies downward force alone, while the other fingers maintain their original clamping position without needing to exert additional force. The second pressure sensor captures the amplitude change of this downward pressure in real time, and the six-axis inertial attitude sensor simultaneously confirms that the bow stick 22 is in a horizontal playing posture. However, since the bow speed Vs is still zero at this time, the system has not yet triggered sound production.

[0111] While maintaining pressure with the index finger, the user moves the bow stick 22 horizontally left and right by using the upper arm to drive the forearm. The six-axis inertial attitude sensor detects effective changes in lateral acceleration and angular velocity, and the bow motion detection unit calculates the real-time bow speed V based on this. s It was confirmed that the effective onset speed threshold was exceeded. At this point, the system simultaneously detected the effective bow speed, the effective vertical downward force, and the effective grip state. Once all three conditions were met, the main control terminal immediately determined the onset of sound, and the standardized performance event data packet was generated and sent to the audio rendering unit within milliseconds.

[0112] In soft or low-pitched scenarios, the user consciously relaxes their index finger, reducing the pressure applied. Consequently, the voltage or resistance value output by the second pressure sensor decreases, and the calculated value of the virtual bow pressure P decreases accordingly. Ultimately, a smaller velocity value is output through the MIDI velocity mapping algorithm, and the audio rendering unit plays a low-volume sample accordingly.

[0113] In accented or staccato scenarios, when the user suddenly increases the pressure on their index finger at the moment of starting the bow stroke, the amplitude and rate of change of the output signal of the second pressure sensor increase sharply at the same time. After recognizing this sudden pressure change, the bow motion detection unit combines the bow speed and acceleration rate of change data to automatically mark the corresponding bowing technique as staccato or accented, and writes the mark into the bowing technique type field in the standardized performance event data packet.

[0114] The entire chain described above, from finger force application to sensor response, and then to signal processing and sound determination, is completed within the signal acquisition and processing framework of the smart bow 20 of this invention with millisecond-level time resolution, realizing a digital mapping of the biomechanical characteristics of holding and moving the bow on a traditional violin.

[0115] Furthermore, this embodiment also provides a multi-instrument synchronous teaching system, including the aforementioned intelligent violin, ensemble master control terminal, synchronization module, and teaching error correction module. The ensemble master control terminal is used to assign voice parts to at least one of the following instruments: intelligent violin, intelligent guitar, intelligent electronic wind instrument, intelligent electronic drum, intelligent keyboard, or bass. The synchronization module is used to enable multiple instruments to play or record performance events according to a unified timeline. The teaching error correction module is used to generate single-track scores, full-track scores, or ensemble synchronous scores based on standard scores and real-time performance events.

[0116] In this embodiment, the ensemble control terminal displays a dynamic score including lyrics, electronic wind instruments, electronic drums, keyboard, smart guitar, bass, and a stringless, motion-sensing smart violin, featuring at least two vocal parts. It supports pause, dragging playback points, variable speed playback, independent track selection, single-part practice, full-part ensemble playing, and loading of course tasks. The teaching error correction module calculates practice scores based on at least two of the following: pitch deviation, rhythm deviation, onset deviation, finish deviation, dynamics deviation, bowing technique deviation, and ensemble timing deviation. The scoring results are then sent to the teacher's, student's, or institution's management terminal.

[0117] In summary, the intelligent violin proposed in this invention includes a violin body 10, an intelligent bow 20, and a main control terminal. The violin body 10 has a body 11 and a neck 12 connected to the body 11. A touch sensor array 121 is provided on the surface of the neck 12. The touch sensor array 121 is used to respond to the left hand's string pressing action and output the original touch signal. The intelligent bow 20 is used to collect and output the bowing posture data of the right hand. The main control terminal is installed on the violin body 11. The touch sensor array 121 and the intelligent bow 20 are both communicatively connected to the main control terminal. The main control terminal is used to receive the original touch signal and the bowing posture data and generate a standardized performance event data packet. This design, through the touch sensor array 121 responding to the left hand's string-pressing action, ensures that pitch determination is independent of the pressing depth and force, as the touch signal only recognizes electrical changes at the finger contact position, reducing the risk of fingertip strain. Simultaneously, since the mapping relationship between position and pitch is fixed by the circuit, it is no longer affected by string tension, wear, or fluctuations in pressing force. This allows learners to obtain stable pitch simply by focusing on finger position accuracy, lowering the threshold for intonation control inherent in fretless instruments. Furthermore, the intelligent bow 20 independently collects and outputs bowing posture data, and the main control terminal can analyze the original touch signal and bowing posture data in real time, enabling performers to make targeted adjustments based on quantitative data rather than relying on instantaneous feel, thus reducing the learning difficulty.

[0118] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A smart violin, characterized in that, include: The instrument has a body and a neck connected to the body. The surface of the neck is provided with a touch sensor array, which is used to respond to the left hand's string pressing action and output the original touch signal. A smart bow is used to collect and output data on the right hand's bowing posture. The main control terminal is installed on the instrument body. The touch sensor array and the smart bow are both communicatively connected to the main control terminal. The main control terminal is used to receive raw touch signals and bowing posture data, and generate standardized performance event data packets.

2. The intelligent violin according to claim 1, characterized in that, The main control terminal includes: A touch acquisition chip is electrically connected to the touch sensing array and is used to acquire the original touch signals of the touch sensing array; The first main control unit is electrically connected to the touch acquisition chip and is used to receive the original touch signal and perform preprocessing. An audio output module, electrically connected to the first main control unit, is used to receive audio signals and output audible sound waves. The first wireless communication module is electrically connected to the first main control unit; The first power module, the touch acquisition chip, the first main control unit, the audio output module and the first wireless communication module are all electrically connected to the first power module.

3. The intelligent violin according to claim 2, characterized in that, The touch-sensing array is arranged along the surface of the neck of the instrument, and the touch-sensing array includes: Four virtual string zones are arranged along the Y-axis, with the four virtual string zones spaced apart, and the four virtual string zones are respectively mapped to the positions of the G string, D string, A string, and E string; Multiple pitch segments are arranged along the X-axis, each pitch segment is mapped to a fixed semitone pitch range, and four virtual string areas and multiple pitch segments constitute a matrix sensing unit, wherein the Y-axis direction is perpendicular to the X-axis direction.

4. The intelligent violin according to claim 3, characterized in that, The touch sensing array is a capacitive touch sensor, a pressure film sensor, or an optical touch sensor. Each of the touch sensing arrays is configured to simultaneously acquire three types of raw touch signals, including touch center coordinates, touch contact area, and touch pressure value. Wherein, the touch center coordinates C(t) = (x(t), y(t)), x(t) is the coordinate value along the X-axis direction, and y(t) is the coordinate value along the Y-axis direction.

5. The intelligent violin according to claim 4, characterized in that, The audio output module includes: The built-in amplifier speaker is embedded in the body of the instrument for external audio output; A headphone jack is embedded in the body of the instrument and is electrically connected to the first main control unit for connecting headphones. A line output interface is embedded in the body of the instrument and is electrically connected to the first main control unit for connecting external audio equipment.

6. The intelligent violin according to claim 1, characterized in that, The smart bow includes: The grip is equipped with a six-axis inertial attitude sensor and a first pressure sensor. The six-axis inertial attitude sensor is used to collect raw data of three-axis acceleration and raw data of three-axis angular velocity. The first pressure sensor is used to collect the pressing stress applied by the right hand fingers to output multi-point pressing signals. The bow is equipped with a second pressure sensor, which is used to collect the deformation stress value of the bow. The bow motion detection unit is electrically connected to the six-axis inertial attitude sensor, the first pressure sensor and the second pressure sensor, receives raw data of acceleration, angular velocity and pressure, and outputs bow direction, bow speed, bow pressure, bow change mark and acceleration change rate. The second wireless communication module is electrically connected to the bow motion detection unit and is used to upload motion data in real time. The second power module is electrically connected to the six-axis inertial attitude sensor, the first pressure sensor, the second pressure sensor, the bow motion detection unit, and the second wireless communication module.

7. The intelligent violin according to claim 6, characterized in that, The main control terminal also includes: The second main control unit is used to fuse and process the original touch signal and bow posture data; The audio rendering unit is electrically connected to the second main control unit and is used to generate digital audio signals of violin timbre or MIDI control commands based on the performance event. The third wireless communication module is electrically connected to the second main control unit; The third power supply module is used to supply power to the second main control unit, the audio rendering unit, and the third wireless communication module.

8. A method for controlling the playing of an intelligent violin as described in any one of claims 1 to 7, characterized in that, Includes the following steps: Real-time acquisition of raw multi-point touch signals from the left hand on the touch sensor array; The smart bow collects real-time data on the right hand's bowing posture, which includes at least bow speed, bow pressure, and direction of movement. The main control terminal receives the original touch signal and bow posture data, and aligns the original touch signal and bow posture data along the time axis according to the acquisition time. The original touch signal, aligned with the timeline, is encapsulated with the bowing posture data to generate a standardized performance event data packet carrying a global timing identifier.

9. The method for controlling the playing of an intelligent violin according to claim 8, characterized in that, It also includes a preprocessing step of pitch resolution of the raw touch signal, specifically including the following steps: Match the corresponding virtual string area based on the Y-axis coordinate value in the touch center coordinate system; Match the corresponding phoneme segment based on the X-axis coordinate value in the touch center coordinate system; Map the matched virtual string regions and pitch segments to the corresponding standard pitches; When multi-finger touch is detected, a valid pitch is locked from multiple touch points according to a preset rule, which is either the highest pitch priority rule or the lowest pitch priority rule.

10. A multi-instrument synchronous teaching system, characterized in that, include: The smart violin according to any one of claims 1 to 7; The ensemble master control terminal is used to assign vocal parts to at least one of the following instruments: smart violin, smart guitar, smart electric wind instrument, smart electronic drum, smart keyboard, or bass. The synchronization module is used to enable multiple instruments to play or record performance events according to a unified timeline; The teaching error correction module is used to generate single-track scores, full-track scores, or ensemble synchronous scores based on standard scores and real-time performance events.

Images