Erhu, Erhu teaching system, teaching method, equipment and computer readable medium

Abstract

The invention discloses an Erhu, an Erhu teaching system, an Erhu teaching method, Erhu teaching equipment and a computer readable medium, and relates to the technical field of musical instruments. A specific embodiment of the method comprises the steps of collecting and generating a student multi-dimensional associated data packet through a sensor group and a control assembly of a student-end Erhu; the server side executes space-time alignment operation on the teacher multi-dimensional associated data packet and the student multi-dimensional associated data packet so as to eliminate time delay and space reference difference between the teacher multi-dimensional associated data packet and the student multi-dimensional associated data packet; the server carries out comparative analysis based on the teacher multi-dimensional associated data packet and the student multi-dimensional associated data packet after the space-time alignment operation to obtain comparative data; and the server side constructs and sends guidance information according to the comparison data, and the student side displays the comparison data and the guidance information. According to the embodiment, subtle changes in the playing process of the urheen can be captured, so that interaction in the playing process of the urheen is improved.

Description

Technical Field

[0001] This invention relates to the field of musical instrument technology, and in particular to an erhu, an erhu teaching system, a teaching method, equipment, and a computer-readable medium. Background Technology

[0002] The erhu is a traditional Chinese bowed string instrument with two strings. Its beautiful tone allows it to express profound emotions. Erhu teaching typically involves the teacher demonstrating a performance, and the student learns by observing the teacher's movements during the performance.

[0003] In the process of realizing this invention, the inventors discovered that the prior art has at least the following problems: In the process of erhu online teaching, because students cannot observe the teacher's performance from multiple angles, even if the teacher's performance process is captured from multiple angles by adding cameras, it is difficult to fully capture the subtle changes in the erhu performance process. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide an erhu, an erhu teaching system, a teaching method, an equipment, and a computer-readable medium, which can capture subtle changes in the erhu performance process to improve the interactivity of the erhu performance process.

[0005] To achieve the above objectives, according to one aspect of the present invention, an erhu is provided, comprising: a soundbox, a neck, strings, a bow, a sensor group, and a control component; The sensor group includes: A first sensor is installed on the bow stick of the bow. The first sensor includes a bow pressure sensor array and an inertial measurement sensor. The bow pressure sensor array is used to collect bow pressure data, and the inertial measurement sensor is used to collect bow stick movement trajectory data. The second sensor is mounted on the string, and the second sensor includes a flexible strain sensor and a vibration sensor. The flexible strain sensor is used to collect the string pressing data corresponding to the string, and the vibration sensor is used to collect the acoustic characteristic data of the string. The control component is connected to the sensor group and is configured to extract and combine data from different sensors to generate a multidimensional associated data packet, the multidimensional associated data packet including bow pressure data, bow movement trajectory data, string pressing data and acoustic feature data associated according to timestamps.

[0006] The piezoelectric thin film array of the bow pressure sensor array is disposed on the inner side of the bow stick corresponding to the contact area between the bow and the string; The inertial measurement sensor is positioned at the tail of the bow; The flexible strain sensor is attached to the finger position area of ​​the string; The control component is configured to: establish an erhu body coordinate system based on the bow movement trajectory data, with the soundbox as the reference, and map the bow pressure data, the string pressing data, and the acoustic feature data to the same time point of the erhu body coordinate system to generate the multidimensional associated data packet.

[0007] The control component is configured to: extract the string bending frequency based on the periodic changes in the pressure signal from the flexible strain sensor, and calculate the string bending trajectory based on the bow movement trajectory data; The sampling frequency of the sensors in the sensor group is adjusted according to the vibrato frequency and the vibrato trajectory.

[0008] According to a second aspect of the present invention, an erhu teaching system is provided, applied to a terminal, the system including the erhu and a display component described above; The control component on the student's end uploads the student's multidimensional association data packet; The display component receives comparison data of the teacher's multidimensional association data packet and the student's multidimensional association data packet, and displays the comparison data and the guidance information of the comparison data overlaid or side by side to show the difference between the teacher's end and the student's end playing the music. The comparison data is obtained by comparing the teacher's multidimensional association data packet and the student's multidimensional association data packet after spatiotemporal alignment.

[0009] The display component displays the student multidimensional association data packet superimposed on the teacher multidimensional association data packet, so as to simultaneously compare the data in the teacher multidimensional association data packet and the student multidimensional association data packet; or, The display component displays the corresponding data in the teacher's multidimensional association data package and the student's multidimensional association data package side by side, and displays the difference data in a preset color.

[0010] The system also includes an operating component. The operation component responds to the operation command and retrieves the teacher multidimensional association data packet from the database based on the music identifier in the operation command. The teacher multidimensional association data packet is a data set obtained by the teacher using the erhu to play the music corresponding to the music identifier. or, The operation component responds to the operation command and sends the music identifier in the operation command to the teacher's terminal; The display component receives multidimensional association data packets from the teacher's end.

[0011] The display component is also used to display the sheet music corresponding to the student's multidimensional association data package, and the musical phrase in the sheet music corresponding to the largest difference data in the comparison data; The system also includes an audio component, which is used to output the audio of the musical score while displaying the musical score corresponding to the student's multidimensional association data packet.

[0012] According to a third aspect of the present invention, an erhu teaching method is provided, the method using the erhu teaching system as described above, comprising: The sensor group and control components of the erhu at the student end are used to collect and generate multi-dimensional related data packets for the student. The server performs a spatiotemporal alignment operation on the teacher multidimensional association data packet and the student multidimensional association data packet to eliminate the time delay and spatial reference difference between the teacher multidimensional association data packet and the student multidimensional association data packet; The server-side performs comparative analysis on the teacher multidimensional association data packet and the student multidimensional association data packet after spatiotemporal alignment to obtain the comparison data. The server constructs and sends guidance information based on the comparison data, and the student displays the comparison data and the guidance information.

[0013] According to a fourth aspect of the present invention, an electronic device for teaching the erhu is provided, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the methods described above.

[0014] According to a fifth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described above.

[0015] One embodiment of the above invention has the following advantages or beneficial effects: the control component on the student's end uploads the student's multidimensional association data package; the display component receives comparison data of the teacher's multidimensional association data package and the student's multidimensional association data package, and displays the comparison data and the guidance information of the comparison data overlaid or side by side to show the differences between the teacher's end and the student's end playing the music. The comparison data is obtained by comparing the teacher's multidimensional association data package and the student's multidimensional association data package after spatiotemporal alignment. The multidimensional association data package represents the multidimensional changes in the erhu performance process, and the interaction in the erhu performance process is improved by comparing the teacher's multidimensional association data package and the student's multidimensional association data package.

[0016] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0017] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein: Figure 1 This is a schematic diagram of the structure of the erhu according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an erhu teaching system according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the erhu teaching method according to an embodiment of the present invention; Figure 4 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied; Figure 5 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0018] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0019] Traditional erhu teaching often uses a single audio and video transmission mode, lacking data acquisition capabilities and unable to quantify and capture subtle changes during erhu performance. The unique vibrato and bowing techniques of erhu playing are difficult to demonstrate in remote teaching.

[0020] To capture subtle changes during erhu performance, the following technical solutions from the embodiments of the present invention can be adopted.

[0021] See Figure 1 , Figure 1 This is a schematic diagram of the structure of an erhu according to an embodiment of the present invention. The erhu includes a neck 10, strings 20, a soundbox 30, a bow 40, a sensor group, and a control assembly.

[0022] The sensor group includes various types of sensors. The sensor group includes a first sensor and a second sensor.

[0023] The first sensor is mounted on the bow 40 and is used to acquire parameters used by the player to control the bow 40. Specifically, the first sensor includes a bow pressure sensor array and an inertial measurement sensor. The bow pressure sensor array is used to collect bow pressure data. This data allows the player to determine the force applied to the bow 40. The inertial measurement sensor is used to collect data on the bow's movement trajectory. This allows the player to identify the trajectory of their bow movements, such as pulling, pushing, staccato, spiccato, and slingshot.

[0024] The inertial measurement sensor is based on the erhu neck coordinate system, with the origin set at the top of the soundbox, and establishes a three-dimensional coordinate system for bow movement: X: forward and backward, Y: left and right, Z: up and down, to achieve trajectory reconstruction at the 0.1mm level.

[0025] The second sensor is mounted on the string 20 and is used to acquire parameters used by the player to control the string 20. Specifically, the second sensor includes a flexible strain sensor and a vibration sensor. The flexible strain sensor is used to collect string pressing data corresponding to the string 20. For example, string pressing data includes the string pressing pressure value, pressure contact area, and string pressing position. The vibration sensor is used to collect acoustic characteristic data of the string 20. For example, acoustic characteristic data includes pitch and frequency. The vibration sensor includes an electrical thin-film sensor. The electrical thin-film sensor is located at the erhu bridge to collect the acoustic characteristic data of the string.

[0026] The control component connects to the sensor array. For example, the control component connects to the sensor array via Bluetooth. The control component is configured to extract and combine data from different sensors to generate a multidimensional correlated data packet, which includes bow pressure data, bow movement trajectory data, string pressing data, and acoustic feature data correlated according to timestamps.

[0027] The sensor array transmits the collected data to the control component via a connection, and the control component constructs a multidimensional correlated data packet. This multidimensional correlated data packet includes data collected by multiple types of sensors. The control component correlates bow pressure data, bow movement trajectory data, string pressing data, and acoustic characteristic data according to timestamps, thus enabling it to characterize the data changes of the performer during erhu playing through the multidimensional correlated data packet.

[0028] In one embodiment of the invention, the piezoelectric thin film array of the bow pressure sensor array is disposed on the inner side of the bow stick corresponding to the contact area between the bow and the string. For example, the piezoelectric thin film array is an 8×8 array. The resolution of the piezoelectric thin film array is 0.05N.

[0029] An inertial measurement sensor is used to position the bow's tail. For example, an inertial measurement sensor with a sampling rate of 200Hz and an attitude accuracy of ±0.1° can be used to acquire the bow's three-dimensional trajectory and angular velocity.

[0030] A flexible strain sensor is attached to the fingering area of ​​a musical string. For example, a flexible strain gauge is embedded in the string, and the flexible strain gauge contains a flexible strain sensor.

[0031] The control component is configured to: establish a coordinate system of the erhu body based on the bow movement trajectory data, and map the bow pressure data, string pressing data and acoustic feature data to the same point in time in the erhu body coordinate system to generate a multi-dimensional associated data package.

[0032] The data in the multidimensional associated data package has a time correlation relationship, that is, for the same timestamp, there are corresponding bow pressure data, bow stick movement trajectory data, string pressing data and acoustic feature data.

[0033] In an embodiment of the invention, vibrato is a technique in erhu playing where the fingers make regular, controlled rubbing motions while accurately pressing the notes, thereby producing regular fluctuations in the sound. Vibrato parameters can be collected using a flexible strain sensor.

[0034] Specifically, the control component is configured to: extract the string bending frequency based on the periodic changes in the pressure signal from the flexible strain sensor, and calculate the string bending trajectory based on the bow movement trajectory data; adjust the sampling frequency of the sensors in the sensor group based on the string bending frequency and the string bending trajectory.

[0035] For example, if the vibrato frequency is greater than or equal to the frequency threshold, and the displacement of the vibrato trajectory is greater than or equal to the displacement threshold, it indicates that data corresponding to the vibrato needs to be collected, and the sampling frequency of the sensors in the sensor group should be increased. If the vibrato frequency is less than the frequency threshold, and the displacement of the vibrato trajectory is less than the displacement threshold, it indicates that the performance did not involve vibrato, and the default sampling frequency of the sensors in the sensor group should be restored.

[0036] For example, a vibrato feature vector can be generated by acquiring the vibrato frequency and trajectory during performance using flexible strain sensors and inertial measurement sensors. This feature vector is then used to characterize the vibrato parameters, allowing adjustment of the sampling frequency of the sensors in the sensor array.

[0037] To improve data accuracy, the control components are configured to use wavelet transform to process the sensor group's data for noise reduction.

[0038] In the above embodiments, a sensor group and control components are used to quantitatively capture parameters during the erhu playing process, and the sampling frequency of the sensors in the sensor group can be adjusted for vibrato in the erhu to ensure the integrity and real-time performance of the vibrato data.

[0039] See Figure 2 , Figure 2 This is a schematic diagram of an erhu teaching system according to an embodiment of the present invention. Figure 2 Secondary school students passed Figure 1The system uses an erhu (a two-stringed bowed instrument) to play music. The erhu's sensor array and control components collect and generate multi-dimensional correlation data packets for the learners, and then send these packets to the server. For example, the system sends the learner's multi-dimensional correlation data packets to the server. The display component receives and compares the teacher's and learner's multi-dimensional correlation data packets. For example, the display component receives and compares the teacher's and learner's multi-dimensional correlation data packets sent by the server.

[0040] The display component overlays or displays comparison data and guidance information on the comparison data to show the differences between the teacher's and student's performance of the music. The comparison data is obtained by comparing the spatiotemporally aligned teacher multidimensional association data packets and student multidimensional association data packets.

[0041] As an example, for music piece A, the display component receives comparative data from the teacher's multidimensional association data package and the student's multidimensional association data package. The comparative data includes: bow pressure difference data, bow stick teacher's movement trajectory data, bow stick student's movement trajectory data, string pressing difference data, and acoustic feature difference data. These difference data are the differences between the data corresponding to the teacher's multidimensional association data package and the data corresponding to the student's multidimensional association data package.

[0042] The guidance provided by the comparative data offers suggestions for learners based on the comparative data. For example, suggestions to increase bow pressure and improve fingering accuracy.

[0043] In one embodiment of the present invention, to display the comparison data, the display component may display the teacher multidimensional association data packet and the student multidimensional association data packet in the following manner: The display component receives the teacher multidimensional association data packet, the student multidimensional association data packet, and comparison data between the teacher multidimensional association data packet and the student multidimensional association data packet. Thus, the teacher multidimensional association data packet and the student multidimensional association data packet are displayed simultaneously with the comparison data.

[0044] Method 1

[0045] The display component shows the student multidimensional association data packet overlaid on top of the teacher's multidimensional association data packet, allowing for simultaneous comparison of data from both packets. For example, the display component shows two layers of data: the lower layer is from the teacher's multidimensional association data packet, and the upper layer is from the student's multidimensional association data packet, overlaying the two layers to achieve a comparison.

[0046] Method 2

[0047] The display component shows the corresponding data in the teacher's multidimensional association data package and the student's multidimensional association data package side by side, and displays the difference data in a preset color.

[0048] The display component shows data from the teacher's multidimensional association data package on the left side of the display interface and data from the student's multidimensional association data package on the right side. The displayed data correspond to each other. Data that differs from the preset color is highlighted. For example, data in the student's multidimensional association data package that differs from data in the teacher's multidimensional association data package is marked in red.

[0049] In one embodiment of the present invention, the erhu teaching system further includes an operation component. The operation component is used to acquire multi-dimensional associated data packets from teachers.

[0050] The teacher multidimensional association data package can be set in the database. For example, a teacher can use the erhu in this embodiment of the invention to construct multiple teacher multidimensional association data packages for different pieces of music. The operation component responds to the operation command and retrieves the teacher multidimensional association data package from the database based on the music identifier in the operation command. The teacher multidimensional association data package is a data set obtained by the teacher using the erhu to play the music corresponding to the music identifier.

[0051] Students and teachers can also play the same piece of music simultaneously, enabling them to share the screen in the display component. Specifically, the operation component responds to operation commands by sending the music identifier from the operation command to the teacher's end; the display component receives the teacher's multidimensional association data packet from the teacher's end.

[0052] In an embodiment of the present invention, in order to promptly remind students of the differences between the music played by students and teachers, the display component can also display musical phrases, while the audio component outputs the audio of the musical score.

[0053] Specifically, the display component is also used to display the sheet music corresponding to the student's multidimensional associated data package, and the musical phrase in the sheet music corresponding to the largest difference data in the comparison data. For example, the musical phrase in the sheet music corresponding to the largest difference data in the comparison data is the third musical phrase.

[0054] The audio component displays the sheet music corresponding to the student's multidimensional associated data package while simultaneously outputting the audio of the sheet music. Specifically, it outputs the audio of the third musical phrase. It can output the audio of the student playing the third musical phrase first, followed by the audio of the teacher playing the third musical phrase. This allows for playback of audio corresponding to the comparison data.

[0055] In the above embodiments, the student's end and / or the teacher's end can be set on a mobile terminal or a computer. When playing the erhu, the interaction between the student and the teacher can be realized through the mobile terminal or computer. On the one hand, the erhu uses the data of the student's and teacher's performance of the music, and the student's performance and the teacher's performance can be compared through the mobile terminal or computer, thereby improving the interaction during the erhu performance.

[0056] See Figure 3 , Figure 3 This is a flowchart illustrating an erhu teaching method according to an embodiment of the present invention. Specifically, it includes the following steps: S301. Collect and generate multi-dimensional data packets related to the student through the sensor group and control components of the erhu at the student end.

[0057] In embodiments of the present invention, the student's end interacts with the server to implement the erhu teaching method. Similarly, the teacher's end can interact with the server, i.e., upload a multi-dimensional association data package to the server. The multi-dimensional association data package can be stored in a database.

[0058] The sensor array and control components of the erhu are used to collect data on the student's performance and generate a multi-dimensional data package related to the student.

[0059] In one embodiment of the present invention, the multidimensional associated data packet includes bow pressure data, bow movement trajectory data, string pressing data, and acoustic feature data associated according to timestamps. The data collected by the different sensors serve different purposes; therefore, corresponding priorities can be set to allocate bandwidth according to priority for uploading the data to the server. For example, bow movement trajectory data and string pressing data are high-priority data; bow pressure data is medium-priority data; and acoustic feature data is low-priority data. This method of transmitting the multidimensional associated data packet improves bandwidth utilization.

[0060] S302. Perform spatiotemporal alignment operations on the server side for the teacher multidimensional association data packet and the student multidimensional association data packet to eliminate the time delay and spatial reference difference between the teacher multidimensional association data packet and the student multidimensional association data packet.

[0061] After receiving the student multidimensional association data, the server performs a spatiotemporal alignment operation on both the teacher and student multidimensional association data packets to compare them with the teacher multidimensional association data packets.

[0062] As an example, using the timestamps within the teacher's and student's multidimensional association data packets, the data in the teacher's and student's multidimensional association data packets are aligned to the same timeline. Then, the spatial data in the teacher's and student's multidimensional association data packets, based on their respective instrument bodies, are transformed into spatial data in a coordinate system based on the standard erhu.

[0063] As another example, a first feature sequence representing the performance rhythm is extracted from the teacher's multidimensional association data packet, and a second feature sequence representing the performance rhythm is extracted from the student's multidimensional association data packet. The optimal matching path between the first and second feature sequences is calculated using a dynamic time warping algorithm. Based on this optimal matching path, a non-linear time mapping relationship from student time to teacher time is generated. Based on this non-linear time mapping relationship, the time axis of the student's multidimensional association data packet is recalibrated to ensure that the time difference between corresponding events in the teacher-student performance is less than 5 milliseconds, thereby eliminating the time delay between the teacher's and student's multidimensional association data packets.

[0064] S303. The server side performs comparative analysis on the teacher multidimensional association data packet and the student multidimensional association data packet after the spatiotemporal alignment operation to obtain comparative data.

[0065] A performance model is established based on the bow pressure data, bow movement trajectory data, string pressing data, and acoustic feature data from the teacher's multidimensional association data package to generate corresponding baseline vectors. For example, the performance model includes vectors for bow pressure data, bow movement trajectory data, string pressing data, and acoustic feature data.

[0066] Using Dynamic Spatiotemporal Convolutional Network (DST-CNN), comparative data was obtained by comparing and analyzing teacher multidimensional association data packets and student multidimensional association data packets from four dimensions: bow pressure, bow stick movement trajectory, string pressing, and acoustic features.

[0067] S304. The server constructs and sends guidance information based on the comparison data, and the student displays the comparison data and guidance information.

[0068] In an embodiment of this invention, the server can construct a dedicated 3D visualization scene for the erhu based on WebGL, recreating the physical models of the neck, strings, and bow, and rendering the bowing trajectories of teachers and students in real time. For example, the teacher's trajectory is blue, the student's trajectory is red, and the differences are highlighted and flashed, with a rendering frame rate of 120fps@4K and a trajectory accuracy of ±0.5mm. WebGL is a low-level JavaScript API that directly renders interactive 2D and 3D graphics in a web browser.

[0069] The Neural Radiation Field (NeRF) rendering technology can also be used to merge the teacher's holographic model with the student's real-time image, supporting two modes: "split-screen comparison" and "same-screen overlay". In the overlay mode, the difference in movement is highlighted through a semi-transparent effect.

[0070] The server constructs and sends guidance information based on the comparison data, while the student displays both the comparison data and the guidance information. As an example, an LRA motor built into the bow is used to highlight deviations in the comparison data; the greater the deviation, the higher the vibration frequency. The guidance information is tailored to the student based on the comparison data, such as increasing bow pressure or improving fingering accuracy.

[0071] In one embodiment of the present invention, teacher multidimensional association data packets and student multidimensional association data packets can be stored via blockchain. This ensures the security, integrity, and traceability of teaching data, providing a basis for teaching evaluation and rights protection.

[0072] A teaching data chain and an interaction log chain are constructed, with each data entry associated with a timestamp and device fingerprint to ensure immutability. The teaching data chain stores multi-dimensional associated data packages for teachers and students. The interaction log chain stores feedback records and data retrieval records.

[0073] To enhance data security, SM4 is used to encrypt and transmit teacher multidimensional association data packets, SM3 is used to generate data hash digests, and SM2 is used for identity authentication to prevent leakage and unauthorized access to teacher multidimensional association data packets.

[0074] Refer to Table 1, which compares the technical solutions adopted in this invention with existing technologies. The time delay error is the time delay error corresponding to the spatiotemporal alignment operation performed on the teacher's multidimensional association data packet and the student's multidimensional association data packet. End-to-end delay refers to the time delay from the student's end to the server's end.

[0075] Table 1

[0076] Figure 4 An exemplary system architecture 400 is shown that can be applied to the erhu teaching method of embodiments of the present invention.

[0077] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, and 403, a network 404, and a server 405. Network 404 serves as the medium for providing communication links between terminal devices 401, 402, and 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0078] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, and 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0079] Terminal devices 401, 402, and 403 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0080] Server 405 can be a server that provides various services, such as a backend management server that supports shopping websites browsed by users using terminal devices 401, 402, and 403 (for example only). The backend management server can analyze and process data such as received product information query requests, and feed back the processing results (such as target push information, product information - for example only) to the terminal device.

[0081] It should be noted that the erhu teaching method provided in this embodiment of the invention is generally executed by server 405.

[0082] It should be understood that Figure 4 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0083] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing a terminal device of the present invention. Figure 5 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0084] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0085] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0086] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.

[0087] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0089] The modules described in the embodiments of this invention can be implemented in software or hardware. The described modules can also be located in a processor. In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: The sensor group and control components of the erhu at the student end are used to collect and generate multi-dimensional related data packets for the student. The server performs a spatiotemporal alignment operation on the teacher multidimensional association data packet and the student multidimensional association data packet to eliminate the time delay and spatial reference difference between the teacher multidimensional association data packet and the student multidimensional association data packet; The server-side performs comparative analysis on the teacher multidimensional association data packet and the student multidimensional association data packet after spatiotemporal alignment to obtain the comparison data. The server constructs and sends guidance information based on the comparison data, and the student displays the comparison data and the guidance information.

[0090] According to the technical solution of this invention, the control component on the student's end uploads a multi-dimensional association data packet for the student; the display component receives comparison data of the teacher's multi-dimensional association data packet and the student's multi-dimensional association data packet, and displays the comparison data and its guidance information overlaid or side-by-side to show the differences between the teacher's and student's performances of the music. The comparison data is obtained by comparing the teacher's and student's multi-dimensional association data packets after spatiotemporal alignment. The multi-dimensional association data packet represents the multi-dimensional changes during the erhu performance process, and the interaction during the erhu performance process is improved by comparing the teacher's and student's multi-dimensional association data packets.

[0091] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention. It should be noted that the acquisition, storage, and application of user personal information involved in the technical solutions of this disclosure comply with relevant laws and regulations and do not violate public order and good morals.

Claims

1. A type of erhu, characterized in that, include: The soundbox, neck, strings, bow, sensor array, and control components; The sensor group includes: A first sensor is installed on the bow stick of the bow. The first sensor includes a bow pressure sensor array and an inertial measurement sensor. The bow pressure sensor array is used to collect bow pressure data, and the inertial measurement sensor is used to collect bow stick movement trajectory data. The second sensor is mounted on the string, and the second sensor includes a flexible strain sensor and a vibration sensor. The flexible strain sensor is used to collect the string pressing data corresponding to the string, and the vibration sensor is used to collect the acoustic characteristic data of the string. The control component is connected to the sensor group and is configured to extract and combine data from different sensors to generate a multidimensional associated data packet, the multidimensional associated data packet including bow pressure data, bow movement trajectory data, string pressing data and acoustic feature data associated according to timestamps.

2. The erhu according to claim 1, characterized in that, The piezoelectric thin film array of the bow pressure sensor array is disposed on the inner side of the bow stick corresponding to the contact area between the bow and the string; The inertial measurement sensor is positioned at the tail of the bow; The flexible strain sensor is attached to the finger position area of ​​the string; The control component is configured to: establish an erhu body coordinate system based on the bow movement trajectory data, with the soundbox as the reference, and map the bow pressure data, the string pressing data, and the acoustic feature data to the same time point of the erhu body coordinate system to generate the multidimensional associated data packet.

3. The erhu according to claim 1, characterized in that, The control component is configured to: extract the string bending frequency based on the periodic changes in the pressure signal from the flexible strain sensor, and calculate the string bending trajectory based on the bow movement trajectory data; The sampling frequency of the sensors in the sensor group is adjusted according to the vibrato frequency and the vibrato trajectory.

4. An erhu teaching system, characterized in that, Applied to a terminal, the system includes the erhu and a display component as described in any one of claims 1 to 3; The control component on the student's end uploads the student's multidimensional association data packet; The display component receives comparison data of the teacher's multidimensional association data packet and the student's multidimensional association data packet, and displays the comparison data and the guidance information of the comparison data overlaid or side by side to show the difference between the teacher's end and the student's end playing the music. The comparison data is obtained by comparing the teacher's multidimensional association data packet and the student's multidimensional association data packet after spatiotemporal alignment.

5. The erhu teaching system according to claim 4, characterized in that, The display component displays the student multidimensional association data packet superimposed on the teacher multidimensional association data packet, so as to simultaneously compare the data in the teacher multidimensional association data packet and the student multidimensional association data packet; or, The display component displays the corresponding data in the teacher's multidimensional association data package and the student's multidimensional association data package side by side, and displays the difference data in a preset color.

6. The erhu teaching system according to claim 4, characterized in that, The system also includes an operating component. The operation component responds to the operation command and retrieves the teacher multidimensional association data packet from the database based on the music identifier in the operation command. The teacher multidimensional association data packet is a data set obtained by the teacher using the erhu to play the music corresponding to the music identifier. or, The operation component responds to the operation command and sends the music identifier in the operation command to the teacher's terminal; The display component receives multidimensional association data packets from the teacher's end.

7. The erhu teaching system according to claim 4, characterized in that, The display component is also used to display the sheet music corresponding to the student's multidimensional association data package, and the musical phrase in the sheet music corresponding to the largest difference data in the comparison data; The system also includes an audio component, which is used to output the audio of the musical score while displaying the musical score corresponding to the student's multidimensional association data packet.

8. A method for teaching the erhu, characterized in that, The method uses the erhu teaching system as described in any one of claims 5-7, including: The sensor group and control components of the erhu at the student end are used to collect and generate multi-dimensional related data packets for the student. The server performs a spatiotemporal alignment operation on the teacher multidimensional association data packet and the student multidimensional association data packet to eliminate the time delay and spatial reference difference between the teacher multidimensional association data packet and the student multidimensional association data packet; The server-side performs comparative analysis on the teacher multidimensional association data packet and the student multidimensional association data packet after spatiotemporal alignment to obtain the comparison data. The server constructs and sends guidance information based on the comparison data, and the student displays the comparison data and the guidance information.

9. An electronic device for teaching the erhu, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in claim 8.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in claim 8.

Images