Intelligent guitar with karaoke singing function

Abstract

The invention relates to the technical field of audio processing, and discloses an intelligent guitar with a karaoke singing function. Comprising the following steps: detecting singing sound of a user in real time, synchronously regulating and controlling audio signal path parameters between a built-in microphone and a loudspeaker, and eliminating delay and synchronization errors caused by transmission; the refresh rate of the display screen is automatically adjusted according to the working frequency of the lyric display module, so that the lyric display time is strictly synchronized with the rolling of the music progress bar to eliminate the display delay; performing multi-stage filtering processing on the acquired audio data sung by the user, and comparing the audio data with a preset standard intonation model to provide high-precision sound acquisition and accurate intonation judgment; and the optimal sound proportion and reverberation effect are ensured to be achieved under various playing conditions. According to the intelligent guitar with the karaoke singing function, real-time and accurate audio playing can be realized, the audio-visual experience of a user in scenes of listening to music, watching videos and the like is improved, and discomfort caused by asynchronism of sound and pictures is avoided.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and in particular to an intelligent guitar with karaoke singing function. Background Art

[0002] A smart guitar with karaoke singing capabilities combines the playing experience of traditional instruments with modern digital audio technology, aiming to provide users with a more interactive and diverse way to create and entertain music. The design of this smart guitar faced several key challenges: first, audio input and output control, requiring sophisticated algorithms and low-latency technology to address potential delays and synchronization errors during audio signal transmission and processing. Second, lyric display and progress bar scrolling control utilize real-time monitoring and prediction technology to ensure that lyrics are accurately synchronized with the music playback process and prevent display delays from affecting the user's singing experience. Third, addressing sound acquisition and pitch determination during singing, a high-precision microphone array and real-time sound analysis algorithms enable real-time pitch monitoring and provide user feedback to improve performance. Fourth, balancing the accompaniment track at different volume levels utilizes adaptive dynamic compression technology and ambient monitoring to ensure the accompaniment maintains an appropriate loudness ratio regardless of the playback environment. Finally, the interaction between string vibration and the built-in pickup sensor is finely tuned to optimize the response characteristics of the internal pickup system, resulting in better reverberation and authentic sound reproduction. In summary, the application of these technologies enables this smart guitar to not only have excellent sound quality and user experience, but also greatly enhance the player's sense of participation and expressiveness. Summary of the Invention

[0003] In order to solve the problems raised by the above background technology, the present application provides a smart guitar with karaoke singing function.

[0004] This application provides a smart guitar with karaoke singing function, which adopts the following technical solutions: An intelligent guitar with karaoke singing function, comprising: S101, detecting the user's singing voice in real time and synchronously adjusting the audio signal path parameters between the built-in microphone and the speaker to eliminate delays and synchronization errors caused by transmission; S102, automatically adjusting the display screen refresh rate according to the operating frequency of the lyrics display module, so that the lyrics display time is strictly synchronized with the scrolling of the music progress bar to eliminate display delay; S103, performing multi-stage filtering on the collected user singing audio data and comparing it with a pre-set standard pitch model to provide high-precision sound collection and accurate pitch judgment; S104: Dynamically adjust the balanced output characteristics of different accompaniment tracks and string pickup sensors based on the above-obtained information to ensure that the optimal sound ratio and reverberation effect are achieved under various playing conditions.

[0005] Preferably, the method of dynamically adjusting the balanced output characteristics of different accompaniment tracks and string pickup sensors based on the information obtained in the above steps is as follows: Analyze the energy distribution of each audio segment to determine the base gain Gbase of each track; The actual gain Gfinal after real-time adjustment is calculated according to the preset formula F(Gin, Vol), where F represents the gain adjustment formula, Gin represents the initial volume, and Vol represents the total volume of the current performance. If Gfinal < Gmin, the minimum gain compensation is set; The accelerometer is used to collect the guitar's motion data Acc, and the preset algorithm P is used to optimize the data of the string pickup sensor to achieve the best reverberation effect; Dynamically adjust the sensitivity of the microphone and speaker based on the ambient noise level Nlv to improve the authenticity of sound reproduction in low-noise environments.

[0006] Preferably, the method further comprises: Perform fast Fourier transform (FFT) on each audio frame and analyze the peak frequency point Pmax within the frequency range; Establish the track equalization response curve Rcurve, and make adaptive corrections based on Pmax to make the sound of each channel more harmonious and balanced; Dynamically control the overall timbre adjustment level AdjLevel according to the formula If(ENV > EnvLimit, AdjLevel++, AdjLevel) where ENV is a measure of external environmental factors and EnvLimit is a threshold set by the system to determine whether to increase or decrease the timbre processing intensity. Adjust the display contrast parameters and lyrics synchronization rate to ensure that lyrics and background music are displayed synchronously without delay.

[0007] Preferably, the further supplementary description is based on the following content: Capture key characteristic parameters Sfeat (such as onset, pitch change rate, etc.) of the user's singing and build a personalized acoustic model Muser; Compare Sfeat and the standard pitch model in multiple dimensions using Vcmp = Sim(Sfeat,Mstd), where Sim is defined as the similarity function between the two features to evaluate the degree of match between them. Calculate the real-time deviation Dreal. If the judgment condition |Dreal| > Tolerance is met (the absolute value of the tolerance error Tolerance is greater than a certain limit), prompt the user to make corrections. The human voice part is extracted as feedback information and provided to the system for later correction reference.

[0008] Preferably, the expansion of the technical steps further includes: Use nonlinear filtering NonLinFilter to suppress howling and prevent sound signal distortion under extreme conditions that affect the sound quality experience; Construct a dynamic sound effect matrix MatEff and accurately adjust the matrix parameters based on the guitar posture data Posture to produce different spatial sound field perceptions and enhance the stereo effect; Realize intelligent automatic switching of EQ Preset settings to ensure optimal reverberation and sound balance ratio in any environment; The current performance (PerfEval) is evaluated using the IF(PerfEval<=PerfThreshold,AdjustMix,ContinuePlaying) logic. If the score is lower than the system's default minimum standard, PerfThreshold, the Mix settings are adjusted; otherwise, the status quo remains unchanged.

[0009] Preferably, the deepening is based on the description: Specific mathematical algorithms such as Hilbert Transform (HilbTrans) are applied to the string vibration signals to accurately measure the vibration amplitude and decay rate (Ampi, SpeedDec) of each string, ensuring the true reproduction of the performance. After collecting RawSignal data from the pickup device, we apply machine learning (ML) technology to classify it (RawSignal) to accurately distinguish different sources (such as finger touch). A new digital audio source conversion protocol, DigiConvert, has been designed to enable more efficient encoding of original analog audio into digital form for further processing. Based on the real-time environment brightness Lbright, the color ColorChose and brightness BrightnessCtrl of the LED light are controlled to create a better stage atmosphere to assist the music performance.

[0010] In summary, this application includes at least one of the following beneficial technical effects: This smart guitar with karaoke singing function can achieve real-time and accurate audio playback, enhance the user's audio-visual experience in scenarios such as listening to music and watching videos, and avoid the discomfort caused by the lack of synchronization between sound and picture.

[0011] This smart guitar with karaoke singing function can help users sing along with songs more accurately, enhance the singing experience and sense of participation, enable users to better grasp the rhythm and melody of the song, and enhance their understanding and expression of the song.

[0012] This smart guitar with karaoke singing function provides users with accurate singing assessment feedback, helping users understand their singing level and existing problems, facilitating targeted practice and improvement, and stimulating users' singing enthusiasm and learning motivation.

[0013] This smart guitar with karaoke singing function allows users to enjoy balanced and harmonious music effects in various volume environments, allowing the accompaniment to complement the singing or other audio elements, improving the overall music quality and meeting users' music playback needs in different scenarios.

[0014] This smart guitar with karaoke singing function allows guitarists to control the reverberation effect more precisely, flexibly adjust the reverberation level according to the repertoire, style and occasion, create richer and more unique musical expression, and enhance the artistic appeal of the performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of an intelligent guitar with karaoke singing function according to the present invention. DETAILED DESCRIPTION

[0016] Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings.

[0017] Throughout this specification, reference to the terms "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0018] Next, refer to the attached Figure 1, describing an intelligent guitar with karaoke singing function of the present invention. The core design of the guitar lies in its unique audio input and output control, lyrics display timing and progress bar scrolling control, user voice collection and pitch judgment control, and fine adjustment of reverberation and volume balance during performance. The specific steps are as follows: first, low-latency audio input and output control is achieved; second, the lyrics display time and music playback are strictly synchronized by automatically refreshing the display screen; third, high-precision audio data multi-stage filtering and pitch comparison are performed to provide accurate singing performance evaluation; then, according to the data provided in the previous steps, the balanced output characteristics between the accompaniment track and the string pickup sensor feedback are dynamically adjusted in real time to ensure coordinated sound effects; finally, the best overall performance is achieved through meticulous debugging and parameter optimization of each link.

[0019] In terms of solving the delay and asynchrony caused by signal transmission, the user's singing is detected in real time and the time difference between the sound data received by the built-in microphone and the sound output by the speaker is synchronously calibrated. This includes the use of digital signal processing technology to identify any subtle time drift or frequency response fluctuation problems and correct them in time to avoid delays. When the guitar is used as a karaoke device, it will use the embedded computing unit to quickly complete all necessary mathematical calculations and analysis work to achieve a seamless experience. For example, in one embodiment, the intelligent software algorithm can automatically sense the slightest interval between each key transition and then respond immediately to ensure that the singing immediately follows the melody.

[0020] As for the update rate of the lyrics screen, it is necessary to set a suitable rate value according to the actual singing progress to match the specific changing rules of each piece of music, so that the singer can see the text prompts that appear at the right time and keep scrolling forward instead of lagging behind the current song progress. This will not affect the singer's accurate vocalization according to the established rhythm. In order to ensure this, the smart instrument is equipped with a dedicated visual engine that can efficiently process text and image information streams and flexibly change the refresh cycle to meet diverse on-site requirements. For example, fast songs can speed up the speed of the subtitles to keep up with the rapid beat, while slow works can slow down the rhythm appropriately to create a soothing atmosphere and bring more levels of change space to the performance. Specifically, the system will predefine a set of reasonable parameter ranges according to the characteristics of different types of works, and select the most appropriate set of settings according to the actual situation during the performance.

[0021] After collecting the live recording of the singer, preliminary noise reduction should be performed on it to eliminate all interference sources that may affect the accuracy of the evaluation, such as noise, current hum, etc. Then a series of complex calculations should be taken to convert it into a more uniform waveform form with alternating transformations, so that it can be compared with the authoritative standard reference samples stored in the cloud or local resource database in advance, and then the fine-tuning range and overall deviation of each pronunciation part can be determined to provide quantitative indicators for the subsequent teaching and guidance scoring mechanism. If a particularly complex multi-language mixed singing scenario is encountered, then this mechanism should also have intelligent perception capabilities to correctly identify and distinguish the unique intonation habits of each language and match the corresponding judgment criteria to improve the accuracy of judgment.

[0022] Based on the previously obtained sound characteristic details and other related information, such as the intensity distribution characteristics of the accompaniment mode selection, we further refined the management of the working status of each participating sound-producing part to ensure that it always maintains the best matching ratio. This is especially true for certain specific environments such as outdoor concerts or small indoor sharing sessions. The sound reinforcement requirements in these occasions vary greatly, which makes it easy for the traditional fixed gain configuration to cause a significant contrast in the listening experience of the audience in the front and back rows. Therefore, every time a new piece of music starts, the program will check the venue conditions in advance and then decide how to allocate the strength contrast relationship between the main channel and other auxiliary channels to maintain stable and clear audibility without losing the original charm and characteristics. For example, in a street flash mob event, in order to combat the environmental noise, the drum beats may be appropriately enhanced while reducing some unnecessary background background. However, in a relatively quiet small salon gathering, perhaps the deep low frequencies such as the bass should be weakened, and the ethereal piano scales should be emphasized to bring a soft atmosphere. Such adjustments require precise back-end measurement and real-time monitoring.

[0023] Finally, the unique sound of the guitar itself - that is, the resonance effect produced by the vibration and shaking of the strings must also be carefully polished and perfected to prevent excessive exaggeration or excessive blandness and weakness. This involves a comprehensive analysis and evaluation of the picked up vibration amplitude and the amount of energy contained in its duration, combined with the current position whether it is a solo-led or choral accompaniment, and then comprehensively considering the cooperation of other peripheral auxiliary instruments to formulate targeted improvement measures to maintain the natural and realistic touch reproduction without overly obscuring the dominant position of the human voice. In practice, this means not only to grasp the technical breakthroughs at the basic physical property level, but also to respect the traditional handicraft aesthetic principles to find the best compromise and intersection between the two to create a new-age music tool that meets the tastes of modern consumers and has great potential for personalized expression. For example, when plucking a fast arpeggio melody, the echo reverberation effect is cleverly used to create a three-dimensional surround space sense and enhance the live appeal.

[0024] Next, another embodiment of the present invention is described. First, the subject matter of the invention is accurately stated. For example, in a scenario involving a smart guitar with karaoke singing function, it is necessary to define the guitar as a unique musical instrument device that combines music playback and singing functions. This step is to enable potential reviewers or other readers to quickly understand the core features of the product and the industry to which it belongs, so as to avoid misunderstandings and clearly identify the boundaries of the technical solution.

[0025] The independent claims are then listed. This is the core part of the entire document and contains a combination of technical elements necessary to achieve a specific function. For the smart guitar case, an independent claim may be described as an electronic guitar, including an audio input device for receiving sound signals generated by the user through a microphone, and a processing module connected to the above-mentioned audio input end for analysis, recognition and mixing. This expression ensures that all novel points are fully protected. In this process, important factors such as the interaction principle of each component need to be considered.

[0026] Then, dependent claims are drafted to further refine or limit the content of the independent claims, enhance protection and expand coverage. For example, in any of the aforementioned electronic guitars, the processing module further integrates a real-time accompaniment track matching algorithm A=bx+cy (x refers to the frequency characteristic factor, and y corresponds to the user's singing emotion value), b and c are preset adjustment parameters, and the value range is limited to [0.8~1.3], and the optimal value is about 1.1. The application of this formula can enhance user experience and performance accuracy. Since the different frequency characteristics of human voices and the degree of emotional involvement during singing affect the overall effect, adjusting the two coefficients within a certain range can ensure adaptation to various singing scenarios. The selection of approximately 1.1 as the optimal parameter value is the result obtained after comprehensive consideration of multiple factors such as computational complexity and resource consumption.

[0027] Similar provisions may apply to smart guitars: any of the guitars described above may also include an ergonomic handle on the outer surface that facilitates carrying without affecting playing; or support Bluetooth wireless connection mode to facilitate the connection of more peripheral devices and expand functionality. These additional terms make the rights and interests ultimately obtained after authorization more solid and reliable, while reflecting the essential attributes and value of the invention as comprehensively as possible.

[0028] Next, the present invention is further described. The smart guitar according to claim 1 is optimized based on the karaoke singing function, and achieves better music effects by dynamically adjusting the balanced output characteristics of different accompaniment tracks and string pickup sensors. The specific steps are to analyze the energy distribution of each audio segment to establish a basic gain, and then calculate the actual gain based on the state during performance and make necessary compensation. Then, the pickup information is optimized in combination with the data collected by the motion sensor to ensure optimal reverberation, and the sensitivity of the input and output devices is adjusted in real time based on the ambient noise level to adapt to different application scenarios, such as changes in noise conditions in live performances or private practice scenarios, thereby improving the user experience quality.

[0029] In one embodiment, analyzing the energy distribution of audio means quantifying the strength of each independent channel or the overall output of the guitar accompaniment track or the sound frequency produced by the singer through the frequency domain or time window. For example, the energy performance during the transition between the chorus and the interlude of a pop song will be very different. Here, Gbase can be defined as an average energy value between -30dB and +12dB as the basic gain of each audio track.

[0030] Specifically, the preset formula is used to guide the transformation from the original sound characteristics to an enhanced result that is more suitable for the current situation. For example, at this stage, if a sudden highlight moment occurs during the playback of a song, such as the small climax before the arrival of a guitar solo, then Gin can be regarded as a specific value received by the system at that moment, usually ranging from 200mV to 4V; and Vol reflects the user's playing strength and the size of the pronunciation, taking the level range of 0dBu to +54dBu. The role of the F (Gin, Vol) formula is to describe the relationship between the two from a mathematical level. In this process, the nonlinear response mechanism of human perception of audio characteristics is taken into account. If it is found through calculation that Gfinal is lower than the set threshold Gmin (possibly below -48dB, which is a bottom-line protection measure to avoid distortion caused by signal loss during extremely quiet periods), then the minimum gain compensation operation will be triggered to readjust the output to the appropriate intensity.

[0031] The guitar's motion posture or movement frequency, or Acc, collected by the accelerometer, can be used in conjunction with the P algorithm to effectively process the pickup data and improve the echo quality. An example might be when a player shakes the instrument vigorously to enhance the rock style expression or gently swings it to create a warm atmosphere. The system can automatically sense and convert the vibration into additional reverberation layers to add to the performance. Here, the Acc data coverage range can be from 0.0g (completely stationary) to over 9.8g (the unit of the standard gravitational acceleration multiplier), and the P algorithm will adjust the effect mode to meet the expected effect according to the ideal acoustic model in different scenarios.

[0032] Based on the noisy background of the external surroundings, intelligent control of microphone pickup and speaker output capabilities is also a crucial improvement direction. The Nlv parameter refers to the level of interference sound detected in the current use environment. For example, when preparing to record in a relatively closed studio where other electrical noise may exist, Nlv will be around 40dBA. At this time, the microphone capture accuracy should be appropriately improved and the speaker leakage power should be reduced to ensure clarity. When holding a concert in an open and clean prairie, the value may be close to the natural bottom line of 30dBA or lower. In order to make the performance closer to the original ecological feeling, it is necessary to timely reduce the degree of hardware intervention in filtering out subtle noise. This approach allows the device to adaptively adjust performance parameters in various environments, thereby providing higher-quality sound reproduction authenticity and immersive user experience.

[0033] Next, the present invention is described further. For a smart guitar with a karaoke singing function, first, a fast Fourier transform (FFT) is performed on each audio frame and the peak frequency point Pmax within the frequency range is analyzed. The FFT can convert the time domain signal into the frequency domain, which can more accurately identify at which frequency or frequencies the sound represented by the current guitar has the strongest energy. For example, when the guitar starts to play "I'm Willing to Do It for You", this process can identify whether the main prominent sound in each measure of the song is the bass or the treble melody part, thereby obtaining the main performance frequency value Pmax of each time period.

[0034] An audio track equalization response curve Rcurve is established, and then adaptively corrected based on the peak frequency to achieve a more natural and pleasant harmonious balance between each channel. For example, the initial state of the response curve is an empirical value curve diagram formulated according to the sound characteristics of common musical instruments. After the above-mentioned Pmax is determined, this curve will be fine-tuned according to the actual data obtained. During the performance of "I'm Willing to Do It for You", the interaction between channels such as bass and lead vocals is just right.

[0035] According to the formula If(ENV > EnvLimit, AdjLevel++, AdjLevel), ENV is a comprehensive measure of external environmental factors such as the number of audience members or the degree of interference from surrounding traffic sounds (assuming a range of 0 to 100), EnvLimit serves as the judgment boundary (for example, set to 50, which is the optimal empirical value), and AdjLevel is the sound adjustment level for the entire system. This formula means that if external conditions are more complex and chaotic than the estimated limits, the depth of the music effect processing will be strengthened; otherwise, the current level will be maintained. For example, if the performance venue is moved from a quiet room to an outdoor square, due to the greater number of uncertainties in the square, AdjLevel will be increased to enhance the ability to respond to various emergencies, thereby ensuring that the performers are always in a good performance environment.

[0036] It is also worth mentioning that the display contrast parameters should be adjusted accordingly, and the synchronization rate should also match the rhythm of the lyrics and background music to ensure that the two can be presented seamlessly in front of the eyes without any obvious lack of synchronization. For example, when the guitar is connected to the device to display the real-time music score and accompaniment content of "I'm Willing to Do It for You", the system should ensure that the picture update speed keeps up with the sound changes to avoid any delay.

[0037] Next, further supplementary explanations of the present invention are described based on the following content: in a smart guitar with a karaoke singing function, key characteristic parameters Sfeat in the user's singing, such as the onset frequency and pitch change rate, are captured. This step is to analyze the user's voice characteristics and construct an individualized acoustic model Muser based on this. This helps to customize a vocal model suitable for each user and improve the realism and accuracy of human-computer interaction. Specifically, if a user tries to play a popular song, the user's voice parameters will be recorded at this stage. These parameters include but are not limited to the onset frequency range (for example, from 110 Hz to 987.77 Hz) and the pitch change rate (such as 20-50 times per second), thereby forming a corresponding personalized sound profile.

[0038] To evaluate individual singing, the smart guitar performs a multi-dimensional comparison between the extracted user Sfeat and a pre-set standard pitch model (Vcmp=Sim(Sfeat, Mstd)). Here, the function Sim defines a measure of the similarity between the two, and its value is usually limited to [0, 1]. The larger the value, the stronger the similarity. When Sim is close to or equal to 1 in the optimal state, it is considered that the ideal imitation level has been achieved. In one embodiment, if a learner deviates significantly from the correct tune during guitar practice, the calculated result Sim will be significantly lower than the ideal situation.

[0039] When the absolute value of the calculated real-time deviation Dreal exceeds a given limit, the judgment condition |Dreal|>Tolerance is met, and the user is reminded to make timely corrections. The tolerance error Tolerance here varies according to different application scenarios. For amateur guitar enthusiasts, a more relaxed threshold of around 10 cents can be set; for professional players with more stringent requirements, this parameter may be set to a smaller value of around 3-5 cents. For example, a novice player who first uses this karaoke guitar and practices the song "I'm Willing for You, I'm Willing" will automatically generate a corrective guidance signal to help the player improve if Dreal reaches or even exceeds the specified error tolerance.

[0040] In addition, through speech separation algorithms or other similar technologies, the human voice component can be accurately obtained and used as feedback information for internal calibration reference of the system to ensure that every music experience can be as close to the expected goal as possible. This feedback can not only serve as a helpful tool for improving singing skills, but also as important data for later software optimization, providing a reliable basis for the iteration of subsequent versions. Specifically, in the novice singing scenario mentioned above, the smart guitar can identify and retain pure human voice track data as sample data for in-depth analysis, so as to adjust and optimize the relevant algorithm logic so as to provide more personalized guidance and suggestions in the future.

[0041] Next, the expanded technical steps of the present invention are described, in which a nonlinear filter NonLinFilter is used to suppress howling. This is particularly important in a karaoke environment. When the microphone and speaker are placed close to each other or the directionality is inappropriate, howling may occur in the system, causing distortion of the sound signal and thus damaging the user's sound quality experience. For example, in one embodiment, the nonlinear filter can identify the changing characteristics of the sound signal before howling is about to occur, and instantly adjust the filter parameters to suppress the energy growth of the frequency band, thereby effectively reducing or even eliminating possible distortion, so as to ensure that the singer hears a natural and real mixture of his own voice and guitar music, ensuring the sense of presence.

[0042] A dynamic sound effect matrix, MatEff, is constructed and adjusted according to the guitar posture data, Posture, collected in real time. This process enables the smart guitar to automatically change according to different playing postures and strengths, creating a rich sense of spatial layering. Specifically, suppose a guitarist gradually stands up from lying on his knees and adopts an exaggerated playing posture. The system detects these changes through sensors built into the instrument and transmits the information to the built-in processor to calculate and update the various values ​​of the sound effect matrix. For example, it enhances the treble while slightly reducing the intensity of the low frequency band, thereby forming a sound directional movement effect that is more in line with the natural rhythm of the human body and creating an immersive musical performance.

[0043] The automatic switching of intelligent EQ Preset is designed to provide optimal de-reverberation settings and overall channel ratio management for different venue environments, ensuring that the consistency of good sound output can be maintained regardless of the type of occasion. In a certain scenario example, when it detects that the surrounding environment has changed from a relatively open small square to a small indoor studio, the device will immediately judge this change and automatically convert the equalization configuration to the preset mode required by the latter - increasing the attenuation of certain frequencies that are prone to resonance and reasonably adjusting the dry and wet levels to appropriate values, so that the singing and the background of the guitar will not appear hollow or too thick and stuffy. This operation method can greatly improve the professionalism of adapting to changing recording needs during improvisation and practice.

[0044] The mechanism designed according to the formula IF(PerfEval<=PerfThreshold,AdjustMix,ContinuePlaying) is used to compare the current performance Performance Evaluation (PerfEval) score with the system-defined minimum standard score threshold PerfThreshold (usually set in the range of 60-80 points) to determine whether to adjust the existing track mix ratio or continue normal playback without changing any settings. Here, the PerfEval parameter is used to reflect the overall quality of the current performance; the PerfThreshold The minimum limit value allowed by the system is set; both belong to the floating range and their best combination scheme is continuously optimized and adjusted iteratively with version upgrades. The reason for formulating such a strategy is that in actual application, external factors such as changes in the audience's mood or other uncontrollable conditions will inevitably affect the original ideal arrangement structure, which may cause the original ideal arrangement structure to lose its due coherence or coordinated beauty. In this case, it is necessary to immediately activate temporary emergency control measures. If the calculation results show that the current level fails to exceed the preset passing score, the corresponding remedial measures should be triggered to immediately correct the relative intensity distribution problem between each sound source; if the result shows that it is still above the safe level, it will not be processed and will continue to work according to the predetermined process until the new cycle begins to be judged again. This feedback self-calibration mechanism is conducive to maintaining high-quality output status throughout the performance and improving user experience.

[0045] Next, the invention is described in depth based on the following description: applying specific mathematical algorithms such as Hilbert transform (HilbTrans) to the string vibration signal to accurately measure the vibration amplitude and decay rate Ampi and SpeedDec of each string, thereby ensuring the true reproduction of the performance. This step processes the vibration data of the strings through HilbTrans, generates time domain and frequency domain features in the form of complex numbers, and extracts information such as vibration amplitude and decay rate. For example, when playing a song on a smart guitar, the system collects and analyzes the vibration of each string in real time. When the vibration of a string reaches a maximum displacement of 0.2 cm, it gradually stops vibrating at a speed of 10% per second. At this time, the Ampi value is 0.2 cm, and the SpeedDec is 0.1 / second.

[0046] After collecting RawSignal data from the pickup device, machine learning technology (ML) is applied to perform Classify (RawSignal) classification to accurately distinguish different sources (such as finger touch). During this process, a pre-trained model or a custom neural network framework is used to receive the raw signal input and, combined with the sample feature set, complete the task of identifying different types of sound sources. For example, lightly touching the E string on a smart guitar and firmly pressing the G string will output different current waveforms. The ML algorithm can identify these subtle differences and send the results to the control system for corresponding feedback actions.

[0047] The newly designed digital audio source conversion protocol DigiConvert allows original analog audio to be more efficiently encoded into digital form for further processing. DigiConvert specifies a set of standardized interfaces and format conversion rules, supports high-fidelity lossless compression and low-latency streaming transmission characteristics, and ensures that audio files are not distorted and synchronously played smoothly in various environments. Specifically, during the recording process, the pickup microphone captures the singer's singing voice and generates a binary data stream through analog-to-digital conversion to be transmitted to subsequent mixing plug-in operations; if this protocol is not followed, sound distortion may occur.

[0048] ColorChose and BrightnessCtrl are used to adjust the color and brightness of LED lights according to the real-time ambient brightness Lbright to optimize the live atmosphere experience of music performances. The brightness control algorithm dynamically adjusts the color scheme and luminous flux ratio based on the light intensity level read by the sensor. The brightness range is usually defined as an integer value between 0 and 255. The appropriate interval is selected to make the visual presentation harmonious and beautiful. For example, in darker conditions at night, setting the stage lights to warmer tones increases the intimate atmosphere and helps karaoke singers show their personal charm; on the contrary, during the day or in well-lit places, cool white tones are used to maintain a bright and open atmosphere to promote communication and interaction. Each formula reasonably sets the relevant parameter value range and the optimal solution based on the actual situation to achieve the best demonstration purpose.

[0049] The present invention provides an intelligent guitar with karaoke singing function, comprising: The smart guitar introduces advanced sound processing and synchronous display technologies based on traditional instruments, solving several key technical problems in existing audio input and output, lyrics display, user singing pitch judgment, accompaniment balance adjustment, and string pickup sensor reverberation adjustment, significantly improving the performance and user interactive experience. The following is the specific implementation process and technical solution path.

[0050] First, in regulating audio input and output to resolve delay and synchronization errors, the smart guitar's built-in microphone is used to capture the user's real-time singing and directly transmit the signal to the dedicated DSP processor in the central processing module (CPU). At the same time, the speaker is also connected to it, forming a closed feedback system. With the help of a high-precision delay correction algorithm and a phase-matching filter network, accurate measurement and correction of the signal transmission time from the acquisition end to the output end are achieved, ensuring a human-computer interaction environment with zero perceived delay (almost zero) regardless of changing environmental conditions.

[0051] Secondly, in terms of how to dynamically adjust the timing of lyric display and the accuracy of progress bar scrolling, a working mechanism that is interconnected with the lyrics database is cleverly adopted. Specifically, it predicts the content to be played next by parsing the encoded information or embedded timestamp metadata in the music file. In addition, it is equipped with an independent working frequency monitoring unit to track the deviation magnitude between the actual refresh times of the LCD / LED panel and the expected value. When there is an overlimit, the system responds immediately, that is, adjusts the driving voltage or refresh interval at the microsecond level, so that the position of text elements on the entire interface always maintains a stable and coherent advancement rhythm, and finally achieves the required high consistency effect, thereby completely solving any visual freeze phenomenon that may be caused by the inherent characteristics of the device.

[0052] Next, in terms of how to ensure a more scientific and fair assessment of the sound quality of users' singing, it is mainly reflected in the standard template matching model constructed by using adaptive multi-level digital filtering technology and machine learning. After the detection module obtains the instantaneous note waveform fragment, it is preprocessed to remove environmental noise interference and converted into a frequency domain expression that is easy to calculate subsequently; then the CPU calls the trained parameter matrix to perform feature point positioning operations - it can not only distinguish the frequency band distribution range corresponding to each effective fundamental tone, but also further quantify its deviation value relative to the theoretical curve as the basis for scoring. This not only improves the credibility of the results but also makes the scoring rules closer to the general industry standards.

[0053] Finally, as for how to enable different types of audio tracks to still achieve the ideal sound field layout effect in any sound pressure scenario, the key lies in the use of a dynamic equalization compensation strategy and its supporting hardware circuit. Whenever there are signs of large fluctuations in the lead vocal line (such as suddenly increasing hand speed), the self-test logic located in the internal DSP core will quickly identify the abnormal state and promptly activate the corresponding preset attenuation program group; at the same time, another set of related tasks is to continuously poll the load ratio relationship of the remaining sub-channels in the current environment, and determine the degree of gain multiplier change based on the results. This is equivalent to establishing a buffer zone between various possible interference sources, ensuring clarity without affecting the efficiency of other content transmission due to the excessive dominance of a certain local area.

[0054] In particular, for the collaborative operation mode between the specific mechanical vibration characteristics generated by the vibration of the guitar strings and the built-in sound pickup sensors, a closed-loop feedback solution based on the deep neural network prediction technology framework is adopted to optimize the overall auditory beauty presented in the end. By pre-recording a large number of actual recording samples for training, a highly abstract mathematical function relationship graph is obtained, and this is used as a guide to develop a set of effective online fine-tuning mechanisms: it can minimize the residual components without changing the original timbre, and can also flexibly adjust the reverberation parameter combination according to different user needs, so that each syllable seems to be placed in an exclusive space and is carefully carved and polished, greatly enriching and improving the expressiveness and artistic appeal of the musical work.

[0055] In summary, the smart guitar proposed in the present invention has achieved the best solution to the above-mentioned problems through careful design and technological innovation of details in each link, and is expected to create a new era of music and cultural dissemination channel with both professional quality and mass entertainment value.

[0056] In the above embodiments, they can be implemented in whole or in part through software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0057] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application, and the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without mutual contradiction.

[0058] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one such feature. In the description of this application, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0059] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment, or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes additional implementations in which the functions may be performed out of the order shown or discussed, including in a substantially simultaneous manner or in the reverse order depending on the functions involved.

[0060] The logic and / or steps represented in the flowchart or otherwise described herein may be considered, for example, as a sequenced list of executable instructions for implementing the logical functions, and may be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device).

[0061] It should be understood that various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0062] In addition, the functional units in the various embodiments of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. If the above-mentioned integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium, which can be a read-only memory, a disk or an optical disk, etc.

[0063] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A smart guitar with karaoke singing function, characterized in that: include: S101, detecting the user's singing voice in real time and synchronously adjusting the audio signal path parameters between the built-in microphone and the speaker to eliminate delays and synchronization errors caused by transmission; S102, automatically adjusting the display screen refresh rate according to the operating frequency of the lyrics display module, so that the lyrics display time is strictly synchronized with the scrolling of the music progress bar to eliminate display delay; S103, performing multi-stage filtering on the collected user singing audio data and comparing it with a pre-set standard pitch model to provide high-precision sound collection and accurate pitch judgment; S104: Dynamically adjust the balanced output characteristics of different accompaniment tracks and string pickup sensors based on the above-obtained information to ensure that the optimal sound ratio and reverberation effect are achieved under various playing conditions.

2. The smart guitar with karaoke singing function according to claim 1, characterized in that: The method of dynamically adjusting the balanced output characteristics of different accompaniment tracks and string pickup sensors based on the information obtained in the above steps is as follows: Analyze the energy distribution of each audio segment to determine the base gain Gbase of each track; The actual gain Gfinal after real-time adjustment is calculated according to the preset formula F, where F represents the gain adjustment formula, Gin represents the initial volume, and Vol represents the total volume of the current performance. If Gfinal < Gmin, the minimum gain compensation is set; The accelerometer is used to collect the guitar's motion data Acc, and the preset algorithm P is used to optimize the data of the string pickup sensor to achieve the best reverberation effect; Dynamically adjust the sensitivity of the microphone and speaker based on the ambient noise level Nlv to improve the authenticity of sound reproduction in low-noise environments.

3. The smart guitar with karaoke singing function according to claim 2, characterized in that: The basis further includes: Perform fast Fourier transform (FFT) on each audio frame and analyze the peak frequency point Pmax within the frequency range; Establish the track equalization response curve Rcurve, and make adaptive corrections based on Pmax to make the sound of each channel more harmonious and balanced; Dynamically controls the overall timbre adjustment level AdjLevel according to the formula If, where ENV is a measure of external environmental factors and EnvLimit is a threshold set by the system to determine whether to increase or decrease the timbre processing intensity. Adjust the display contrast parameters and lyrics synchronization rate to ensure that lyrics and background music are displayed synchronously without delay.

4. The smart guitar with karaoke singing function according to claim 3, characterized in that: Further supplementary explanations are based on the following content: Capture the key characteristic parameters Sfeat of the user's singing and build an individualized acoustic model Muser; Compare Sfeat and the standard pitch model in multiple dimensions Vcmp = Sim, where Sim is defined as the similarity function between the two features to evaluate the degree of match between them; Calculate the real-time deviation Dreal. If the judgment condition |Dreal| > Tolerance is met, prompt the user to make corrections. The human voice part is extracted as feedback information and provided to the system for later correction reference.

5. The smart guitar with karaoke singing function according to claim 4, characterized in that: The technology-based steps described in the expansion also include: Use nonlinear filtering NonLinFilter to suppress howling and prevent sound signal distortion under extreme conditions that affect the sound quality experience; Construct a dynamic sound effect matrix MatEff and accurately adjust the matrix parameters based on the guitar posture data Posture to produce different spatial sound field perceptions and enhance the stereo effect; Realize intelligent automatic switching of EQ Preset settings to ensure optimal reverberation and sound balance ratio in any environment; The current performance is evaluated through the IF logic formula. When the evaluation score is lower than the system default minimum standard PerfThreshold, the Mix settings are adjusted; otherwise, the status quo remains unchanged.

6. The smart guitar with karaoke singing function according to claim 1, characterized in that: The description of the basis is further described as: Specific mathematical algorithms such as Hilbert Transform (HilbTrans) are applied to the string vibration signals to accurately measure the vibration amplitude and decay rate (Ampi, SpeedDec) of each string, ensuring the true reproduction of the performance. After collecting RawSignal data from pickup devices, we apply machine learning (ML) technology to classify and accurately distinguish different sources. A new digital audio source conversion protocol, DigiConvert, has been designed to enable more efficient encoding of original analog audio into digital form for further processing. Based on the real-time environment brightness Lbright, the color ColorChose and brightness BrightnessCtrl of the LED light are controlled to create a better stage atmosphere to assist the music performance.

Images