Method, system and device for sound and picture hot synergy control of karaoke projection device

Abstract

The present application relates to the technical field of intelligent consumer equipment, image communication, audio analysis processing, intelligent voice, projection display control, energy-saving projection equipment and intelligent thermal management, and particularly relates to a sound and picture thermal cooperative control method, system and equipment for karaoke projection equipment. Specifically, microphone human voice audio signals, accompaniment audio signals, lyrics time stamps, projection picture data, thermal state data and fan speed feedback data are synchronously acquired to determine an acoustically maskable heat dissipation window. When the window is not met and a pre-adjusted heat condition is met, a thermal load transfer mark is generated to reduce the thermal load of a non-lyrics display area and maintain the readability of a lyrics display area. The fan speed increment is released at a subsequent window, and the microphone wind noise is suppressed according to a wind noise template. Based on the wind noise suppression result and the thermal state data, the next time window control quantity is closed-loop distributed. The method can balance the human voice intelligibility, lyrics readability and overall stability of heat dissipation.

Description

Technical Field

[0001] This invention relates to the fields of intelligent consumer devices, image communication, audio analysis and processing, intelligent voice, projection display control, energy-saving projection equipment, and intelligent thermal management, and particularly to a method, system, and device for coordinated audio-visual and thermal control of a karaoke projection device. Background Technology

[0002] With the miniaturization and integration of home entertainment devices, karaoke projector devices integrating microphone input, karaoke processing, sound output, and projection display are increasingly being used in home karaoke, party entertainment, and portable audio-visual playback scenarios. For example, Chinese patent CN209132741U, entitled "Karaoke Projector All-in-One Machine," discloses a karaoke projector all-in-one machine. This device includes a sound input module, a karaoke module, a projection module, and a sound output module. The karaoke module and the projection module are communicatively connected, enabling karaoke and projection display functions within the integrated device. Furthermore, projection devices typically require fans to dissipate heat from heat-generating components during operation. Chinese patent CN203587949U, entitled "Projector and its Heat Dissipation Device," discloses a projector heat dissipation device that uses a sound sensor to monitor the external noise level of the projector in real time and controls the fan speed based on the external noise level to balance noise reduction and heat dissipation performance.

[0003] However, for projection devices with integrated karaoke functionality, microphone pickup, accompaniment playback, lyrics display, projection light source heat generation, audio amplifier heat generation, and fan cooling often occur simultaneously. Existing all-in-one karaoke projectors focus on integrating the karaoke module and the projection module, and existing projection cooling solutions focus on the projector's own heat dissipation and fan control in noisy environments. These solutions are insufficient to fully meet the comprehensive requirements of karaoke projection devices in continuous singing scenarios, which demand microphone vocal clarity, lyrics display readability, and overall heat dissipation stability. Therefore, how to reduce the impact of fan noise on microphone vocal audio signals during the operation of karaoke projection devices, while simultaneously ensuring the readability of the lyrics display area and controlling the thermal risks to the optical engine, processor, and audio amplifier, has become a technical problem that needs to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and device for audio-visual thermal coordination control of a karaoke projection device, addressing the shortcomings of existing technologies. This aims to solve the following technical problems: A method and system for audio-visual thermal coordination control of a karaoke projection device is provided to address the lack of effective coordination between fan cooling, microphone pickup, projected image display, and thermal protection of heat-generating components during continuous karaoke projection in existing karaoke projection devices. This leads to issues such as fan noise affecting microphone audio signals, reduced brightness of the entire projected image affecting the readability of the lyrics display area, and difficulty in timely balancing of thermal risks from the optical engine, processor, or audio amplifier.

[0005] This invention achieves the above objective through the following technical solution: a method for audio-visual thermal coordination control of a karaoke projection device, executed by a coordination control unit, comprising the following steps: Data synchronization steps: Synchronously acquire microphone voice audio signal, accompaniment audio signal, lyrics timestamp, projection screen data, thermal status data and fan speed feedback data; Window determination step: Determine whether the current time window is an acoustically maskable heat dissipation window based on the microphone voice audio signal, the accompaniment audio signal and the lyrics timestamp; Transfer marker step: When the current time window is not the acoustically maskable heat dissipation window and the thermal state data meets the pre-heating conditions, a heat load transfer marker is generated, and the fan speed increment is postponed to the subsequent acoustically maskable heat dissipation window; The heat load transfer adjustment step is to reduce the heat load in the non-lyrics display area while maintaining the readability of the lyrics display area according to the heat load transfer marker. Template compensation steps: Based on the fan speed feedback data, call the wind noise template and suppress wind noise in the microphone human voice audio signal; Closed-loop allocation steps: Based on the wind noise suppression results and the thermal state data, allocate the fan speed increment and the reduction of thermal load in the non-lyric display area for the next time window.

[0006] Specifically, the execution entity of this method is the collaborative control unit in the karaoke projection device. The collaborative control unit can be formed by a processor executing a computer program in memory, or it can be implemented jointly by a processor, a digital signal processor, and a microcontroller. The data synchronization step aims to process data from the audio side, the screen side, the thermal status side, and the fan side within the same time window, avoiding misalignments where audio data belongs to the current moment while temperature data belongs to the previous moment. Microphone voice audio signals are used to determine if a singer is currently present; accompaniment audio signals are used to determine if fan noise can be masked by the accompaniment; lyrics timestamps are used to determine if the current moment is near a lyric transition or a singing pause; projected screen data is used to distinguish between lyric display areas and non-lyric display areas; thermal status data is used to determine if cooling intervention is needed; and fan speed feedback data is used to determine the actual source of fan noise. The core of the window determination step is to determine whether it is appropriate to increase the fan speed. If a vocal performance is currently underway and the vocals are strong, increasing the fan speed can easily introduce wind noise into the microphone's vocal audio signal. Conversely, if the accompaniment is strong, the vocals are weak, or there is a lyric transition, the fan noise is more easily masked by the accompaniment. The transfer marking step records any unexecuted fan speed increments in situations where increasing the fan speed is not suitable but heat dissipation is still required, deferring these increments to subsequent acoustically maskable heat dissipation windows. The yield adjustment step reduces the heat load on the non-lyrics display area within the current time window to partially compensate for the fan's cooling pressure, while maintaining the readability of the lyrics display area. The template compensation step calls the corresponding wind noise template based on actual fan speed feedback data to suppress wind noise in the microphone's vocal audio signal. The closed-loop allocation step reallocates the fan speed increment and the reduction in heat load on the non-lyrics display area for the next time window based on the wind noise suppression results and thermal state data, enabling the control process to automatically adjust based on actual residual noise and temperature changes.

[0007] Further solution: The window determination step includes: extracting vocal loudness, vocal fundamental frequency stability, and pause duration from the microphone vocal audio signal; extracting accompaniment loudness and accompaniment masking from the accompaniment audio signal; and identifying lyric transition intervals based on the lyric timestamps. When the accompaniment masking meets the accompaniment masking condition, and the vocal loudness, vocal fundamental frequency stability, pause duration, or lyric transition interval meets the vocal weakening condition, the current time window is determined as the acoustically maskable heat dissipation window. Specifically, the window determination step can be completed jointly by audio features and lyric time. Vocal loudness is used to represent the strength of the current microphone vocal audio signal; vocal fundamental frequency stability is used to represent whether there is a continuous and stable singing sound; and pause duration is used to represent whether the user is breathing, pausing, or waiting for the next lyric. Accompaniment loudness is used to represent the overall intensity of the accompaniment audio signal, and accompaniment masking is used to represent the accompaniment audio signal's ability to mask fan noise in the wind noise frequency band. The lyric transition interval is used to represent the transition time period between adjacent lyrics, which is usually when the vocal input is weak or there is no continuous singing. In implementation, the microphone vocal audio signal and the accompaniment audio signal can be processed by framing, and then the above parameters can be extracted within a time window consisting of each frame or multiple consecutive frames. Only when the accompaniment masking condition and the vocal weakening condition are met is the current time window determined to be an acoustically maskable heat dissipation window. This avoids increasing the fan speed simply based on the loudness of the accompaniment, because when the accompaniment is strong but the user is singing loudly at the same time, wind noise may still enter the microphone vocal audio signal and affect the clarity of the vocals.

[0008] Further solution: The vocal weakening conditions include: the vocal loudness is lower than a vocal loudness threshold, or the vocal fundamental frequency stability is lower than a fundamental frequency stability threshold, or the pause duration is higher than a pause duration threshold, or the lyrics timestamp indicates that the current time window is within the lyrics transition interval. Specifically, the vocal weakening conditions are used to determine from multiple perspectives whether the current microphone pickup is in a relatively insensitive state. When the vocal loudness is lower than the vocal loudness threshold, it indicates that the current user's singing voice is weak or there is no obvious singing voice. When the vocal fundamental frequency stability is lower than the fundamental frequency stability threshold, it indicates that there is no stable vocal fundamental frequency in the current microphone vocal audio signal, and it may be in a state of breathing, pause, tone transition, or noise interference. When the pause duration is higher than the pause duration threshold, it indicates that there has been no effective singing vocals for a period of time. When the lyrics timestamp indicates that the current time window is within the lyrics transition interval, it indicates that the user is currently switching from one lyric to the next, and this stage is usually more suitable for releasing the fan speed increment than continuous singing segments. Using the above conditions in an "OR" relationship can improve the coverage of window recognition, so that the device does not have to rely on a single audio indicator, thereby reducing the probability of misjudgment.

[0009] Further solution: The accompaniment masking condition includes: the accompaniment loudness is higher than the accompaniment loudness threshold, and the accompaniment masking amount is higher than the accompaniment masking amount threshold; the accompaniment masking amount is generated based on the energy proportion of the accompaniment audio signal in the wind noise frequency band. Specifically, the accompaniment masking condition is used to determine whether the accompaniment audio signal is sufficient to mask the wind turbine noise. When the accompaniment loudness is higher than the accompaniment loudness threshold, it indicates that the overall sound of the current accompaniment is relatively large, but the overall loudness alone is not enough to indicate that it can mask the wind noise. Therefore, it is also necessary to determine whether the accompaniment masking amount is higher than the accompaniment masking amount threshold. The accompaniment masking amount can be generated based on the energy proportion of the accompaniment audio signal in the wind noise frequency band. For example, after determining that the wind turbine noise is mainly concentrated in a certain frequency band during the equipment calibration stage, the co-control unit calculates the energy proportion of the accompaniment audio signal in that frequency band during operation. If the accompaniment audio signal has high energy in the wind noise frequency band, the fan noise is more easily masked by the accompaniment; if the accompaniment audio signal has high overall loudness but low energy in the wind noise frequency band, the fan noise may still be perceived by the user or clearly picked up by the microphone. By setting both the accompaniment loudness threshold and the accompaniment masking threshold simultaneously, the release of fan speed increments can be made more reliable.

[0010] Further solution: The transfer marking step includes: when the current time window is not the acoustically maskable heat dissipation window and the thermal state data meets the pre-adjustment heat conditions, recording the unexecuted fan speed increment in the current time window, and writing the unexecuted fan speed increment into the heat load transfer mark. Specifically, the unexecuted fan speed increment in the transfer marking step refers to the fan speed increment that is temporarily suspended in the current time window due to unsuitable acoustic conditions. For example, the thermal state data indicates that the fan speed needs to be increased by one level, but a vocal performance is taking place in the current time window, so the collaborative control unit does not immediately increase the fan speed, but records the unexecuted fan speed increment. The heat load transfer mark can at least record the unexecuted fan speed increment, the time window in which the mark is generated, the thermal state data that triggers the pre-adjustment heat conditions, and the amount of heat load reduction in the currently used non-lyrics display area. This heat load transfer mark is equivalent to a record of heat dissipation tasks to be processed. Once an acoustically maskable heat dissipation window appears, the collaborative control unit can release the corresponding fan speed increment according to the record, without losing the heat dissipation requirements of the previous time window.

[0011] Further solution: Following the transfer marking step, an incremental release step is included: when a subsequent time window is determined to be an acoustically maskable heat dissipation window, the unexecuted fan speed increment is released according to the heat load transfer mark; when the subsequent time window is still not an acoustically maskable heat dissipation window, the heat load of the non-lyrics display area continues to decrease. Specifically, the incremental release step is used to handle the already delayed fan speed increment. When a subsequent time window is determined to be an acoustically maskable heat dissipation window, the collaborative control unit reads the unexecuted fan speed increment recorded in the heat load transfer mark and converts the unexecuted fan speed increment into a control quantity of the heat dissipation execution unit, so that the fan speed increases during periods when the accompaniment masking ability is strong or the vocals are weak. If the subsequent time window still does not belong to an acoustically maskable heat dissipation window, it means that it is still not suitable to directly increase the fan speed, and the collaborative control unit continues to bear the current thermal pressure by reducing the heat load of the non-lyrics display area. This treatment method enables the fan increment to release across time windows, which can avoid abrupt wind noise during continuous singing segments, and at the same time avoid the continuous accumulation of thermal risks due to the fan speed not being increased for a long time.

[0012] Further solution: In the transfer adjustment step, the thermal load of the non-lyrics display area includes at least one of the following: background display area brightness load, non-lyrics display area dynamic contrast enhancement load, non-lyrics display area sharpening enhancement load, and non-lyrics display area super-resolution processing load. Specifically, the thermal load of the non-lyrics display area refers to the power consumption and heat contribution generated to the projected image and image processing process outside the lyrics display area. The background display area brightness load is related to the projection light source power; reducing the background display area brightness can reduce the output load of the light source driving unit. The non-lyrics display area dynamic contrast enhancement load, non-lyrics display area sharpening enhancement load, and non-lyrics display area super-resolution processing load are related to the computational load of the processor or image processing unit; reducing these image enhancement loads can reduce processor heat generation. In implementation, the collaborative control unit can prioritize reducing the brightness of the background display area, which is not sensitive to the user, then reduce the dynamic contrast enhancement of the non-lyrics display area, and subsequently reduce the sharpening enhancement or super-resolution processing of the non-lyrics display area. This sequence can reduce the thermal load while maintaining the karaoke visual experience as much as possible.

[0013] Further solution: In the transfer adjustment step, maintaining the readability of the lyrics display area includes: maintaining the brightness of the lyrics display area at no lower than the lyrics readability brightness threshold, or maintaining the contrast of the lyrics display area at no lower than the lyrics readability contrast threshold. Specifically, maintaining the readability of the lyrics display area is used to prevent the thermal load reduction process from affecting the user's singing along. The lyrics display area brightness not being lower than the lyrics readability brightness threshold means that the display brightness of the lyrics text or the background of the lyrics area can still meet the requirements for clear display. The lyrics display area contrast not being lower than the lyrics readability contrast threshold means that there is still a sufficient difference between the lyrics text and the lyrics background, allowing the user to quickly identify the lyrics content. In implementation, the collaborative control unit can first identify the lyrics display area based on the projected image data, and then set a lower limit for brightness and a lower limit for contrast for that area. Even if the brightness of the background display area is reduced or the image enhancement is downgraded, the lyrics display area remains readable, thereby preventing the user from interrupting their singing due to not being able to see the lyrics clearly.

[0014] Further solution: The projected image data includes a lyrics display area identifier and a background display area identifier; the yielding adjustment step includes: locking the lyrics display area based on the lyrics display area identifier, determining the non-lyrics display area based on the background display area identifier, and performing brightness reduction or image enhancement degradation on the non-lyrics display area. Specifically, the lyrics display area identifier and background display area identifier in the projected image data are used for regional control of the projected image. The lyrics display area identifier can be directly provided by the karaoke application or generated by the projection display unit based on the lyrics rendering coordinates. The background display area identifier can be determined by display areas other than the lyrics display area, or by display layers such as video image layer, scoring image layer, and interactive prompt layer. The yielding adjustment step first locks the lyrics display area based on the lyrics display area identifier to ensure that the lyrics display area does not participate in significant brightness reduction or image enhancement degradation; then, it determines the non-lyrics display area that can be yielded based on the background display area identifier and performs brightness reduction or image enhancement degradation on the non-lyrics display area. This avoids the entire projected image darkening simultaneously, making the heat load reduction process smoother.

[0015] Further details: The thermal state data includes optical engine temperature, processor temperature, audio amplifier temperature, optical engine heating rate, processor heating rate, and audio amplifier heating rate. The pre-adjustment thermal conditions include: at least one of the optical engine temperature, processor temperature, and audio amplifier temperature reaching a corresponding pre-adjustment temperature threshold, or at least one of the optical engine heating rate, processor heating rate, and audio amplifier heating rate reaching a corresponding pre-adjustment heating rate threshold. Specifically, the thermal state data is used to represent the temperature state of multiple heat-generating components inside the karaoke projection device. The optical engine temperature reflects the thermal state of the projection light source, optical engine components, or related heat dissipation components; the processor temperature reflects the thermal state resulting from audio / video decoding, image enhancement, and collaborative control calculations; and the audio amplifier temperature reflects the thermal state resulting from accompaniment playback and microphone voice output. The optical engine heating rate, processor heating rate, and audio amplifier heating rate are used to determine the temperature change trend. The pre-adjustment thermal conditions can be triggered not only by the temperature reaching the pre-adjustment temperature threshold but also by the heating rate reaching the pre-adjustment heating rate threshold. Even if the current temperature is not very high, but the temperature rises rapidly, the collaborative control unit can perform heat load transfer and yield adjustment in advance to avoid passively reducing power after the temperature rises rapidly.

[0016] Further solution: Before the template compensation step, a template establishment step is included: the control cooling execution unit sequentially outputs multiple fan speed levels, and when the microphone voice input condition is not met, it collects ambient noise audio signals, and generates the wind noise template corresponding to the multiple fan speed levels based on the ambient noise audio signals. Specifically, the template establishment step is used to enable the karaoke projection device to identify the noise characteristics generated by its own cooling fans at different speed levels. When establishing the wind noise template, the cooling execution unit sequentially outputs multiple fan speed levels, such as low speed, medium speed, and high speed. Each fan speed level is maintained for a stable period of time. When the microphone voice input condition is not met, the audio acquisition unit collects ambient noise audio signals. Since there is no effective voice input at this stage, the main noise collected is related to the fan speed level. The co-control unit performs spectrum analysis on the ambient noise audio signal, extracts the wind noise spectrum characteristics of the corresponding fan speed level, generates a wind noise template, and stores it in the memory. During subsequent singing, as long as the current fan speed feedback data is read, the corresponding wind noise template can be called for compensation.

[0017] Further solution: The template compensation step includes: determining the current fan speed level based on the fan speed feedback data, calling the wind noise template corresponding to the current fan speed level, separating the wind noise component from the microphone voice audio signal based on the wind noise template, and outputting the wind noise-suppressed voice audio signal. Specifically, during operation, the template compensation step determines the current fan speed level based on the fan speed feedback data and calls the wind noise template corresponding to the current fan speed level. After the audio processing unit or template compensation module converts the microphone voice audio signal to the frequency domain, it matches it with the corresponding wind noise template, estimates the wind noise component in the microphone voice audio signal, and then separates or weakens the wind noise component from the microphone voice audio signal to obtain the wind noise-suppressed voice audio signal. Since the wind noise template corresponds to the fan speed level, the compensation object is not general environmental noise, but the noise generated by the heat dissipation execution unit of the karaoke projection device itself at the corresponding speed, thus the wind noise suppression is more targeted.

[0018] Further solution: The closed-loop allocation step includes: when the wind noise suppression result indicates that the residual wind noise is higher than the residual wind noise threshold, reducing the fan speed increment in the next time window and increasing the heat load reduction in the non-lyrics display area in the next time window; when the thermal status data indicates that the thermal risk continues to rise, increasing the fan speed increment or increasing the heat load reduction in the non-lyrics display area in the next time window. Specifically, the closed-loop allocation step adjusts the control quantity of the next time window according to the changing trends of residual wind noise and thermal risk. When the wind noise suppression result indicates that the residual wind noise is higher than the residual wind noise threshold, it means that the current fan speed increment still has a significant impact on the microphone's human voice audio signal. Therefore, the next time window should reduce the fan speed increment and increase the heat load reduction in the non-lyrics display area, so that more thermal control pressure is borne by the projection screen side. When the thermal status data indicates that the thermal risk continues to rise, it means that the current heat dissipation is insufficient or the amount of heat transfer is insufficient. The next time window needs to increase the fan speed increment or further increase the heat load reduction in the non-lyrics display area. This closed-loop allocation method can rebalance the impact of wind noise and thermal protection requirements in each time window, avoiding excessive sacrifice on one side due to a fixed strategy.

[0019] Further solution: Before the window determination step, a pre-heating step is included: identifying the upcoming chorus time period based on the lyrics timestamp, determining the predicted vocal volume and predicted accompaniment volume corresponding to the chorus time period based on historical singing data, and releasing the delayed fan speed increment in the acoustically maskable heat dissipation window before the chorus time period, or reducing the thermal load of the non-lyrics display area. Specifically, the pre-heating step is used to handle thermal risks in advance when the song structure is predictable. The lyrics timestamp can not only represent the current lyrics but also identify the upcoming chorus time period. The chorus time period usually has high accompaniment and vocal volume, and the audio amplifier output power, processor processing load, and user singing intensity may increase simultaneously. The co-control unit can determine the predicted vocal volume and predicted accompaniment volume corresponding to the chorus time period based on historical singing data, and find the acoustically maskable heat dissipation window before the chorus time period arrives. If there is a delayed fan speed increment, the fan speed increment is released in the acoustically maskable heat dissipation window before the chorus time period; if it is not suitable to release the fan speed increment, the thermal load of the non-lyrics display area is reduced in advance. This pre-heating method can avoid the need to force the brightness to drop or the fan to increase only after the temperature suddenly rises during the chorus.

[0020] Further solution: Following the transfer adjustment step, a power amplifier coordination step is also included: when the thermal state data meets the power limiting condition, the upper limit of the audio power amplifier output power is reduced, and the voice frequency band clarity compensation is increased, so that the voice audio signal after wind noise suppression maintains voice clarity after the upper limit of the audio power amplifier output power is reduced. Specifically, the power amplifier coordination step is used to handle situations where the audio power amplifier temperature is high. When the thermal state data meets the power limiting condition, it indicates that at least one heat-generating component in the audio power amplifier, optical engine, or processor has reached a state requiring power limitation. If the upper limit of the audio power amplifier output power is directly reduced, the user may perceive a weakening or decreased clarity in the voice. Therefore, when reducing the upper limit of the audio power amplifier output power, the coordination control unit simultaneously increases the voice frequency band clarity compensation. The voice frequency band clarity compensation can be achieved by increasing the gain of the main voice frequency band, moderately reducing low-frequency muddiness, or enhancing consonant clarity. Since the template compensation step has already output the voice audio signal after wind noise suppression, the power amplifier coordination step can maintain voice clarity while reducing power amplifier heat generation.

[0021] A sound and image thermal coordination control system for a karaoke projection device includes: The data synchronization module is used to synchronously acquire microphone voice audio signals, accompaniment audio signals, lyrics timestamps, projection screen data, thermal status data, and fan speed feedback data. The window determination module is used to determine whether the current time window is an acoustically maskable heat dissipation window based on the microphone voice audio signal, the accompaniment audio signal and the lyrics timestamp; The transfer marker module is used to generate a heat load transfer marker when the current time window is not the acoustically maskable heat dissipation window and the thermal state data meets the preheating conditions, and to postpone the fan speed increment to the subsequent acoustically maskable heat dissipation window. The heat transfer adjustment module is used to reduce the heat load in the non-lyrics display area and maintain the readability of the lyrics display area according to the heat load transfer mark. The template compensation module is used to call the wind noise template based on the fan speed feedback data and suppress wind noise in the microphone human voice audio signal; The closed-loop allocation module is used to allocate the fan speed increment and the reduction of heat load in the non-lyrics display area for the next time window based on the wind noise suppression result and the thermal state data.

[0022] Specifically, the system employs a modular approach to achieve coordinated audio-visual and thermal control. The data synchronization module is responsible for collecting and aligning multi-source data; the window determination module determines whether increasing the fan speed is appropriate; the transfer marker module generates a thermal load transfer marker when increasing the fan speed is not suitable; the yield adjustment module reduces the thermal load in the non-lyrics display area while protecting the readability of the lyrics display area; the template compensation module calls the wind noise template based on the fan speed feedback data and suppresses wind noise in the microphone's human voice audio signal; and the closed-loop allocation module reallocates the fan speed increment and the reduction in thermal load in the non-lyrics display area within the next time window. All of these modules can run entirely within the processor, or they can be partially implemented collaboratively by an audio processing chip, an image processing chip, and a microcontroller. The modules are connected via data streams and control signals, forming a complete link from data acquisition to window determination, thermal load transfer, image yield, wind noise compensation, and closed-loop allocation.

[0023] Further solution: The window determination module includes a voice feature submodule, an accompaniment masking submodule, and a lyrics interval submodule. The voice feature submodule extracts voice loudness, fundamental frequency stability, and pause duration. The accompaniment masking submodule extracts accompaniment loudness and accompaniment masking amount. The lyrics interval submodule identifies lyric transition intervals based on the lyrics timestamp. Specifically, the voice feature submodule, accompaniment masking submodule, and lyrics interval submodule in the window determination module process data from different sources. The voice feature submodule extracts voice loudness, fundamental frequency stability, and pause duration from the microphone voice audio signal to determine whether the current voice is in a sensitive state. The accompaniment masking submodule extracts accompaniment loudness and accompaniment masking amount from the accompaniment audio signal to determine whether the accompaniment sound is sufficient to mask fan noise. The lyrics interval submodule identifies lyric transition intervals based on the lyrics timestamp to determine whether it is appropriate to release the fan speed increment from the perspective of song progression. The outputs of the three sub-modules are input to the window determination module, which allows the determination of the acoustically maskable heat dissipation window to consider the human voice, accompaniment, and lyric rhythm simultaneously, avoiding misjudgment based on a single indicator.

[0024] Further solution: The transfer marking module includes an incremental recording submodule and a delayed release submodule. The incremental recording submodule records the fan speed increments that are not executed within the current time window, and the delayed release submodule releases these unexecuted fan speed increments when a subsequent time window is determined to be an acoustically maskable heat dissipation window. Specifically, the incremental recording submodule in the transfer marking module records the fan speed increments that are not executed within the current time window. These unexecuted fan speed increments can be calculated from thermal state data and pre-adjusted heating conditions, for example, based on temperature difference, heating rate, or the current fan speed setting. The delayed release submodule releases these unexecuted fan speed increments when a subsequent time window is determined to be an acoustically maskable heat dissipation window. If multiple subsequent time windows do not meet the acoustically maskable heat dissipation window conditions consecutively, the delayed release submodule can continue recording and notify the transfer adjustment module to increase the reduction in heat load in the non-lyrics display area. Through the cooperation of the incremental recording submodule and the delayed release submodule, the heat dissipation control is no longer limited to a single time window but can be transferred across time windows based on acoustic conditions.

[0025] Further solution: The concession adjustment module includes a region locking submodule, a brightness concession submodule, and an image quality concession submodule. The region locking submodule is used to lock the lyrics display area and non-lyrics display area based on the projected image data. The brightness concession submodule is used to reduce the brightness of the non-lyrics display area, and the image quality concession submodule is used to reduce the image enhancement computation load of the non-lyrics display area. Specifically, the region locking submodule in the concession adjustment module is used to lock the lyrics display area and non-lyrics display area based on the projected image data. The brightness concession submodule is used to reduce the brightness of the non-lyrics display area, reducing the load on the light source driving unit. The image quality concession submodule is used to reduce the image enhancement computation load of the non-lyrics display area, such as reducing the computation intensity of dynamic contrast enhancement, sharpening enhancement, or super-resolution processing. The region locking submodule first determines which areas belong to the lyrics display area, and the brightness concession submodule and image quality concession submodule only process the non-lyrics display area, avoiding excessive dimming or degradation of the lyrics display area. This reduces the thermal load while maintaining the readability of the lyrics display area.

[0026] Further solution: The template compensation module includes a template creation submodule and a template calling submodule. The template creation submodule is used to create wind noise templates according to multiple fan speed levels. The template calling submodule is used to call the corresponding wind noise template based on the fan speed feedback data and output the suppressed human voice audio signal. Specifically, the template creation submodule in the template compensation module is used to create wind noise templates according to multiple fan speed levels. During the equipment initialization phase, the template creation submodule can control the heat dissipation execution unit to sequentially enter different fan speed levels and collect fan noise when there is no effective human voice input, forming a wind noise template corresponding to each level. The template calling submodule is used to call the corresponding wind noise template based on the fan speed feedback data and output the suppressed human voice audio signal. Since the template calling submodule reads the fan speed feedback data in real time, when the fan speed changes from low speed to medium speed or high speed, the template calling submodule can synchronously switch the wind noise template, so that the wind noise suppression always matches the actual fan noise.

[0027] A karaoke projection device includes a processor, a memory, an audio acquisition unit, an audio processing unit, a projection display unit, a light source driving unit, a temperature acquisition unit, and a heat dissipation execution unit. The memory stores a computer program, and the processor, when executing the computer program, implements the audio-visual-thermal coordinated control method of the karaoke projection device described above. Specifically, the karaoke projection device includes the hardware foundation required to implement the above control method. The processor executes the computer program in the memory, which stores the control program, threshold parameters, wind noise templates, and historical singing data. The audio acquisition unit acquires microphone audio signals and can receive accompaniment audio signals or provide accompaniment-related data to the coordinated control unit. The audio processing unit performs voice enhancement, reverberation, volume equalization, and amplifier output control on the wind noise-suppressed voice audio signal. The projection display unit generates projection image data and displays lyrics display areas and non-lyric display areas. The light source driving unit adjusts the projection brightness according to the projection control quantity. The temperature acquisition unit acquires the optical engine temperature, processor temperature, and audio amplifier temperature. The heat dissipation execution unit executes fan speed increments and feeds back fan speed feedback data to the coordinated control unit. The aforementioned hardware units are connected through a collaborative control unit to form a closed loop, enabling the device to independently complete audio-visual and thermal collaborative control.

[0028] The beneficial effects of this invention are: This invention synchronously acquires microphone vocal audio signals, accompaniment audio signals, lyrics timestamps, projection screen data, thermal status data, and fan speed feedback data through a data synchronization step. This allows data from the audio side, display side, thermal status side, and fan side to be processed collaboratively within the same time window, avoiding isolated control based on a single type of data. The window determination step determines whether the current time window is an acoustically maskable heat dissipation window based on the microphone vocal audio signal, accompaniment audio signal, and lyrics timestamp. This links the timing of fan speed increments with the karaoke performance status, accompaniment masking capability, and lyrics timing, thereby reducing interference from sudden changes in fan speed on the microphone vocal audio signal.

[0029] When the current time window is not an acoustically maskable heat dissipation window and the thermal state data meets the pre-adjusted thermal conditions, the transfer marking step generates a thermal load transfer mark and postpones the fan speed increment to the subsequent acoustically maskable heat dissipation window. This allows heat dissipation needs that are not currently suitable for increasing the fan speed to be recorded and processed later, rather than directly introducing wind noise by increasing the fan speed. The yield adjustment step reduces the thermal load of the non-lyrics display area based on the thermal load transfer mark while maintaining the readability of the lyrics display area. This allows thermal risk control to be initially handled by the brightness load of the background display area and the image enhancement load of the non-lyrics display area, while preventing the brightness or contrast of the lyrics display area from decreasing to the point of affecting singing along. Thus, the karaoke projection device can reduce the thermal load of the projection light source and processor without significantly damaging the lyrics display effect.

[0030] The template compensation step calls the wind noise template based on the fan speed feedback data and suppresses wind noise in the microphone's human voice audio signal, ensuring that the wind noise suppression target corresponds to the actual fan speed. This compensates for the wind noise characteristics formed by the heat dissipation execution unit at different fan speed levels. The closed-loop allocation step allocates the fan speed increment and the reduction in heat load in the non-lyrics display area for the next time window based on the wind noise suppression result and thermal state data, enabling subsequent control to dynamically adjust according to changes in residual wind noise and thermal risk. When the residual wind noise is high, the fan speed increment in the next time window can be reduced while the reduction in heat load in the non-lyrics display area can be increased; when the thermal risk continues to rise, the fan speed increment or the reduction in heat load in the non-lyrics display area can be increased. Through the above audio-visual-thermal collaborative control link, this invention can balance microphone voice clarity, lyrics display area readability, and overall heat dissipation stability during continuous karaoke projection. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the signal flow for the execution steps of the method of the present invention.

[0032] Figure 2 This is a schematic block diagram of the overall system structure of the present invention.

[0033] Figure 3 This is a schematic diagram illustrating the relationship between the acoustically maskable heat dissipation window and the closed-loop allocation of the present invention. Detailed Implementation

[0034] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0035] like Figures 1 to 3 As shown, this embodiment provides a method and system for coordinated audio-visual thermal control of a karaoke projection device. This karaoke projection device simultaneously performs microphone voice acquisition, accompaniment playback, lyrics display, projection display, light source driving, temperature acquisition, and heat dissipation control within the same device. Since increasing the fan speed increases the wind noise that the microphone can acquire, increasing the projection light source power and enhancing the image in the non-lyrics display area increases the thermal load on the optical engine and processor, and increasing the audio amplifier output power increases the audio amplifier temperature, this embodiment does not solely increase the fan speed based on temperature, nor does it solely decrease the projection brightness. Instead, it uses acoustically maskable heat dissipation windows, heat load transfer markers, heat load reduction in the non-lyrics display area, wind noise templates, and closed-loop allocation steps to ensure that fan cooling, projection display, and microphone voice processing are executed collaboratively within the same time window.

[0036] like Figure 1 As shown, Figure 1 This is a schematic diagram of the signal flow for the method execution steps. Figure 1 The method is executed by a collaborative control unit, which can be a software control unit formed by executing a computer program in memory in the processor of the karaoke projection device, or a control unit formed by at least one of a processor, a digital signal processor, and a microcontroller.

[0037] In the S101 data synchronization step, the collaborative control unit synchronously acquires microphone voice audio signals, accompaniment audio signals, lyrics timestamps, projection screen data, thermal status data, and fan speed feedback data. Microphone voice audio signals are acquired by the audio acquisition unit, while accompaniment audio signals can be provided by local accompaniment files, wireless playback devices, or karaoke applications. Lyric timestamps indicate the playback time corresponding to each line of lyrics, each word of lyrics, or the interval between lyric phrase transitions. Projection screen data includes lyrics display area identifiers, background display area identifiers, current screen brightness, current screen area division results, and current display mode. Thermal status data includes optical engine temperature, processor temperature, audio amplifier temperature, optical engine heating rate, processor heating rate, and audio amplifier heating rate. Fan speed feedback data is provided by the heat dissipation execution unit, indicating the current fan speed level or current fan speed value. All the above data are aligned using the same time window, which can be set from 20 milliseconds to 200 milliseconds depending on device performance, allowing audio changes, lyrics changes, screen changes, and temperature changes to be compared and processed within the same control cycle.

[0038] For ease of implementation, the sources and uses of various types of data in the data synchronization process can be configured as shown in Table 1.

[0039] Table 1. Data Sources and Purposes in the Data Synchronization Steps

[0040] In the S102 window determination step, the collaborative control unit determines whether the current time window is an acoustically maskable heat dissipation window based on the microphone vocal audio signal, the accompaniment audio signal, and the lyrics timestamp. An acoustically maskable heat dissipation window refers to a time window within which increasing the fan speed allows the fan noise to be masked by the accompaniment sound without significantly affecting the pickup of the singing vocals. Specifically, the collaborative control unit first extracts the vocal loudness, fundamental frequency stability, and pause duration from the microphone vocal audio signal, extracts the accompaniment loudness and accompaniment masking amount from the accompaniment audio signal, and identifies the lyric phrase transition interval based on the lyrics timestamp. If the accompaniment masking amount meets the accompaniment masking condition, and the vocal loudness, fundamental frequency stability, pause duration, or lyric phrase transition interval meets the vocal weakening condition, then the current time window is determined to be an acoustically maskable heat dissipation window.

[0041] The accompaniment masking amount can be calculated based on the energy proportion of the accompaniment audio signal within the wind noise frequency band. The wind noise frequency band can be predetermined based on the fan structure, duct structure, and microphone installation location of the karaoke projection equipment; for example, it can be set to the low-frequency and mid-low-frequency range where the main noise from the fan speed is located. One possible calculation method is as follows: ; Among them, the amount of accompaniment masking This indicates the masking ability of the accompaniment audio signal to wind noise, and represents the set of wind noise frequency bands. This indicates the preset main frequency band for wind noise, and the entire frequency band set. This represents all frequency bands involved in the analysis of the accompaniment audio signal; the energy of the accompaniment frequency bands. This indicates the frequency point of the accompaniment audio signal. Energy at that location, stability constant Used to avoid a denominator of 0. Accompaniment masking amount The larger the value, the more significant the energy of the accompaniment audio signal in the wind noise frequency band, and the easier it is for the wind turbine noise to be masked by the accompaniment sound.

[0042] Vocal fundamental frequency stability can be calculated based on the degree of change in the vocal fundamental frequency between adjacent time windows. The smaller the fundamental frequency change, the more stable the singing voice, and the more important the microphone signal. A larger fundamental frequency change, or the inability to stably detect the fundamental frequency, indicates that the singer may be in a pause, breathing phase, during an accompaniment interlude, or in a weak vocal state. One possible calculation method is as follows: ; Among them, the fundamental frequency stability of human voice This indicates the stability of the fundamental frequency of the human voice within the current time window; the current fundamental frequency... This indicates the fundamental frequency of the human voice detected in the current time window, and the previous fundamental frequency. This represents the fundamental frequency of the human voice detected in the previous time window, and its stability constant. Used to avoid a denominator of 0. When the fundamental frequency stability of human voice... When the frequency is below the fundamental frequency stability threshold, the current human voice is considered unstable or weakened, which is conducive to the release of the fan speed increment.

[0043] like Figure 1 As shown in the judgment diamond, after the window determination step, the collaborative control unit determines whether the current time window is an acoustically maskable heat dissipation window. If the judgment result is "no" and the thermal state data meets the pre-adjustment heat conditions, the S103 transfer mark step is executed; if the judgment result is "yes", the incremental release step can be entered.

[0044] In the S103 transfer marker step, when the current time window is not an acoustically maskable heat dissipation window and the thermal state data meets the pre-adjusted thermal conditions, the collaborative control unit generates a thermal load transfer marker and postpones the fan speed increment to the subsequent acoustically maskable heat dissipation window. The pre-adjusted thermal conditions can be that at least one of the optical engine temperature, processor temperature, and audio amplifier temperature reaches the corresponding pre-adjusted temperature threshold, or at least one of the optical engine heating rate, processor heating rate, and audio amplifier heating rate reaches the corresponding pre-adjusted heating rate threshold. The thermal load transfer marker is used to record the fan speed increment that was not executed in the current time window because it was not suitable to increase the fan speed, the amount of thermal load reduction in the non-lyrics display area that has been executed or is planned to be executed, the corresponding thermal state data, and the subsequent time window conditions to be postponed. By setting the thermal load transfer marker, the collaborative control unit can convert the contradiction of "currently needing heat dissipation but not suitable for increasing wind noise" into cross-time window processing, preventing a sudden increase in fan speed during periods when the user is singing or when voice pickup is sensitive.

[0045] In step S104, the collaborative control unit reduces the thermal load of the non-lyrics display area based on the thermal load transfer marker while maintaining the readability of the lyrics display area. The thermal load of the non-lyrics display area includes at least one of the following: background display area brightness load, non-lyrics display area dynamic contrast enhancement load, non-lyrics display area sharpening enhancement load, and non-lyrics display area super-resolution processing load. Specifically, the projected image data includes a lyrics display area identifier and a background display area identifier. The collaborative control unit locks the lyrics display area based on the lyrics display area identifier, determines the non-lyrics display area based on the background display area identifier, and performs brightness reduction or image enhancement degradation on the non-lyrics display area. Since the lyrics display area is highly important for the karaoke experience, when reducing the thermal load of the non-lyrics display area, the collaborative control unit maintains the brightness of the lyrics display area at no lower than the lyrics readability brightness threshold, or maintains the contrast of the lyrics display area at no lower than the lyrics readability contrast threshold. This reduces the load on the light source and processor while preventing the lyrics from suddenly becoming dark or unclear.

[0046] exist Figure 1 In the incremental release step shown in the dashed box on the right, when the subsequent time window is determined to be an acoustically maskable heat dissipation window, the co-control unit releases the unexecuted fan speed increment based on the heat load transfer flag. That is, if the current time window is not suitable for increasing the fan speed, the heat load in the non-lyrics display area is temporarily transferred first; when the subsequent accompaniment masking amount is high, the vocal volume is low, or there is a lyric phrase transition or a pause, the previously unexecuted fan speed increment is released, thereby completing the delayed heat dissipation.

[0047] In the S105 template compensation step, the collaborative control unit calls the wind noise template based on the fan speed feedback data and suppresses wind noise in the microphone voice audio signal. The wind noise template can be established during factory testing, initial startup, or user-triggered calibration of the karaoke projection device. The template establishment steps can be as follows: the control cooling execution unit sequentially outputs multiple fan speed levels; when the microphone voice input conditions are not met, the ambient noise audio signal is collected; and a wind noise template corresponding to the multiple fan speed levels is generated based on the ambient noise audio signal. During actual singing, the collaborative control unit or audio processing unit determines the current fan speed level based on the fan speed feedback data, calls the wind noise template corresponding to the current fan speed level, separates the wind noise component from the microphone voice audio signal based on the wind noise template, and outputs the wind noise-suppressed voice audio signal.

[0048] Wind noise suppression can be achieved by frequency domain subtraction, and one possible implementation method is as follows: ; Among them, the human voice audio signal after wind noise suppression Indicates at frequency point and time window The frequency domain signal after wind noise suppression is below, and the microphone voice audio signal is below. Indicates at frequency point and time window Microphone signal acquisition, wind noise template Indicates the fan speed setting Corresponding wind noise frequency domain template, suppression coefficient This indicates the intensity of wind noise template participation in the subtraction. To avoid excessive subtraction leading to vocal distortion, the suppression coefficient... It can be dynamically adjusted according to the amount of residual wind noise, the amount of accompaniment masking, and the loudness of human voices.

[0049] In the S106 closed-loop allocation step, the collaborative control unit allocates the fan speed increment and the reduction in thermal load in the non-lyrics display area for the next time window based on the wind noise suppression results and thermal status data. The wind noise suppression results may include residual wind noise, changes in vocal clarity, or the signal-to-noise ratio of the vocal audio signal after wind noise suppression. If the residual wind noise is higher than the residual wind noise threshold, it indicates that the current fan noise still significantly affects the microphone's vocal audio signal; therefore, the next time window should reduce the fan speed increment and increase the reduction in thermal load in the non-lyrics display area. If the thermal status data indicates that the thermal risk continues to increase, it means that the current concession amount alone is insufficient to reduce the thermal risk; therefore, the next time window should increase the fan speed increment or increase the reduction in thermal load in the non-lyrics display area. The closed-loop allocation step enables this embodiment to dynamically allocate control quantities between "acoustic influence" and "thermal risk," rather than using a fixed single control strategy.

[0050] The control quantity for the next time window in the closed-loop allocation step can be calculated as follows: ; Among them, the reduction in heat load in the non-lyrics display area This indicates the reduction in heat load in the non-lyrics display area for the next time window, and the current reduction in heat load in the non-lyrics display area. This indicates the reduction in heat load and thermal risk in the non-lyrics display area during the current time window. This indicates the degree of thermal risk obtained from thermal state data, and the residual wind noise level. This indicates the residual wind noise level obtained from the wind noise suppression results, with a thermal risk weight. This indicates the degree of impact of thermal risk on the reduction of thermal load in non-lyric display areas, with residual wind noise weighting. This indicates the extent to which residual wind noise affects the reduction of heat load in non-lyric display areas. In practical applications, if the thermal risk... If the temperature is increased, the reduction in heat load in the non-lyrics display area will be increased; if the residual wind noise is... An increase indicates that the fan increment should not be further expanded. Alternatively, the heat load reduction in the non-lyrics display area can be increased simultaneously to compensate for some of the fan cooling pressure.

[0051] The above Figure 1 In the illustrated process, the "No" branch corresponds to the case where the current time window is not an acoustically maskable heat dissipation window. In this case, the interference of fan noise on human voice pickup is reduced through the transfer marking step and the yield adjustment step. The "Yes" branch corresponds to the case where the current time window is an acoustically maskable heat dissipation window. In this case, the unexecuted fan speed increment can be released. Both branches eventually enter the template compensation step and the closed-loop allocation step, so that fan control, projection control, and audio processing form a continuous closed loop until the end of the current control cycle.

[0052] like Figure 2 As shown, Figure 2 This is a schematic block diagram of the overall system structure. The karaoke projection device includes a processor, memory, audio acquisition unit, audio processing unit, projection display unit, light source driving unit, temperature acquisition unit, heat dissipation unit, and collaborative control unit. The processor is connected to the memory, which stores the computer program and the fan noise template. When the processor executes the computer program, it forms the collaborative control unit or calls various modules within the collaborative control unit to execute. Figure 1 The method is shown. The collaborative control unit includes a data synchronization module, a window determination module, a transfer marking module, a transfer adjustment module, a template compensation module, and a closed-loop allocation module. Each module transmits control data sequentially and forms an input-output closed loop with the external unit.

[0053] The audio acquisition unit outputs microphone voice audio signals, accompaniment audio signals, and lyrics timestamps to the data synchronization module. In practice, the microphone voice audio signals can come from the device's built-in microphone or an external microphone, the accompaniment audio signals can come from Bluetooth, wireless network, local storage, or the audio processing unit, and the lyrics timestamps can be generated by the karaoke application after parsing the lyrics file. The projection display unit outputs projection screen data to the data synchronization module, which is used to distinguish between the lyrics display area and the non-lyrics display area. The temperature acquisition unit outputs thermal status data to the data synchronization module. The temperature acquisition unit can include temperature sensors located near the optical engine, processor, and audio amplifier. The heat dissipation execution unit outputs fan speed feedback data to the data synchronization module and receives fan speed increments output by the closed-loop distribution module. The template compensation module outputs the wind noise-suppressed voice audio signal to the audio processing unit, enabling the audio processing unit to continue performing voice enhancement, reverberation, volume equalization, or amplifier output. The closed-loop distribution module outputs the amount of heat load reduction in the non-lyrics display area to the projection display unit and the light source driving unit, so that the projection display unit reduces the image enhancement calculation load in the non-lyrics display area, or the light source driving unit reduces the brightness load corresponding to the non-lyrics display area.

[0054] Figure 2 In this context, the data synchronization module is used to complete... Figure 1 The data synchronization steps in the process. The window determination module is used to complete... Figure 1 The window determination step in the process. The transfer marker module is used to complete this. Figure 1 The system implements a transfer marker step and outputs a heat load transfer marker. The transfer adjustment module reduces the heat load on the non-lyrics display area while maintaining the readability of the lyrics display area based on the heat load transfer marker. The template compensation module calls a wind noise template based on fan speed feedback data and suppresses wind noise in the microphone's human voice audio signal. The closed-loop allocation module allocates the fan speed increment and the amount of heat load reduction in the non-lyrics display area for the next time window based on the wind noise suppression result and thermal state data. Through the above module configuration, the karaoke projection device can integrate audio acquisition, projection display, temperature acquisition, and fan control into a single control link, achieving coordinated audio-visual-thermal control without the need for additional complex mechanical structures.

[0055] To facilitate the following Figure 2 To establish system connectivity, the inputs, processing, and outputs of each module can be configured as shown in Table 2.

[0056] Table 2 Inputs, Processing, and Outputs of Each Module in the Cooperative Control Unit

[0057] like Figure 3 As shown, Figure 3 This is a schematic diagram showing the relationship between the acoustically maskable heat dissipation window and the closed-loop allocation. Figure 3 The control logic of this embodiment is explained using the coordinate relationship that varies with time windows. Figure 3 The upper curve represents the relationship between accompaniment masking and vocal loudness as a function of time windows. The areas marked by dashed lines or shaded areas represent acoustically maskable heat dissipation windows. These time windows are more likely to be identified as acoustically maskable heat dissipation windows when the accompaniment masking is high, the vocal loudness is low, or the lyrics timestamp indicates a lyric transition interval. Within these windows, the fan noise resulting from increased fan speed is more easily masked by the accompaniment audio signal and has less impact on microphone vocal pickup.

[0058] Figure 3 The curve in the middle represents the relationship between the fan speed increment and the time window. When the acoustically maskable heat dissipation window has not been entered and the thermal status data meets the pre-adjustment heating conditions, the cooperative control unit does not immediately release the fan speed increment, but instead generates a heat load transfer marker. Figure 3The heat load transfer marker in the middle indicates that the current time window records any unexecuted fan speed increments. When a subsequent time window is determined to be an acoustically maskable heat dissipation window, the co-control unit releases the unexecuted fan speed increments, causing the heat dissipation execution unit to increase the fan speed. This process avoids sudden increases in fan speed during times when the user is singing loudly or when voice pickup is sensitive.

[0059] Figure 3 The lower curve illustrates the relationship between the reduction in thermal load in the non-lyrics display area and the change over a time window. When the acoustically maskable heat dissipation window has not yet been entered, but thermal risk still needs to be controlled, the co-control unit increases the reduction in thermal load in the non-lyrics display area. This can be achieved by reducing the brightness of the background display area, reducing the dynamic contrast enhancement load, reducing the sharpening enhancement load, or reducing the super-resolution processing load in the non-lyrics display area. Once the subsequent acoustically maskable heat dissipation window is entered and the fan speed increment is released, the reduction in thermal load in the non-lyrics display area can be reduced accordingly to restore some of the background display effect. Figure 3 The wind noise residual amount and closed-loop allocation steps on the right indicate that when there is still a high residual wind noise after wind noise suppression, the collaborative control unit reduces the fan speed increment or increases the heat load reduction in the non-lyrics display area in the next time window; when the thermal status data still shows an increased thermal risk, the collaborative control unit increases the fan speed increment or increases the heat load reduction in the non-lyrics display area, thereby achieving a dynamic balance between audio quality and thermal protection.

[0060] pass Figure 3 As can be seen, the key to this embodiment is not simply increasing the fan speed when the temperature is high, nor is it simply reducing the projection brightness, but rather transferring and releasing heat at the time window level. When the current time window is not conducive to increasing the fan speed, the heat load is temporarily borne by the non-lyrics display area; when subsequent time windows have conditions for accompaniment masking and vocal weakening, the delayed fan speed increment is released. At the same time, the wind noise template compensates for the microphone vocal audio signal, and the residual wind noise is fed back into the closed-loop allocation step, so that the fan speed increment and the reduction of heat load in the non-lyrics display area in the next time window can be automatically adjusted according to the actual results.

[0061] In a complete operational example, the karaoke projection device plays the accompaniment and displays the lyrics while the user sings the verse. The collaborative control unit acquires the microphone vocal audio signal, accompaniment audio signal, lyrics timestamp, projection screen data, thermal status data, and fan speed feedback data through the data synchronization module. If the vocal volume is high and the fundamental frequency stability is high, but the optical engine temperature reaches the pre-adjusted temperature threshold, the window determination module determines that the current time window is not an acoustically maskable heat dissipation window. The transfer marker module generates a thermal load transfer marker and records the currently unexecuted fan speed increment. The yield adjustment module locks the lyrics display area based on the projection screen data and reduces the brightness of the background display area and the image enhancement computation load of the non-lyrics display area, while maintaining the brightness or contrast of the lyrics display area to meet readability requirements. When the song enters an accompaniment interlude or a lyric phrase change interval, the accompaniment masking increases and the vocal volume decreases. The window determination module determines the subsequent time window as an acoustically maskable heat dissipation window, and the collaborative control unit releases the unexecuted fan speed increment based on the thermal load transfer marker. As the fan speed increases, the template compensation module calls the corresponding wind noise template based on the fan speed feedback data to suppress wind noise in the microphone's human voice audio signal. The closed-loop distribution module determines whether to continue releasing the fan speed increment in the next time window or increase the heat load reduction in the non-lyrics display area based on the residual wind noise and thermal status data. Thus, the entire device achieves heat dissipation control without significantly compromising lyric readability or human voice pickup quality.

[0062] In the above embodiments, the acoustically maskable heat dissipation window, the heat load transfer marker, the heat load reduction in the non-lyrics display area, the wind noise template, and the closed-loop allocation steps work together to form a closed-loop control around the acoustics, image, and thermal state of the karaoke projection device. This closed-loop control can reduce the probability of fan noise entering the microphone's human voice audio signal, reduce the risk of lyrics becoming unreadable due to direct dimming of the entire image, and reduce the risk of overheating of the optical engine, processor, and audio amplifier during continuous karaoke use.

[0063] Finally, it should be noted that those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the protection scope of this invention.

Claims

1. A method for coordinated audio-visual and thermal control of a karaoke projection device, characterized in that, Includes the following steps: Data synchronization steps: Synchronously acquire microphone voice audio signal, accompaniment audio signal, lyrics timestamp, projection screen data, thermal status data and fan speed feedback data; Window determination step: Determine whether the current time window is an acoustically maskable heat dissipation window based on the microphone voice audio signal, the accompaniment audio signal and the lyrics timestamp; Transfer marker step: When the current time window is not the acoustically maskable heat dissipation window and the thermal state data meets the pre-heating conditions, a heat load transfer marker is generated, and the fan speed increment is postponed to the subsequent acoustically maskable heat dissipation window; The heat load transfer adjustment step is to reduce the heat load in the non-lyrics display area while maintaining the readability of the lyrics display area according to the heat load transfer marker. Template compensation steps: Based on the fan speed feedback data, call the wind noise template and suppress wind noise in the microphone human voice audio signal; Closed-loop allocation steps: Based on the wind noise suppression results and the thermal state data, allocate the fan speed increment and the reduction of thermal load in the non-lyric display area for the next time window.

2. The method for coordinated audio-visual and thermal control of a karaoke projection device according to claim 1, characterized in that, The window determination step includes: extracting the loudness, fundamental frequency stability, and pause duration of the human voice based on the microphone audio signal; extracting the loudness and masking amount of the accompaniment based on the accompaniment audio signal; identifying the lyric transition interval based on the lyric timestamp; and determining the lyric transition interval as the current time window when the accompaniment masking amount meets the accompaniment masking condition, and the loudness, fundamental frequency stability, pause duration, or lyric transition interval meets the human voice weakening condition. The vocal weakening conditions include: the vocal loudness is lower than the vocal loudness threshold, or the vocal fundamental frequency stability is lower than the fundamental frequency stability threshold, or the pause duration is higher than the pause duration threshold, or the lyrics timestamp indicates that the current time window is within the lyrics phrase change interval; The accompaniment masking conditions include: the accompaniment loudness is higher than the accompaniment loudness threshold, and the accompaniment masking amount is higher than the accompaniment masking amount threshold; the accompaniment masking amount is generated based on the energy proportion of the accompaniment audio signal in the wind noise frequency band.

3. The method for coordinated audio-visual and thermal control of a karaoke projection device according to claim 2, characterized in that, The transfer marking step includes: when the current time window is not the acoustically maskable heat dissipation window and the thermal state data meets the pre-heating conditions, recording the fan speed increment that has not been executed in the current time window, and writing the unexecuted fan speed increment into the thermal load transfer mark; The transfer marking step is followed by an incremental release step: when the subsequent time window is determined to be the acoustically maskable heat dissipation window, the unexecuted fan speed increment is released according to the heat load transfer mark; when the subsequent time window is still not the acoustically maskable heat dissipation window, the heat load of the non-lyrics display area continues to decrease. In the transfer adjustment step, the thermal load of the non-lyrics display area includes at least one of the following: background display area brightness load, non-lyrics display area dynamic contrast enhancement load, non-lyrics display area sharpening enhancement load, and non-lyrics display area super-resolution processing load. In the transfer adjustment step, maintaining the readability of the lyrics display area includes: maintaining the brightness of the lyrics display area at no less than the lyrics readability brightness threshold, or maintaining the contrast of the lyrics display area at no less than the lyrics readability contrast threshold.

4. The method for coordinated audio-visual and thermal control of a karaoke projection device according to claim 3, characterized in that, The projected image data includes a lyrics display area identifier and a background display area identifier; The transfer adjustment step includes: locking the lyrics display area according to the lyrics display area identifier, determining the non-lyrics display area according to the background display area identifier, and performing brightness reduction or image enhancement degradation on the non-lyrics display area; The thermal state data includes the optical engine temperature, processor temperature, audio amplifier temperature, optical engine heating rate, processor heating rate, and audio amplifier heating rate; the pre-adjustment thermal conditions include: at least one of the optical engine temperature, the processor temperature, and the audio amplifier temperature reaches the corresponding pre-adjustment temperature threshold, or at least one of the optical engine heating rate, the processor heating rate, and the audio amplifier heating rate reaches the corresponding pre-adjustment heating rate threshold.

5. The method for coordinated audio-visual and thermal control of a karaoke projection device according to claim 4, characterized in that, Before the template compensation step, there is also a template creation step: controlling the heat dissipation execution unit to output multiple fan speed levels in sequence, collecting ambient noise audio signals when the microphone voice input conditions are not met, and generating the wind noise template corresponding to the multiple fan speed levels based on the ambient noise audio signals. The template compensation step includes: determining the current fan speed level based on the fan speed feedback data, calling the wind noise template corresponding to the current fan speed level, separating the wind noise component from the microphone human voice audio signal based on the wind noise template, and outputting the wind noise suppressed human voice audio signal. The closed-loop allocation steps include: when the wind noise suppression result indicates that the residual wind noise is higher than the residual wind noise threshold, reducing the fan speed increment in the next time window and increasing the heat load reduction in the non-lyrics display area of ​​the next time window; when the thermal status data indicates that the thermal risk continues to rise, increasing the fan speed increment in the next time window or increasing the heat load reduction in the non-lyrics display area of ​​the next time window.

6. The method for coordinated audio-visual and thermal control of a karaoke projection device according to claim 5, characterized in that, Before the window determination step, a pre-heating step is also included: identifying the upcoming chorus time period based on the lyrics timestamp, determining the predicted vocal volume and predicted accompaniment volume corresponding to the chorus time period based on historical singing data, and releasing the delayed fan speed increment in the acoustically maskable heat dissipation window before the chorus time period, or reducing the heat load of the non-lyrics display area. The transfer adjustment step is followed by a power amplifier coordination step: when the thermal state data meets the power limiting condition, the upper limit of the audio power amplifier output power is reduced, and the voice frequency band clarity compensation is increased, so that the voice audio signal after wind noise suppression maintains the voice clarity after the upper limit of the audio power amplifier output power is reduced.

7. A sound-image-thermal collaborative control system for a karaoke projection device, characterized in that, include: The data synchronization module is used to synchronously acquire microphone voice audio signals, accompaniment audio signals, lyrics timestamps, projection screen data, thermal status data, and fan speed feedback data. The window determination module is used to determine whether the current time window is an acoustically maskable heat dissipation window based on the microphone voice audio signal, the accompaniment audio signal and the lyrics timestamp; The transfer marker module is used to generate a heat load transfer marker when the current time window is not the acoustically maskable heat dissipation window and the thermal state data meets the preheating conditions, and to postpone the fan speed increment to the subsequent acoustically maskable heat dissipation window. The heat transfer adjustment module is used to reduce the heat load in the non-lyrics display area and maintain the readability of the lyrics display area according to the heat load transfer mark. The template compensation module is used to call the wind noise template based on the fan speed feedback data and suppress wind noise in the microphone human voice audio signal; The closed-loop allocation module is used to allocate the fan speed increment and the reduction of heat load in the non-lyrics display area for the next time window based on the wind noise suppression result and the thermal state data.

8. The audio-visual thermal collaborative control system for the karaoke projection device according to claim 7, characterized in that, The window determination module includes a voice feature submodule, an accompaniment masking submodule, and a lyrics interval submodule; the voice feature submodule is used to extract voice loudness, voice fundamental frequency stability, and pause duration; the accompaniment masking submodule is used to extract accompaniment loudness and accompaniment masking amount; and the lyrics interval submodule is used to identify the lyric phrase change interval based on the lyric timestamp. The transfer marking module includes an incremental recording submodule and a delayed release submodule; the incremental recording submodule is used to record the fan speed increments that have not been executed in the current time window, and the delayed release submodule is used to release the unexecuted fan speed increments when a subsequent time window is determined to be the acoustically maskable heat dissipation window.

9. The audio-visual thermal collaborative control system for the karaoke projection device according to claim 8, characterized in that, The transfer adjustment module includes a region locking submodule, a brightness transfer submodule, and an image quality transfer submodule; The region locking submodule is used to lock the lyrics display area and the non-lyrics display area according to the projected image data; the brightness yielding submodule is used to reduce the brightness of the non-lyrics display area; and the image quality yielding submodule is used to reduce the image enhancement calculation amount of the non-lyrics display area. The template compensation module includes a template creation submodule and a template invocation submodule; The template creation submodule is used to create the wind noise template according to multiple fan speed levels, and the template calling submodule is used to call the corresponding wind noise template according to the fan speed feedback data and output the human voice audio signal after wind noise suppression.

10. A karaoke projection device, characterized in that, It includes a processor, a memory, an audio acquisition unit, an audio processing unit, a projection display unit, a light source driving unit, a temperature acquisition unit, and a heat dissipation execution unit; the memory stores a computer program, and when the processor executes the computer program, it implements the audio-visual-thermal coordinated control method of the karaoke projection device according to any one of claims 1 to 6.

Images