Ship shutter glass voice interaction and display cooperative control device and working method thereof

Abstract

The invention discloses a ship shutter glass voice interaction and display cooperative control device and a working method thereof, the cooperative control device comprises a power supply unit, a control unit and a voice processing unit, the voice processing unit comprises a voice collection module and a voice recognition module, and the voice recognition module has noise reduction and emotion recognition functions; and the control unit is electrically connected with the voice processing unit, the display unit and the sound unit. Cooperative control is realized through a ship scene exclusive voice model and emotion recognition; the noise reduction module greatly improves the voice recognition accuracy and adapts to the complex noise environment of the ship; the linkage adjustment of display and sound improves the user interaction experience, enriches the ship entertainment and information acquisition functions, and expands the product application scenarios.

Description

Technical Field

[0001] This invention belongs to the field of voice interaction technology, specifically relating to a voice interaction and display collaborative control device for ship glass and its working method. Background Technology

[0002] With the rapid development of technology, users have increasingly higher requirements for the performance and functions of marine louvered windows. The single function of sun shading can no longer meet the requirements of modern use.

[0003] Existing technologies include marine louvered windows that combine display and voice control devices. However, voice control for marine glass typically only achieves single-function switching (such as louver raising / lowering and brightness adjustment), lacking coordination with the display system. Furthermore, voice recognition is susceptible to interference from ship engine noise and wave noise, resulting in low accuracy. The interaction logic is also simple, failing to automatically match display content and sound effects based on user voice commands. Therefore, there is an urgent need to design a more intelligent voice interaction and display collaborative control device for ship louvered glass. Summary of the Invention

[0004] To address the above technical problems, this invention provides a voice interaction and display collaborative control device for ship louvered glass and its working method, which realizes intelligent linkage between the display device and voice commands, thereby improving the user experience.

[0005] The technical solution of the present invention is: a voice interaction and display collaborative control device for ship louvered glass, comprising a power supply unit, a control unit and a voice processing unit, wherein the voice processing unit comprises a voice collection module and a voice recognition module, and the voice recognition module has noise reduction and emotion recognition functions; It also includes a display unit and an audio unit, and the control unit is electrically connected to the voice processing unit, the display unit, and the audio unit.

[0006] Preferably, it also includes a storage unit and a wireless communication unit, wherein the storage unit is used to store an offline instruction library.

[0007] A method for operating a voice interaction and display collaborative control device for ship louvered glass is also provided, comprising the following steps: 1) Collect voice information and perform preprocessing; 2) Perform noise suppression and feature extraction on the preprocessed audio signal, and output the result to the feature matrix; 3) Identify instruction information and emotion information based on the feature matrix; 4) Execute instructions based on instruction information and emotional information.

[0008] Preferably, the specific steps of step 1) are as follows: after the voice recognition module detects a sound with a volume ≥ a preset volume value and a duration ≥ a preset duration value; The sound signal is amplified by a preset factor, the filter circuit filters out noise above the first preset frequency value, the ship noise within the second preset frequency value is initially suppressed, and the audio signal with a signal-to-noise ratio ≥ the first preset signal-to-noise ratio value is output. The first preset frequency value is greater than the second preset frequency value.

[0009] Preferably, the specific steps of step 2) are as follows: Noise suppression involves collecting ambient noise and generating inverse sound waves to cancel it out, thereby further improving the signal-to-noise ratio to ≥ a second preset signal-to-noise ratio value; the second preset signal-to-noise ratio value is greater than the first preset signal-to-noise ratio value. Feature extraction uses Mel frequency cepstral coefficients to extract speech features, and simultaneously collects pitch, speech rate, and volume data, which are then stored in the feature matrix.

[0010] Preferably, the specific steps of step 3) are as follows: The input feature matrix is ​​matched with the ship-specific instruction library, and the similarity is calculated using a dynamic time warping algorithm; If the similarity is greater than or equal to the first similarity preset value, it is determined to be a valid instruction and the instruction code is output. When the similarity is between the first and second preset similarity values, the display module will prompt the user to repeat the command. No response is given if the similarity is less than the third similarity preset value. First similarity preset value > Second similarity preset value > Third similarity preset value; Emotion recognition: Input the 3D features into a pre-trained supervised learning algorithm model, and output sentiment labels: When the pitch is within the first preset pitch range, the speech rate is within the first preset speech rate range, and the volume is within the first preset volume range, the output label is "pleasant". When the pitch is within the second preset pitch range, the speech rate is within the second preset speech rate range, and the volume is within the second preset volume range, the output label is "calm". When the pitch is within the third pitch preset range, the speech rate is within the third speech rate preset range, and the volume is within the third volume preset range, the output label is "irritable".

[0011] Preferably, the specific steps of step 4) are as follows: Control commands: Invoke the corresponding functional components, synchronously feed back the status to the display unit, output confirmation tone to the audio unit, and adjust the volume of the voice to match the mood; Content-related instructions: Prioritize reading content from local storage; if content is unavailable locally, call cloud resources to the display unit; adapt sound effects; and dynamically adjust volume according to ambient noise. Emergency commands: Force interrupt the current function, execute with the highest priority, set the display unit to full-screen warning with flashing at maximum brightness, and set the audio unit to emit an alarm sound.

[0012] Preferably, it also includes step 5) feedback and adaptive adjustment: Feedback includes: Display feedback: Real-time acquisition of ambient light, and adjustment of display unit brightness according to light intensity; Audio feedback: By collecting the actual volume of the speakers, corrections are made when the target volume difference is greater than or equal to the preset difference; Offline switching: Detects network connection, sets the offline flag when the network is disconnected, disables cloud interaction, and only calls the core instructions and basic content stored locally; Adaptive adjustment of logic from high to low priority, as follows: 1. Emergency commands will be enforced, interrupting all other functions; 2. Manual intervention commands cover both voice control and automatic adaptation; 3. Voice control commands will automatically adapt. 4. Automatic adaptation commands will be executed by default.

[0013] The beneficial effects of this invention are: collaborative control is achieved through a ship-specific voice model and emotion recognition; the noise reduction module greatly improves the accuracy of voice recognition, adapting to the complex noise environment of ships; the linkage adjustment of display and audio enhances the user interaction experience, enriches ship entertainment and information acquisition functions, and expands product application scenarios. Attached Figure Description

[0014] Figure 1 This is a circuit diagram of the control unit in an embodiment of the present invention. Figure 2 This is a circuit diagram of the voice acquisition module in the voice processing unit. Figure 3 This is a circuit diagram of the noise reduction module in the speech processing unit. Figure 4 This is a circuit diagram of the speech recognition and emotion recognition modules in the speech processing unit. Figure 5 This is a circuit diagram of the audio control unit. Figure 6 This is a circuit diagram of the charging circuit. Figure 7 This is a circuit diagram for voltage regulation. Figure 8 This is a circuit diagram of a storage unit. Figure 9 This is a circuit diagram of a wireless communication unit. Figure 10This is a circuit diagram for display. Figure 11 This is the circuit diagram of the louver drive unit. Detailed Implementation

[0015] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] A collaborative control device for voice interaction and display of ship louvered glass includes a power supply unit, a control unit and a voice processing unit. The voice processing unit includes a voice collection module and a voice recognition module. The voice recognition module has noise reduction and emotion recognition functions. It also includes a display unit and an audio unit, and the control unit is electrically connected to the voice processing unit, the display unit, and the audio unit.

[0017] In this embodiment, a storage unit and a wireless communication unit are also included, wherein the storage unit is used to store an offline instruction library.

[0018] This system is specifically designed for shipboard scenarios, with "voice interaction as the core and multi-device collaboration as the goal" to achieve fully automated control of the entire process of "voice command → intelligent decision-making → display / audio linkage", while also being compatible with manual intervention and offline operation.

[0019] A method for operating a voice interaction and display collaborative control device for ship louvered glass is also provided, comprising the following steps: 1) Collect voice information and perform preprocessing; 2) Perform noise suppression and feature extraction on the preprocessed audio signal, and output the result to the feature matrix; 3) Identify instruction information and emotion information based on the feature matrix; 4) Execute instructions based on instruction information and emotional information.

[0020] Preferably, the specific steps of step 1) are as follows: after the voice recognition module detects a sound with a volume ≥ a preset volume value and a duration ≥ a preset duration value; The sound signal is amplified by a preset factor, the filter circuit filters out noise above the first preset frequency value, the ship noise within the second preset frequency value is initially suppressed, and the audio signal with a signal-to-noise ratio ≥ the first preset signal-to-noise ratio value is output. The first preset frequency value is greater than the second preset frequency value.

[0021] Preferably, the specific steps of step 2) are as follows: Noise suppression involves collecting ambient noise and generating inverse sound waves to cancel it out, thereby further improving the signal-to-noise ratio to ≥ a second preset signal-to-noise ratio value; the second preset signal-to-noise ratio value is greater than the first preset signal-to-noise ratio value. Feature extraction uses Mel frequency cepstral coefficients to extract speech features, and simultaneously collects pitch, speech rate, and volume data, which are then stored in the feature matrix.

[0022] Preferably, the specific steps of step 3) are as follows: The input feature matrix is ​​matched with the ship-specific instruction library, and the similarity is calculated using a dynamic time warping algorithm; If the similarity is greater than or equal to the first similarity preset value, it is determined to be a valid instruction and the instruction code is output. When the similarity is between the first and second preset similarity values, the display module will prompt the user to repeat the command. No response is given if the similarity is less than the third similarity preset value. First similarity preset value > Second similarity preset value > Third similarity preset value; Emotion recognition: Input the 3D features into a pre-trained supervised learning algorithm model, and output sentiment labels: When the pitch is within the first preset pitch range, the speech rate is within the first preset speech rate range, and the volume is within the first preset volume range, the output label is "pleasant". When the pitch is within the second preset pitch range, the speech rate is within the second preset speech rate range, and the volume is within the second preset volume range, the output label is "calm". When the pitch is within the third pitch preset range, the speech rate is within the third speech rate preset range, and the volume is within the third volume preset range, the output label is "irritable".

[0023] Preferably, the specific steps of step 4) are as follows: Control commands: such as dimming the portholes, opening the blinds, calling the corresponding functional components, synchronously feeding back the status to the display unit, outputting a confirmation tone to the audio unit, and adjusting the volume of the voice to match the mood; Content-related commands, such as playing maritime news and displaying navigation maps, prioritize reading content from the local storage. If the content is not available locally, they will be retrieved from the cloud and sent to the display unit. The sound effects are adapted, and the volume is dynamically adjusted according to ambient noise. Emergency commands: such as emergency call, displaying fault information, forcibly interrupting the current function, executing with the highest priority, displaying a full-screen warning with flashing at maximum brightness, and emitting an alarm sound from the audio unit.

[0024] Preferably, it also includes step 5) feedback and adaptive adjustment: Feedback includes: Display feedback: Real-time acquisition of ambient light, and adjustment of display unit brightness according to light intensity; Audio feedback: By collecting the actual volume of the speakers, corrections are made when the target volume difference is greater than or equal to the preset difference; Offline switching: Detects network connection, sets the offline flag when the network is disconnected, disables cloud interaction, and only calls the core instructions and basic content stored locally; Adaptive adjustment of logic from high to low priority, as follows: 1. Emergency commands will be enforced, interrupting all other functions; 2. Manual intervention commands cover both voice control and automatic adaptation; 3. Voice control commands will automatically adapt. 4. Automatic adaptation commands will be executed by default.

[0025] Example 1 Hardware components: Speech processing unit Voice collection module: Multi-microphone array (SPH0641LM4H); Voice processing module: S810 AI voice module (noise reduction and emotion recognition); Display unit: Flexible AMOLED display film; Audio unit: Amplifier module + speaker Storage unit: Local Flash (offline instruction library + display resources) Wireless communication unit: WBR2 (Wifi+BLE) module Control unit: STM32F103C8T6 main controller (cooperative control logic core), display driver, dynamic resolution adjustment.

[0026] The connection method is a standard technique and will not be described in detail further. Please refer to the circuit diagram. Figures 1 to 11 .

[0027] All signal links adopt RC filtering + TVS diode lightning protection design; PCB layout adopts "single-point grounding + electromagnetic shielding strip" to resist electromagnetic interference from ships; core modules (S810, STM32) are equipped with backup power supply channels to avoid functional interruption caused by voltage fluctuations.

[0028] All modules are connected in parallel with a 1000μF electrolytic capacitor and a 100nF ceramic capacitor to filter out voltage ripple; the PCB design includes a reserved mounting position for an electromagnetic shield to protect against electromagnetic interference from ship radar and communication equipment.

[0029] Reliability: All core components are industrial-grade wide-temperature models, supporting operation from -40 to +85℃; fuses and TVS diodes are added to the circuit to resist overcurrent, overvoltage and lightning strike risks.

[0030] Working method of the voice interaction and display collaborative control device for ship louvered glass Step 1) Detect volume ≥ 50dB (voice threshold) for duration ≥ 200ms (to avoid false triggering); Processing logic: The LM386 amplifies the weak microphone signal by 40 times, the RC filter circuit filters out high-frequency noise above 2kHz, and the TPA3118D2 performs preliminary suppression of ship noise from 50-2000Hz, outputting an audio signal with a signal-to-noise ratio ≥30dB.

[0031] Step 2) The S810 module runs an adaptive noise cancellation algorithm (ANC): it collects ambient noise through a reference microphone, generates an inverse sound wave to cancel the noise, and further improves the signal-to-noise ratio to ≥35dB; Feature extraction: Speech features are extracted using Mel-frequency cepstral coefficients (MFCC), and pitch (100-500Hz), speech rate (1-5 words / second), and volume (50-90dB) data are collected simultaneously and stored in the feature matrix.

[0032] Step 3) 1. Command recognition: The input feature matrix is ​​matched with the ship-specific command library (local and cloud), and the similarity is calculated using the Dynamic Time Warping (DTW) algorithm; When the similarity is ≥85%, it is determined to be a valid instruction and the instruction code is output (e.g., "dimming" = 0x01, "play music" = 0x02, "emergency call" = 0x0A). When the similarity is between 60% and 84%, the display module will prompt "Please repeat the instruction"; No response when similarity is less than 60%.

[0033] Emotion recognition: Input the 3D features into a pre-trained SVM model to output sentiment labels: Pleasant tone: 300-500Hz + speech rate 2-3 words / second + volume 60-70dB; Calm: Tone 200-300Hz + speaking speed 1.5-2.5 words / second + volume 50-60dB; Irritability: Tone 100-200Hz + Speech rate 3-5 words / second + Volume 70-90dB.

[0034] Step 4) Instruction and Emotional Information: See the table below for execution instructions: Step 5) Feedback and Adaptive Adjustment (Continuous Operation) Display feedback: TSL2561 collects ambient light in real time (1-65535 lux), and STM32 corrects the display brightness according to the light intensity (brightness 400 cd / ㎡ when strong light ≥ 5000 lux, brightness 200 cd / ㎡ when weak light ≤ 2000 lux). Audio feedback: The volume feedback circuit acquires the actual volume of the speaker through an ADC. When the difference between the actual volume and the target volume is ≥10%, it corrects it through a PWM signal. Offline switching: The WBR2 module checks the network connection every 3 seconds. When the network is disconnected, the STM32 sets the offline flag, disables cloud interaction, and only calls the 100+ core instructions and basic content stored locally on the W25Q64.

[0035] (ii) Priority logic (from high to low) Emergency commands (such as "emergency call" and "fault alarm") are enforced, interrupting all other functions; Manual intervention commands (button / wireless remote control) cover both voice control and automatic adaptation; Voice control commands are automatically adapted. Automatic adaptation commands (display / sound adjustment based on ambient light / noise) are enabled by default.

[0036] This device integrates a noise reduction module and a voice model for ship scenarios, which can filter environmental noise such as engine noise and sea waves, and accurately identify control commands and content request commands. The display control unit and the voice processing unit communicate bidirectionally. After receiving content request commands, the display control unit matches the corresponding display content from local storage and cloud database through the content retrieval module, and adjusts the display color and brightness according to the voice emotion recognition results. The audio control unit works in conjunction with the display control unit to automatically adjust the speaker sound effect parameters (such as volume and equalizer mode) according to the type of display content (such as music, video, news). It also includes an offline command library to ensure basic interactive functions in environments without network access.

[0037] This invention is not limited to the above embodiments. Based on the technical solutions disclosed in this invention, those skilled in the art can make some substitutions and modifications to some of the technical features without creative effort, and all such substitutions and modifications are within the protection scope of this invention.

Claims

1. A voice interaction and display collaborative control device for ship louvered glass, comprising a power supply unit, a control unit, and a voice processing unit, characterized in that, The voice processing unit includes a voice collection module and a voice recognition module, and the voice recognition module has noise reduction and emotion recognition functions. It also includes a display unit and an audio unit, and the control unit is electrically connected to the voice processing unit, the display unit, and the audio unit.

2. The voice interaction and display collaborative control device for ship louvered glass according to claim 1, characterized in that, It also includes a storage unit and a wireless communication unit, wherein the storage unit is used to store an offline instruction library.

3. A method for operating a voice interaction and display collaborative control device for ship louvered glass, characterized in that, Includes the following steps: 1) Collect voice information and perform preprocessing; 2) Perform noise suppression and feature extraction on the preprocessed audio signal, and output the result to the feature matrix; 3) Identify instruction information and emotion information based on the feature matrix; 4) Execute instructions based on instruction information and emotional information.

4. The working method of the ship louvered glass voice interaction and display collaborative control device according to claim 3, characterized in that, Step 1) Specific steps: After the voice recognition module detects a sound with a volume ≥ volume preset value and a duration ≥ duration preset value; The sound signal is amplified by a preset factor, the filter circuit filters out noise above the first preset frequency value, the ship noise within the second preset frequency value is initially suppressed, and the audio signal with a signal-to-noise ratio ≥ the first preset signal-to-noise ratio value is output. The first preset frequency value is greater than the second preset frequency value.

5. The working method of the ship louvered glass voice interaction and display collaborative control device according to claim 4, characterized in that, The specific steps for step 2) are as follows: Noise suppression involves collecting ambient noise and generating inverse sound waves to cancel it out, thereby further improving the signal-to-noise ratio to ≥ a second preset signal-to-noise ratio value; the second preset signal-to-noise ratio value is greater than the first preset signal-to-noise ratio value. Feature extraction uses Mel frequency cepstral coefficients to extract speech features, and simultaneously collects pitch, speech rate, and volume data, which are then stored in the feature matrix.

6. The working method of the ship louvered glass voice interaction and display collaborative control device according to claim 5, characterized in that, The specific steps for step 3) are as follows: The input feature matrix is ​​matched with the ship-specific instruction library, and the similarity is calculated using a dynamic time warping algorithm; If the similarity is greater than or equal to the first similarity preset value, it is determined to be a valid instruction and the instruction code is output. When the similarity is between the first and second preset similarity values, the display module will prompt "Please repeat the command". No response is given if the similarity is less than the third similarity preset value. First similarity preset value > Second similarity preset value > Third similarity preset value; Emotion recognition: Input the 3D features into a pre-trained supervised learning algorithm model, and output sentiment labels: When the pitch is within the first preset pitch range, the speech rate is within the first preset speech rate range, and the volume is within the first preset volume range, the output label is "pleasant". When the pitch is within the second preset pitch range, the speech rate is within the second preset speech rate range, and the volume is within the second preset volume range, the output label is "calm". When the pitch is within the third pitch preset range, the speech rate is within the third speech rate preset range, and the volume is within the third volume preset range, the output label is "irritable".

7. The working method of the ship louvered glass voice interaction and display collaborative control device according to claim 6, characterized in that, The specific steps for step 4) are as follows: Control commands: Invoke the corresponding functional components, synchronously feed back the status to the display unit, output confirmation tone to the audio unit, and adjust the volume of the voice to match the mood; Content-related instructions: Prioritize reading content from local storage; if content is unavailable locally, call cloud resources to the display unit; adapt sound effects; and dynamically adjust volume according to ambient noise. Emergency commands: Force interrupt the current function, execute with the highest priority, set the display unit to full-screen warning with flashing at maximum brightness, and set the audio unit to emit an alarm sound.

8. The working method of the ship louvered glass voice interaction and display collaborative control device according to claim 3, characterized in that, It also includes step 5) Feedback and adaptive adjustment: Feedback includes: Display feedback: Real-time acquisition of ambient light, and adjustment of display unit brightness according to light intensity; Audio feedback: By collecting the actual volume of the speakers, corrections are made when the target volume difference is greater than or equal to the preset difference; Offline switching: Detects network connection, sets the offline flag when the network is disconnected, disables cloud interaction, and only calls the core instructions and basic content stored locally; Adaptive adjustment of logic from high to low priority, as follows:

1. Emergency commands will be enforced, interrupting all other functions; 2. Manual intervention commands cover both voice control and automatic adaptation; 3. Voice control commands will automatically adapt.

4. Automatic adaptation commands will be executed by default.

Images