Headset with lip reading function

Abstract

This invention provides a noise-canceling headset with lip-reading functionality, comprising earcups, an active noise-canceling unit, and a microphone. A miniature camera is fixedly mounted on the microphone housing to capture real-time images of the user's dynamic lip movements. A data processing unit, physically connected to the camera, receives the images and converts them into digital speech signals using a lip-reading algorithm. A speech synthesis unit, electrically connected to the data processing unit, converts the digital signals into output synthesized speech signals. This device, through the coordinated processing of video and audio, replaces traditional microphone input in noisy environments, significantly improving voice communication quality. This invention can improve voice communication quality and support silent information transmission in quiet states.

Description

Technical Field

[0001] This invention relates to the field of intelligent audio device technology, and more specifically, to a noise-canceling headset with lip-reading recognition function. Background Technology

[0002] Existing noise-canceling headsets are widely used in both civilian and military fields. Many products feature active noise cancellation, and some even achieve dual-mode noise cancellation, meaning active noise cancellation is present on both the receiver and transmitter. Through a combination of active and passive noise cancellation technologies, these headsets can achieve a combined noise reduction capability of over 30dB, covering a frequency range of 50Hz to 8000Hz, meeting the national standard GB / T1 / 3 octave band test requirements. Active noise cancellation technology primarily works by using a microphone to collect ambient noise, which is then processed digitally to generate an inverse sound wave that cancels out the ambient noise, thus achieving noise reduction.

[0003] However, in real-world, high-noise environments, such as airport ground staff scenarios, when the ambient noise level exceeds 130dB, existing active noise-canceling headsets struggle to guarantee voice communication quality due to the sound masking effect. Especially at the microphone end, intense noise can drown out the user's voice output, making effective communication impossible. Furthermore, in certain special environments, such as places requiring silence, users may be unable to communicate via voice, and existing headsets cannot meet the needs of such silent communication.

[0004] In the process of implementing the embodiments of the present invention, the inventors discovered that the prior art has at least the following problems or defects: in high-intensity noise environments, existing headband noise-canceling headsets cannot effectively guarantee the quality of voice communication; in special environments where quietness is required, users cannot conduct silent communication through existing headsets. Summary of the Invention

[0005] This invention provides a noise-canceling headset with lip-reading recognition function, comprising: The earcups, active noise cancellation unit, and microphone are characterized in that they further include: A miniature camera is fixedly mounted on the surface of the microphone housing to capture images of the user's dynamic lip movements in real time. The data processing unit is connected to the miniature camera via a physical link, and is used to receive the dynamic change image of the lips, and convert the image data into digital speech signals through a lip reading algorithm; The speech synthesis unit, electrically connected to the data processing unit, is used to convert the digitized speech signal into an outputtable synthesized speech signal.

[0006] Furthermore, the field of view of the miniature camera is 80° to 100°, and its installation position is aligned with the pickup direction of the microphone to ensure that the dynamic changes of the lips are synchronized with the voice input.

[0007] Furthermore, the data processing unit includes: A digital signal processor (DSP) is used to perform frame segmentation, sampling, and feature extraction on the dynamic lip change image; The codec chip is used to match the extracted lip feature data with a pre-stored lip reading database and generate the corresponding digital speech signal.

[0008] Furthermore, the lip-reading algorithm is the YOLOv5 algorithm, which includes an input terminal, a Backbone network, a Neck network, and a Prediction network, wherein: The Backbone network is used to extract multi-scale features from images of dynamic changes in the lips. The Neck network is used to fuse features at different scales; The Prediction network is used to output the predicted speech content corresponding to lip movements.

[0009] Furthermore, it also includes a power processing module, which is integrated inside the earcup and connected to the data processing unit, miniature camera and voice synthesis unit, and achieves low-power power management through dynamic voltage regulation technology.

[0010] Furthermore, the outer surface of the left earcup is provided with a mode switching button, which is used to control the activation or deactivation of the active noise cancellation function and the lip reading function.

[0011] Furthermore, the physical link includes a shielded video signal transmission line and an independent power supply line to avoid signal interference and ensure the stable operation of the miniature camera.

[0012] Furthermore, the processing flow for the dynamic lip change image specifically includes: The continuously acquired images of dynamic changes in the lips are processed in frames, with each frame sampled at 30fps. The feature data of lip contour, motion trajectory and shape changes are extracted using a digital signal processor; The feature data is input into the codec chip and matched with standard lip movements in the pre-stored lip reading database to generate a digital voice signal.

[0013] Furthermore, when the ambient noise level exceeds 130dB, the lip-reading function is automatically activated to replace the microphone for voice input, ensuring the quality of voice communication.

[0014] Furthermore, the active noise reduction unit has a comprehensive noise reduction capability of over 30dB, and the noise reduction frequency band covers 50Hz to 8000Hz, meeting the national standard GB / T1 / 3 octave band test requirements.

[0015] The embodiments of the present invention have at least the following beneficial effects: The noise-canceling headset of the present invention, by adding a miniature camera and a data processing unit to the microphone, can convert the dynamic changes in the user's lip movements into digital speech signals and output them, thereby effectively improving the quality of voice communication in high-intensity noise environments and ensuring that users can still clearly communicate in scenarios with noise levels exceeding 130dB. Simultaneously, the headset can achieve silent communication in quiet environments, allowing users to accurately transmit information through lip movements without uttering a sound, meeting communication needs in special scenarios.

[0016] Furthermore, the power processing module of the headset of this invention adopts dynamic voltage regulation technology, which can realize low power consumption management and extend the usage time of the device; its physical link design includes shielded video signal transmission line and independent power supply line, which can avoid signal interference, ensure the stable operation of miniature camera, and further improve the reliability and practicality of the device. Attached Figure Description

[0017] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated in the drawings by way of example and not limitation, wherein: Figure 1 This is a schematic diagram of the module structure of a noise-canceling headset with lip-reading recognition function according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a lip-reading recognition flowchart with lip-reading recognition function provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the physical link of a headset provided in an embodiment of the present invention. Detailed Implementation

[0018] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided to make the invention more thorough and complete, and to fully convey the scope of the invention to those skilled in the art.

[0019] Those skilled in the art will recognize that embodiments of the present invention can be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present invention can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0020] It should be noted that the number of any elements in the accompanying drawings is for illustrative purposes only and not as a limitation, and any naming is for distinction only and has no limiting meaning.

[0021] The following is for reference. Figure 1 , Figure 1 This is a schematic diagram of a noise-canceling headset with lip-reading recognition function provided in an embodiment of the present invention. Figure 1 As shown, a noise-canceling headset with lip-reading recognition function includes: The earcups 101, active noise cancellation unit 102, and microphone 103 also include: A miniature camera is fixedly mounted on the surface of the microphone housing to capture images of the user's dynamic lip movements in real time. The data processing unit is connected to the miniature camera via a physical link, and is used to receive the dynamic change image of the lips, and convert the image data into digital speech signals through a lip reading algorithm; The speech synthesis unit, electrically connected to the data processing unit, is used to convert the digitized speech signal into an outputtable synthesized speech signal.

[0022] It should be noted that the noise-canceling headset of this invention adds a miniature camera and a data processing unit to the traditional headset. This allows for real-time acquisition of images of the user's dynamic lip movements, and the conversion of the image data into digital speech signals using a lip-reading algorithm. The miniature camera is a small optical imaging device mounted on the surface of the microphone housing, used to capture continuous images of the user's lips. The data processing unit is an electronic module containing a digital signal processor (DSP) and a codec chip, responsible for processing and analyzing the acquired images to ultimately generate a speech signal. This design enables the headset to provide effective voice communication in noisy environments or in silent conditions.

[0023] Specifically, the miniature camera has a field of view of 80° to 100°, ensuring that it covers the user's entire lip area while avoiding unnecessary background interference. Its installation position is aligned with the microphone's pickup direction, guaranteeing that the dynamic changes in the lips are synchronized with the speech input, thus improving the accuracy and naturalness of voice communication. The DSP in the data processing unit is responsible for framing, sampling, and feature extraction of the acquired images, such as extracting lip contours, movement trajectories, and shape changes. The codec chip then matches these feature data with a pre-stored lip-reading database to generate the corresponding digitized speech signal. The lip-reading algorithm uses the YOLOv5 algorithm, which consists of an input layer, a Backbone network, a Neck network, and a Prediction network. The Backbone network extracts multi-scale features, the Neck network fuses features from different scales, and the Prediction network outputs the predicted speech content corresponding to the lip movements.

[0024] Preferably, the lip-reading function can be manually controlled via a mode switching button on the outer surface of the earcup, or it can be automatically activated based on the ambient noise level. When the ambient noise level exceeds 130dB, the lip-reading function automatically starts, replacing the microphone for voice input to ensure the quality of voice communication. During data processing, continuously acquired images of dynamic lip changes are processed in frames at a sampling rate of 30 frames per second (30fps) to ensure the continuity and integrity of the image data. Lip feature data extracted by the DSP, such as lip contours, movement trajectories, and shape changes, are input into the codec chip and precisely matched with standard lip movements in the pre-stored lip-reading database to generate accurate digital voice signals. This automated processing not only improves communication efficiency but also enhances the device's adaptability in complex environments.

[0025] In some embodiments, the field of view of the miniature camera is 80° to 100°, and its installation position is aligned with the pickup direction of the microphone to ensure that the dynamic changes in lip movements are synchronized with the voice input.

[0026] It's important to note that the field of view of the miniature camera is set to 80° to 100°. This range ensures the camera can accurately capture the dynamic changes in the user's lips while minimizing background interference, thus improving the accuracy of lip reading. The field of view refers to the spatial range that the camera can observe; its size directly affects the camera's coverage of the target object and its ability to capture details. Aligning the installation position with the microphone's pickup direction means that the camera and microphone are spatially coordinated, ensuring temporal consistency between the dynamic changes in the lips and the voice input. This is crucial for the naturalness and accuracy of voice communication.

[0027] Specifically, the field of view is set based on the opening and closing angle of the mouth during normal communication and the relative position of the camera to the mouth. An 80° to 100° field of view ensures that the camera can fully capture the dynamic changes of the lips in various head postures. The microphone's pickup direction refers to the direction in which the microphone is most sensitive to sound, usually consistent with the direction of the user's mouth. The alignment of the camera and microphone involves not only physical alignment but also time synchronization during operation to ensure the matching degree of image and sound. This design allows the headset to accurately reflect the user's speech intentions even in noisy environments, even if the microphone's voice signal is interfered with, and thus convert it into a clear voice signal output through lip reading algorithms.

[0028] Preferably, to further improve the accuracy and efficiency of lip reading, the miniature camera can be a model with autofocus to adapt to changes in facial features and viewing distances of different users. The autofocus function automatically adjusts the focus based on the distance between the lips and the camera, ensuring the lip image is always clear and discernible. Furthermore, the camera's frame rate can be optimized according to the actual usage scenario; for example, the frame rate can be appropriately reduced in quiet environments to save power, while in noisy environments, the frame rate can be increased to capture more subtle lip movements. In the data processing unit, the DSP can preprocess the acquired images, such as denoising and contrast enhancement, to improve image quality and thus enhance the performance of the lip reading algorithm. These optimization measures work together to ensure that the headset provides stable and reliable lip reading functionality in various complex environments.

[0029] In some embodiments, the data processing unit includes: A digital signal processor (DSP) is used to perform frame segmentation, sampling, and feature extraction on the dynamic lip change image; The codec chip is used to match the extracted lip feature data with a pre-stored lip reading database and generate the corresponding digital speech signal.

[0030] It should be noted that the data processing unit is the core of this invention. It includes a digital signal processor (DSP) and a codec, used to process the dynamic lip movement images captured by the miniature camera and convert them into digital speech signals. The DSP is a microprocessor specifically designed for processing digital signals, capable of efficiently framing, sampling, and extracting features from image data. The codec is responsible for matching the extracted feature data with a pre-stored lip-reading database and generating the corresponding digital speech signal. This design enables the headset to achieve clear voice communication through lip-reading technology in noisy environments or silent conditions.

[0031] Specifically, the Digital Signal Processor (DSP) in the data processing unit is responsible for the initial processing of image data. It segments continuously acquired images of dynamic lip movements into frames, typically operating at a sampling rate of 30 frames per second (30fps) to ensure the continuity and integrity of the image data. The DSP extracts feature data such as lip contours, movement trajectories, and shape changes using specific algorithms; this feature data forms the basis for subsequent speech synthesis. The codec chip then uses this feature data to match it with a pre-stored lip-reading database. The lip-reading database is a collection containing various standard lip movements and their corresponding speech signals. The codec generates the corresponding digitized speech signal by searching and matching these data. This matching process is based on the mapping relationship between image features and speech signals, ensuring the accuracy and reliability of lip-reading recognition.

[0032] Preferably, to further improve the accuracy and efficiency of lip reading recognition, the DSP can employ advanced image processing algorithms, such as edge detection and morphological analysis, to extract lip features more accurately. The codec chip can incorporate machine learning algorithms during the matching process, continuously optimizing the matching model through learning and training on a large amount of lip reading data to improve matching accuracy. Furthermore, the data processing unit can be equipped with a feedback mechanism to monitor the accuracy and reliability of the recognition results in real time and dynamically adjust processing parameters based on the feedback results to adapt to different usage scenarios and user needs. These optimization measures work together to ensure that the headset provides stable and reliable lip reading recognition functionality in various complex environments.

[0033] In some embodiments, the lip-reading algorithm is the YOLOv5 algorithm, which includes an input terminal, a Backbone network, a Neck network, and a Prediction network, wherein: The Backbone network is used to extract multi-scale features from images of dynamic changes in the lips. The Neck network is used to fuse features at different scales; The Prediction network is used to output the predicted speech content corresponding to lip movements.

[0034] It's worth noting that the lip-reading algorithm uses the YOLOv5 algorithm, an advanced object detection algorithm widely used in image recognition. The YOLOv5 algorithm consists of four parts: an input network, a Backbone network, a Neck network, and a Prediction network. The Backbone network extracts multi-scale features from the dynamic lip movement images, the Neck network fuses features from different scales, and the Prediction network outputs the predicted speech content corresponding to the lip movements. This algorithm structure enables the headset to quickly and accurately recognize lip movements in real-time captured lip images and convert them into speech signals, thus achieving efficient voice communication.

[0035] Specifically, the YOLOv5 algorithm receives images of dynamic lip movements from a miniature camera as input. These images are input as consecutive frames, typically acquired at a sampling rate of 30 frames per second (30fps). The Backbone network, a deep convolutional neural network, extracts multi-scale features from the images through multiple convolutional operations. These features include the lip contour, shape changes, and motion trajectory. The Neck network further fuses and processes these features to enhance their expressive power. The Prediction network ultimately generates speech content predictions corresponding to the lip movements based on the fused features. These predictions are then matched with a pre-stored lip-reading database using a codec chip in the data processing unit to generate the final digitized speech signal. This hierarchical processing structure not only improves recognition accuracy but also ensures high efficiency in the processing.

[0036] Preferably, to further improve the accuracy and real-time performance of lip reading recognition, the Backbone network of the YOLOv5 algorithm can employ pre-trained models. These models, trained on large-scale image datasets, are better able to extract lip features. In practical applications, the Backbone network can be fine-tuned to adapt to specific lip reading tasks. The Neck network can incorporate an attention mechanism to highlight important features through weighted summaries, thereby improving feature fusion performance. The output of the Prediction network can be optimized through post-processing steps, such as removing duplicate prediction boxes using Non-Maximum Suppression (NMS) to ensure the accuracy of the final speech content. Furthermore, the data processing unit can be equipped with a real-time feedback mechanism to dynamically adjust the algorithm's parameters to adapt to different usage scenarios and user needs. These optimization measures work together to ensure that the headset provides stable and reliable lip reading recognition functionality in various complex environments.

[0037] In some embodiments, a power processing module is also included, which is integrated inside the earcup and connected to the data processing unit, the miniature camera and the speech synthesis unit, and achieves low-power power management through dynamic voltage regulation technology.

[0038] It's worth noting that the power management module is a crucial component of the headset. Integrated within the earcups, it connects to the data processing unit, miniature camera, and speech synthesis unit, employing dynamic voltage regulation technology for low-power power management. Dynamic voltage regulation is a technology that dynamically adjusts the supply voltage based on the device's actual workload, thereby minimizing power consumption while ensuring normal operation. This design is particularly important for over-ear noise-canceling headsets, as they need to maintain stable performance during extended use while reducing power consumption and extending the device's lifespan.

[0039] Specifically, the power processing module is integrated inside the earcups. This is to fully utilize the internal space of the headset, avoid external interference, and ensure stable and reliable connections with various functional modules. The core of dynamic voltage regulation technology lies in adjusting the supply voltage in real time according to the device's operating status. For example, it provides a higher voltage when the data processing unit is performing high-intensity calculations, and reduces the voltage during standby or low-load states, thereby achieving energy savings. This technology relies on a power management chip (PMIC), which monitors the device's current and voltage requirements and dynamically adjusts the output voltage through internal voltage regulation circuits and control algorithms. The low-power design of the power processing module not only helps extend battery life but also reduces device heat generation, improving user comfort.

[0040] Preferably, the power processing module can employ advanced power management chips with high-efficiency voltage and current regulation capabilities, enabling rapid voltage adjustment in different operating modes. For example, a power management chip with multi-mode switching can be used, automatically switching to the most energy-efficient mode under different operating conditions. Furthermore, the power processing module can be equipped with an intelligent power management system that automatically optimizes power distribution by monitoring the device's real-time power consumption and battery status. For instance, when low battery power is detected, the system can automatically reduce the power consumption of non-critical functional modules, prioritizing the normal operation of core functions such as lip reading and speech synthesis. This intelligent power management strategy not only improves the device's energy efficiency but also enhances its availability under low battery conditions, ensuring users can still use the headset for voice communication in critical moments.

[0041] In some embodiments, the outer surface of the left earcup of the earcup is provided with a mode switching button, which is used to control the activation or deactivation of the active noise cancellation function and the lip reading function.

[0042] It's worth noting that the left earcup features a mode switch button on its outer surface. This button controls whether the active noise cancellation and lip-reading functions are enabled or disabled. This design allows users to flexibly switch the headset's function modes according to actual usage scenarios to adapt to different environmental needs. For example, in quiet environments, users can choose to disable lip-reading to save power; while in noisy environments, users can enable lip-reading to ensure clear voice communication.

[0043] Specifically, the mode switch button is a physical button installed on the outer surface of the left earcup for easy user operation. It connects to the headset's control circuitry, controlling the activation or deactivation of the active noise cancellation unit and the lip-reading unit via circuit signals. Active noise cancellation refers to the headset using a microphone to collect ambient noise and generating inverse sound waves through digital signal processing to cancel out the noise. Lip-reading, on the other hand, uses a miniature camera to capture dynamic images of lip movements and converts them into speech signals through a data processing unit. The mode switch button allows users to quickly switch between the two functions to suit different usage scenarios.

[0044] Preferably, the mode switch button can be designed as a multi-functional button, such as long press and short press to switch between different functions. A long press switches the overall operating mode of the headset, such as from normal mode to high noise mode; a short press switches between enabling and disabling active noise cancellation and lip reading within the same mode. Furthermore, the button can be equipped with indicator lights that use different colors or flashing patterns to display the current function status, such as green indicating active noise cancellation is enabled and red indicating lip reading is enabled. This design not only improves user convenience but also enhances the device's interactivity and user experience.

[0045] In some embodiments, the physical link includes a shielded video signal transmission line and an independent power supply line to avoid signal interference and ensure the stable operation of the miniature camera.

[0046] It's important to note that the physical link is a crucial component connecting the miniature camera, data processing unit, and other functional modules. Its design aims to ensure stable signal transmission and reliable device operation. The physical link includes a shielded video signal transmission line and an independent power supply line. This design effectively prevents signal interference, ensuring that image data captured by the miniature camera is accurately transmitted to the data processing unit. The shielded video signal transmission line, through its special materials and structural design, prevents external electromagnetic interference, thus guaranteeing the integrity of the image signal. The independent power supply line provides a stable power source for the miniature camera, ensuring its normal operation under various working conditions.

[0047] Specifically, a shielded video signal transmission line is a cable with electromagnetic shielding capabilities. It typically consists of a shielding layer made of conductive material wrapped around the signal transmission line, effectively blocking external electromagnetic interference. This shielding layer can be a braided metal mesh or metal foil, and through grounding, interference signals are guided to the ground wire, thus protecting the internal signals from interference. An independent power supply line provides a separate power path for the miniature camera, avoiding voltage fluctuations that may occur when sharing power with other functional modules. This design usually includes voltage regulation and filtering circuits to ensure the stability and purity of the power supply. The parameters of the physical link, such as the impedance of the transmission line and the shielding effectiveness, are determined according to the specific requirements of the miniature camera and data processing unit to ensure the performance and reliability of the entire system.

[0048] Preferably, to further improve the performance of the physical link, a multi-layer shielding structure can be adopted in the shielded video signal transmission line. For example, an additional shielding layer can be added on top of the inner shielding layer to enhance the electromagnetic shielding effect. Simultaneously, the independent power supply line can be equipped with an intelligent power management system to monitor the power supply status in real time and adjust the voltage and current to adapt to the needs of the miniature camera under different workloads. Furthermore, the design of the physical link can also consider environmental adaptability. For example, in harsh environments such as high temperature, low temperature, or high humidity, weather-resistant materials and protective measures can be used to ensure the long-term stable operation of the link. These optimization measures work together to enable the headset to provide stable and reliable lip-reading recognition functionality in various complex environments.

[0049] In some embodiments, the processing flow of the dynamically changing lip image specifically includes: The continuously acquired images of dynamic changes in the lips are processed in frames, with each frame sampled at 30fps. The feature data of lip contour, motion trajectory and shape changes are extracted using a digital signal processor; The feature data is input into the codec chip and matched with standard lip movements in the pre-stored lip reading database to generate a digital voice signal.

[0050] It's important to note that the processing of dynamic lip movement images is a crucial step in achieving lip-reading recognition. This process involves segmenting continuously acquired dynamic lip movement images into frames, extracting feature data on lip contours, movement trajectories, and shape changes using a digital signal processor (DSP), and then inputting this feature data into a codec chip to match it with standard lip movements in a pre-stored lip-reading database, ultimately generating a digital speech signal. This process ensures efficient conversion from image acquisition to speech output, enabling clear voice communication even in noisy or silent environments.

[0051] Specifically, frame-by-frame processing refers to dividing a continuous video stream into individual frames for analysis and processing frame by frame. The sampling rate for each frame is set to 30fps (30 frames per second), a commonly used video sampling rate that ensures smoothness and continuity. A digital signal processor (DSP) extracts key lip features from each frame using specific image processing algorithms, such as edge detection, contour extraction, and motion analysis. These features include the lip contour, movement trajectory, and shape changes; this feature data forms the basis for subsequent lip-reading recognition. The codec chip then uses this feature data to match it against a pre-stored lip-reading database. The lip-reading database is a collection containing various standard lip movements and their corresponding speech signals. Through the matching process, the codec can generate a digital speech signal corresponding to the lip movements.

[0052] Preferably, to further improve the accuracy and efficiency of lip reading recognition, the DSP can employ a multi-level feature extraction algorithm. First, a coarse contour extraction is performed, followed by progressive refinement to specific lip movement features. For example, the lip contour can be extracted first using an edge detection algorithm, and then the lip movement trajectory can be analyzed using optical flow. Furthermore, the codec chip can incorporate machine learning algorithms during the matching process. By learning and training on a large amount of lip reading data, the matching model can be continuously optimized, improving the matching accuracy. During processing, a feedback mechanism can also be implemented to monitor the accuracy and reliability of the recognition results in real time, and dynamically adjust processing parameters based on the feedback to adapt to different usage scenarios and user needs. These optimization measures work together to ensure that the headset provides stable and reliable lip reading recognition functionality in various complex environments.

[0053] In some embodiments, when the ambient noise level exceeds 130dB, the lip reading function is automatically activated to replace the microphone for voice input, ensuring the quality of voice communication.

[0054] It should be noted that the noise-canceling headset of this invention has the feature of automatically activating lip-reading recognition when the ambient noise level exceeds 130dB. This function is achieved based on the real-time monitoring of the surrounding noise level by the environmental noise monitoring module. When the detected noise level exceeds a set threshold, the system automatically switches to lip-reading recognition mode, thereby ensuring the quality of voice communication. This automatic switching mechanism not only improves the intelligence of the device but also reduces the user's operational burden in complex environments, enabling the headset to seamlessly switch to lip-reading recognition in noisy environments and ensuring the clarity of voice communication.

[0055] Specifically, the environmental noise monitoring module typically includes one or more high-sensitivity microphones, which are used not only for voice acquisition but also for real-time monitoring of environmental noise levels. Noise intensity is usually measured in decibels (dB) using a sound pressure level (SPL) sensor. When the detected environmental noise intensity exceeds 130 dB, the system triggers a signal to notify the data processing unit to switch to lip-reading mode. At this point, a miniature camera begins to work, capturing images of the user's dynamic lip movements, which are then processed and recognized by the data processing unit. The 130 dB threshold is set based on the maximum noise level that the human ear can tolerate and the actual needs of voice communication, ensuring effective switching to lip-reading functionality even in extremely noisy environments.

[0056] Preferably, to further improve the accuracy and reliability of automatic switching, the environmental noise monitoring module can employ an array of multiple microphones, using beamforming technology to enhance noise monitoring accuracy. Furthermore, the system can be set with a short delay, such as a few seconds, to avoid false triggering of the lip-reading function due to brief noise peaks. During automatic switching, the data processing unit can notify the user of the current working mode in real time, for example, through voice prompts built into the headset or flashing indicator lights. Simultaneously, the system can also be configured with a manual overlay function based on user preferences, allowing users to manually adjust the working mode after automatic switching. These optimizations work together to ensure stable and reliable voice communication in various complex environments, while also catering to individual user needs.

[0057] In some embodiments, the active noise reduction unit has a comprehensive noise reduction capability of over 30dB and a noise reduction frequency band covering 50Hz to 8000Hz, meeting the national standard GB / T1 / 3 octave band test requirements.

[0058] It should be noted that the noise-canceling headset of this invention possesses powerful active noise cancellation capabilities, achieving a comprehensive noise reduction of over 30dB, covering a noise reduction frequency band from 50Hz to 8000Hz, and meeting the national standard GB / T1 / 3 octave band test requirements. The active noise cancellation function collects ambient noise through a microphone and generates inverse sound waves through digital signal processing to cancel out the noise, thereby achieving a noise reduction effect. This design allows the headset to provide users with a clear voice communication experience in various noisy environments, while meeting relevant national standards and ensuring product reliability and safety.

[0059] Specifically, the core of the active noise cancellation unit is digital signal processing technology. It uses a microphone to collect ambient noise in real time and then uses algorithms to generate sound waves with the opposite phase to the noise, thus canceling it out. A noise reduction capability of over 30dB means the headset can significantly reduce the intensity of ambient noise, allowing users to use it comfortably even in noisy environments. The noise reduction frequency band covers 50Hz to 8000Hz, encompassing most noise frequencies perceptible to the human ear, ensuring the headset has good noise reduction effects against various types of noise. The national standard GB / T1 / 3 octave band test requirement is a standard method for evaluating the performance of noise cancellation devices. It assesses the overall performance of the device by measuring the noise reduction effect in different frequency bands. The headset of this invention has passed this standard test, demonstrating that its noise reduction performance meets industry standards.

[0060] Preferably, to further enhance the active noise cancellation effect, the headset can employ a dual-microphone array design. One microphone is used to collect ambient noise, and the other microphone is used to collect noise inside the earcups. This dual feedback mechanism allows for more precise adjustment of the noise cancellation signal. Furthermore, the digital signal processing algorithm can utilize adaptive filtering technology to dynamically adjust the noise cancellation parameters based on changes in ambient noise, thus maintaining optimal noise cancellation performance in various noise scenarios. In practical applications, the headset can also be equipped with a user interface that allows users to adjust the noise cancellation intensity according to their personal preferences. For example, they can select the strongest noise cancellation mode when complete isolation from external noise is needed, or a weaker noise cancellation mode when some ambient sound needs to be preserved. These optimization measures work together to ensure that the headset provides stable and reliable noise cancellation in various complex environments, while also catering to the personalized needs of users.

[0061] Figure 3 This is a diagram illustrating the physical connection of the headset, such as... Figure 3 As shown, in some embodiments, the physical link design of a noise-canceling headset may include the following modules: Power supply module: The headset receives a 5V power supply from an external power source to power the entire system. The power management module is integrated inside the earcups and uses dynamic voltage regulation technology to achieve low-power power management.

[0062] Active noise cancellation module: Noise-canceling microphone: Used to collect ambient noise.

[0063] Active noise cancellation module: Receives ambient noise signals collected by the noise cancellation microphone, and generates inverse sound waves through digital signal processing to cancel out the ambient noise, thereby achieving a noise reduction effect.

[0064] Receiver (speaker): Receives the noise-canceling signal generated by the active noise cancellation module and outputs a clear voice signal.

[0065] lip reading module Miniature camera: Fixedly mounted on the surface of the microphone housing, used to capture real-time images of the user's lip movements. The miniature camera has a field of view of 80° to 100°, and its installation position is aligned with the microphone's pickup direction to ensure synchronization between the lip movement images and the voice input.

[0066] Microphone: Used to collect the user's voice input.

[0067] Lip reading processing module: includes a data processing unit and a speech synthesis unit.

[0068] Data processing unit: Connected to a miniature camera via a physical link, it receives images of dynamic lip movements and converts the image data into digital speech signals using a lip-reading algorithm. The data processing unit includes a digital signal processor (DSP) and a codec chip.

[0069] Digital signal processor (DSP): performs frame segmentation, sampling, and feature extraction on images of dynamic lip changes.

[0070] Codec chip: Matches the extracted lip feature data with a pre-stored lip reading database and generates the corresponding digital speech signal.

[0071] Speech synthesis unit: converts digitized speech signals into output synthesized speech signals.

[0072] Switching: A mode switch button is located on the outer surface of the left earcup, used to control the activation or deactivation of active noise cancellation and lip reading functions. When the ambient noise level exceeds 130dB, the lip reading function is automatically activated, replacing the microphone for voice input to ensure voice communication quality.

[0073] physical link Shielded video signal transmission cable: used to connect miniature cameras and data processing units to avoid signal interference and ensure stable transmission of images showing dynamic changes in the lips.

[0074] Independent power supply line: Provides an independent power path for the miniature camera, avoiding voltage fluctuations that may occur when sharing power with other functional modules, and ensuring the stable operation of the miniature camera.

[0075] Voice output: The synthesized voice signal is output through the receiver (speaker) to realize the conversion of lip reading to speech, ensuring the quality of voice communication for users in noisy environments or silent states.

[0076] Through the above physical link design, the noise-canceling headset of the present invention can achieve clear voice communication through lip reading technology in strong noise environment or silent state, and at the same time has a powerful active noise cancellation function to ensure the communication quality of users in various complex environments.

[0077] The various embodiments of the present invention have the following beneficial effects: By integrating a miniature camera and a lip-reading system into a noise-canceling headset, the present invention can automatically switch to lip-reading mode in environments with strong noise levels above 130dB, effectively solving the problem of speech distortion caused by the sound masking effect of traditional microphones. Employing a hardware architecture combining the YOLOv5 algorithm with DSP and codec chips, real-time lip image acquisition, feature extraction, and speech synthesis can be completed, ensuring communication latency is below the perception threshold. Simultaneously, shielded video signal transmission lines and independent power supply lines ensure signal transmission stability.

[0078] In terms of functionality, the camera and microphone, with an 80°-100° field of view aligned, ensure synchronization between lip movements and voice input, while dynamic voltage adjustment technology optimizes system power consumption. A mode switching button allows users to flexibly choose between normal environments and noisy scenarios, and a pre-stored lip-reading database and multi-scale feature extraction algorithms improve recognition accuracy. Combined with active noise reduction capabilities exceeding 30dB, it covers the 50Hz-8000Hz frequency band, forming a dual communication guarantee mechanism with lip-reading recognition, maintaining communication quality under extreme noise and enabling silent information transmission in quiet environments.

[0079] Furthermore, Figure 2 A flowchart for lip reading recognition, such as Figure 2 As shown, the lip-reading recognition process of this invention mainly includes the following steps: A miniature camera is fixedly mounted on the surface of the microphone housing to capture real-time images of the user's lip movements. The miniature camera has a field of view of 80° to 100°, and its installation position is aligned with the microphone's pickup direction to ensure that the lip movement images are synchronized with the voice input.

[0080] The acquired images of dynamic lip changes first undergo video preprocessing. This preprocessing includes noise reduction and contrast enhancement to improve image quality and ensure the accuracy of subsequent feature extraction. A digital signal processor (DSP) then processes the continuously acquired images of dynamic lip changes frame by frame, with each frame sampled at 30fps to ensure the continuity and integrity of the image data.

[0081] A digital signal processor (DSP) extracts features from the preprocessed image. The extracted features include lip contours, motion trajectories, and shape changes. This feature data forms the basis for subsequent lip-reading.

[0082] The extracted lip feature data is input into a codec chip and matched against a pre-stored lip-reading database. The database contains various standard lip movements and their corresponding speech signals. The codec chip generates the corresponding digitized speech signal by matching these feature data.

[0083] The speech synthesis unit is electrically connected to the data processing unit, converting digitized speech signals into outputtable synthesized speech signals. The speech synthesis unit uses specific algorithms to convert digitized speech signals into natural and fluent speech signals.

[0084] The synthesized speech signal is output through the headset's speaker, thereby achieving lip-reading to speech conversion and ensuring the quality of the user's voice communication in noisy or silent environments.

[0085] Through the above process, the noise-canceling headset of the present invention can achieve clear voice communication in a strong noise environment or in a silent state through lip reading technology.

[0086] Furthermore, the storage medium in the embodiments of this application stores program instructions capable of implementing all the above methods. These program instructions can be stored in the storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0087] The above description is merely an explanation of some preferred embodiments of the present invention and the technical principles employed. Those skilled in the art should understand that the scope of the invention as described in the embodiments of the present invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. A noise-canceling headset with lip-reading recognition function, comprising earcups, an active noise-canceling unit, and a microphone, characterized in that, Also includes: A miniature camera is fixedly mounted on the surface of the microphone housing to capture images of the user's dynamic lip movements in real time. The data processing unit is connected to the miniature camera via a physical link, and is used to receive the dynamic change image of the lips, and convert the image data into digital speech signals through a lip reading algorithm; The speech synthesis unit, electrically connected to the data processing unit, is used to convert the digitized speech signal into an outputtable synthesized speech signal.

2. The headband noise-canceling headset with lip-reading recognition function according to claim 1, characterized in that, The miniature camera has a field of view of 80° to 100° and its installation position is aligned with the microphone's pickup direction to ensure that the dynamic changes in lip movements are synchronized with the voice input.

3. The headband noise-canceling headset with lip-reading recognition function according to claim 1, characterized in that, The data processing unit includes: A digital signal processor (DSP) is used to perform frame segmentation, sampling, and feature extraction on the dynamic lip change image. The codec chip is used to match the extracted lip feature data with a pre-stored lip reading database and generate the corresponding digital speech signal.

4. The headband noise-canceling headset with lip-reading recognition function according to claim 3, characterized in that, The lip-reading algorithm is the YOLOv5 algorithm, which includes an input terminal, a Backbone network, a Neck network, and a Prediction network, wherein: The Backbone network is used to extract multi-scale features from images of dynamic changes in the lips. The Neck network is used to fuse features at different scales; The Prediction network is used to output the predicted speech content corresponding to lip movements.

5. The noise-canceling headset with lip-reading recognition function according to claim 1, characterized in that, It also includes a power processing module, which is integrated inside the earcup and connected to the data processing unit, miniature camera and voice synthesis unit, and achieves low-power power management through dynamic voltage regulation technology.

6. The noise-canceling headset with lip-reading recognition function according to claim 1, characterized in that, The outer surface of the left earcup is provided with a mode switching button, which is used to control the activation or deactivation of the active noise cancellation function and the lip reading function.

7. The headband noise-canceling headset with lip-reading recognition function according to claim 1, characterized in that, The physical link includes a shielded video signal transmission line and an independent power supply line, which are used to avoid signal interference and ensure the stable operation of the miniature camera.

8. The noise-canceling headset with lip-reading recognition function according to claim 1, characterized in that, The processing flow of the dynamic lip change image specifically includes: The continuously acquired images of dynamic changes in the lips are processed in frames, with each frame sampled at 30fps. The feature data of lip contour, motion trajectory and shape changes are extracted using a digital signal processor; The feature data is input into the codec chip and matched with standard lip movements in the pre-stored lip reading database to generate a digital voice signal.

9. The headband noise-canceling headset with lip-reading recognition function according to claim 1, characterized in that, When the ambient noise level exceeds 130dB, the lip reading function is automatically activated to replace the microphone for voice input, ensuring the quality of voice communication.

10. The headband noise-canceling headset with lip-reading recognition function according to claim 1, characterized in that, The active noise reduction unit has a comprehensive noise reduction capability of over 30dB, and the noise reduction frequency band covers 50Hz to 8000Hz, meeting the national standard GB / T1 / 3 octave band test requirements.

Images