Sound noise-shielding apparatus and mask earphone

Through the identification and replacement technology of sound noise occlusion devices, the problem of discomfort in specific sounds of patients with phonophobia is solved, effective sound silencing effect is achieved, and the patient's health is improved.

WO2024239298A1PCT designated stage expired Publication Date: 2024-11-28FOK HIU LING
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2023/096153
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-24
Filing Date
2023-05-25
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Patients with phonophobia will feel discomfort when receiving specific sounds. The prior art is difficult to effectively block these discomfort sounds, affecting the health of the patient.

Method used

A sound noise occlusion device is designed, including a sound acquisition module, a sound recognition module, a hazard information acquisition module and a hazard sound replacement module. By identifying sound categories and semantic texts, it generates and transmits sounds and replaces sensitive audio, and realizes the use of voice phonophobia patients. The sound silence of unwilling to hear.

Benefits of technology

It effectively reduces the stress response caused by sound in patients with sound phonophobia, and through separation and replacement techniques, good sound silencing effects are achieved and the patient's health is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023096153_28112024_PF_FP_ABST
    Figure CN2023096153_28112024_PF_FP_ABST
Patent Text Reader

Abstract

Disclosed in the present invention are a sound noise-shielding apparatus and a mask earphone, which integrate an earphone, a hearing aid, and noise-shielding earplugs. The noise-shielding earplugs are used for directly blocking real sounds, and after the earphone receives a sound that can cause a stress response, a bone conduction earphone converts the sound into other more acceptable sounds, so that a misophonia patient can be less influenced by the sound. Since a case that multiple sound segments are overlapped at the same time possibly occurs at times and a computer can not distinguish the sound segments due to the two-phase fusion, a separation network is adopted to separately extract the fused audios, such that sound spectrograms of the individual sounds are correspondingly obtained for detection, and by means of training the network, a sound recognition model capable of accurately detecting sound information is obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Sound noise masking device and masking earphones Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a sound noise shielding device and a noise shielding earphone. Background Art

[0002] Currently, headphones are typically used for listening to music or making phone calls. Our headphones can receive sound and immediately output another sound through bone conduction, similar to the principle of a voice changer. Some people experience misophonia, for example, where certain sounds cause discomfort. Therefore, noise-canceling headphones are needed. These voice-changing headphones can convert unpleasant sounds into sounds the user prefers or even mute them. For example, for someone who is afraid of hearing a dog barking, these headphones can automatically mute the sound upon receiving it. Noise-canceling headphones can help patients with misophonia block out the corresponding sounds, contributing to their health.

[0003] Summary of the Invention

[0004] The object of the present invention is to provide a sound noise masking device and a noise masking earphone to solve the above-mentioned problems existing in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a sound noise masking device, comprising:

[0006] A sound acquisition module is used to obtain received sound; the received sound is the external sound received by the misophonia noise masking headphones for a period of time;

[0007] A sound recognition module is configured to obtain sound information based on the received sound through a sound recognition model; the sound information includes a sound category, a sound duration, and a semantic text; the sound category indicates whether the sound belongs to a living thing or an object; the sound duration includes the sound start time and the sound end time; and the semantic text indicates the words in the sound;

[0008] a danger information acquisition module, configured to obtain a danger information set; the danger information set comprising a danger category set and a danger word set; the danger category set comprising a plurality of danger categories; the danger word set comprising a plurality of danger words; the danger categories representing types of sounds that misophonia sufferers do not want to hear; and the danger words representing words that misophonia sufferers do not want to hear;

[0009] The danger sound replacement module is used to obtain the transmission sound based on the received sound, sound information and danger collection information.

[0010] Optionally, obtaining the transmission sound based on the received sound, the sound information, and the danger set information includes:

[0011] Obtaining a plurality of alternative audio categories; wherein the alternative audio category is a category of audio that replaces sensitive sounds;

[0012] If the sound category belongs to the dangerous category set or the semantic text belongs to the dangerous word set, traverse multiple alternative audio categories in sequence to find an alternative audio of the same alternative audio category as the sound category in the sound information, and obtain an accurate alternative audio; the accurate alternative audio is the alternative information corresponding to the sound information;

[0013] Muting the sound of the sound existence time in the received sound to obtain a muted sound;

[0014] From the starting point of the muted sound, the accurate replacement audio is used to replace it and obtain the delivered sound.

[0015] Optional training methods for the sound recognition model include:

[0016] Obtaining a training set; the training set includes a plurality of training sounds and corresponding plurality of labeled data; the training sounds represent a plurality of sounds previously received by the misophonia noise-masking headphones; the labeled data include a plurality of labeled sound categories, a plurality of labeled semantic texts, a plurality of labeled single sound spectrograms, a plurality of labeled sound start times, and a plurality of labeled sound end times; the labeled sound categories represent sound categories;

[0017] Performing a short-time Fourier transform on the training sound to obtain a training spectrogram;

[0018] Based on the training spectrogram, overlapping sounds are separated to obtain a plurality of training single spectrograms;

[0019] Based on the training single sound spectrogram, training sound information is obtained; the training sound information includes the training sound category, the training sound existence time, and the training semantic pronunciation; the training sound existence time includes the training sound start time and the training sound end time;

[0020] Based on multiple training single sound spectrograms and corresponding multiple labeled single sound spectrograms, a single sound spectrum loss value is obtained;

[0021] The training sound category and the labeled sound category are subjected to a binary cross entropy loss function to obtain a sound category loss value;

[0022] The training semantic pronunciation and the annotated semantic text are subjected to a binary cross entropy loss function to obtain a semantic text loss value;

[0023] The training sound start time and the labeled sound start time are subjected to a binary cross entropy loss function to obtain a sound start time loss value;

[0024] The training sound end time and the labeled sound end time are subjected to a binary cross entropy loss function to obtain a sound end time loss value;

[0025] Obtain a total loss value; the total loss value is the sum of the sound category loss value, the semantic text loss value, the sound start time loss value, and the sound end time loss value;

[0026] Obtaining the current number of training iterations of the sound recognition model and a preset maximum number of training iterations of the sound recognition model;

[0027] When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained sound recognition model.

[0028] Optionally, the step of separating overlapping sounds based on the training spectrogram to obtain a plurality of training single-sound spectrograms includes:

[0029] Multiplying the training spectrogram by a first separation matrix to obtain a first single spectrogram; the first separation matrix is ​​a constant used to separate a single spectrogram obtained through training from the spectrogram; the first single spectrogram is a spectrogram of only one sound separated from overlapping sounds;

[0030] Input the sound spectrogram and the first sound spectrogram into the times switch structure to obtain a first switch value; the first switch value is 1, indicating that the sound spectrogram can be further separated; the first switch value is 0, indicating that the sound spectrogram has been separated;

[0031] If the first switch value is 1, multiply the spectrogram by the second separation matrix to obtain a second single spectrogram;

[0032] Inputting the spectrogram, the first single spectrogram and the second single spectrogram into a times switch structure to obtain a second switch value;

[0033] The switch value in the switch structure is obtained multiple times until the switch value is 0 and the spectrogram separation is stopped, thereby obtaining multiple training single spectrograms; the multiple training spectrograms are multiple single spectrograms separated when the switch value is 1.

[0034] Optionally, inputting the sound spectrogram and the first sound spectrogram into a times switch structure to obtain a switch value includes:

[0035] Gray-scale the spectrogram to obtain a gray-scale spectrogram;

[0036] Gray-scale the first sonogram to obtain a first gray-scale sonogram;

[0037] Subtracting the grayscale value in the grayscale spectrogram from the grayscale value in the first spectrogram to obtain a grayscale difference spectrogram;

[0038] The grayscale difference spectrogram is subjected to a clustering algorithm to obtain a plurality of grayscale value sets of clustered areas;

[0039] A switch value is obtained based on the multiple aggregation area grayscale value sets.

[0040] Optionally, obtaining the switch value based on the plurality of grayscale value sets of the clustered areas includes:

[0041] Obtaining an optimal clustering region grayscale value set; the optimal clustering region grayscale value set is a set of clustering region grayscale value sets that is greater than other clustering region grayscale value sets;

[0042] Obtaining an optimal clustering region grayscale average value; the optimal clustering region grayscale average value is an average value of grayscale values ​​in the optimal clustering region grayscale value set;

[0043] If the grayscale average value of the optimal clustering area is greater than the grayscale threshold, the switch value is set to 1;

[0044] If the grayscale average value of the optimal aggregation area is less than the grayscale threshold, the switch value is set to 0.

[0045] Optionally, obtaining training sound information based on the training single sound spectrogram includes:

[0046] Inputting the single sound spectrogram into a convolutional neural network to extract features to obtain a training single sound feature graph;

[0047] Input the single sound feature map into the fully connected layer to obtain the training sound category;

[0048] Based on the training single sound feature map, obtaining the existence time of the training sound through a neural network;

[0049] If the training sound category is human voice, the single sound feature map is input into the semantic recognition network to obtain the training semantic pronunciation.

[0050] Optionally, obtaining a single spectrum loss value based on a plurality of training single spectrum graphs and a corresponding plurality of labeled single spectrum graphs includes:

[0051] Rearranging the training single sound spectrogram to obtain a training single sound vector;

[0052] Rearranging the labeled single-sound spectrogram to obtain a labeled single-sound vector;

[0053] The training single sound vector and the labeled single sound vector are subjected to a binary cross entropy loss function to obtain a single sound spectrum loss value.

[0054] Optionally, the sound masking device further includes a sound-absorbing structure and a sound-absorbing clip;

[0055] The muffler structure is a gourd-shaped cavity structure, with a sound-absorbing port provided on the neck end of the muffler structure, a plurality of muffler holes penetrating the cavity wall of the muffler structure, and muffler cotton provided on the outer wall of the muffler structure. The muffler structure can expand or contract;

[0056] The silencer clip is arranged on the neck of the silencer structure and is used to clamp or release the neck of the silencer structure.

[0057] In a second aspect, an embodiment of the present invention provides a noise-canceling headset, comprising a noise-canceling earmuff, wherein the noise-canceling earmuff comprises an outer earmuff, an inner earmuff, and the sound-canceling device described in any one of the above items;

[0058] The inner earmuff is detachably arranged inside the outer earmuff via a spring, the outer earmuff can cover the user's ear, and the inner earmuff can be inserted into the user's external auditory canal opening;

[0059] The sound masking device is connected to the outer ear cover. Compared with the prior art, the embodiment of the present invention achieves the following beneficial effects:

[0060] An embodiment of the present invention also provides a sound noise masking device and a masking headset, which obtains received sound, wherein the received sound is external sound received by the misophonia noise masking headset for a period of time. Based on the received sound, sound information is obtained through a sound recognition model. The sound information includes sound category, sound existence time and semantic text. The sound category indicates that the sound belongs to the category of a living thing or an object. The sound existence time includes the sound start time and the sound end time. The semantic text represents the words in the sound. A danger information set is obtained. The danger information set includes a danger category set and a danger word set. The danger category set includes multiple danger categories. The danger word set includes multiple danger words. The danger category represents the type of sound that misophonia patients do not want to hear. Dangerous text is text that misophonia patients do not want to hear. Based on the received sound, sound information and danger set information, a transmitted sound is obtained.

[0061] Headphones, hearing aids, and noise-masking earplugs are now combined into one device. Noise-masking earplugs are used to directly block out real sounds, allowing those with misophonia to reduce the effects of earphones on the sound that can cause stress. Bone conduction headphones convert sounds into more palatable sounds. Sometimes, multiple sounds may mix simultaneously, making the fusion of the two difficult for a computer to distinguish. A separation network is used to extract the fused audio separately, generating spectrograms corresponding to the individual sounds and detecting them. By training the network, a sound recognition model is developed that accurately detects sound information. Weights are obtained by multiplying the spectrogram by an inverse matrix. By cyclically adding the results and using labeled single spectrograms to generate a training matrix, the single spectrograms can be separated to produce distinct single spectrograms. The phase difference between the two images is determined by grayscale differences, for example. An aggregation algorithm removes unnecessary noise.

[0062] In addition, the noise-blocking headphones include outer earmuffs and inner earmuffs. The inner earmuffs extend into the external auditory canal to transmit sound to the user's ears, while the outer earmuffs cover the user's ears, isolating external sounds from the outside. On the one hand, this can reduce the impact of external noise on the user, and on the other hand, it can prevent sound leakage from the user's headphones and affect the surrounding environment. In summary, the noise-blocking headphones provided by this application can achieve noise cancellation by at least three means, and its noise cancellation effect is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] FIG1 is a schematic structural diagram of a mask earphone provided by an embodiment of the present invention.

[0064] FIG2 is a flow chart of a sound noise masking method provided by an embodiment of the present invention.

[0065] FIG3 illustrates a separation process in a sound noise masking method provided by an embodiment of the present invention.

[0066] FIG4 is a block diagram of a masked earphone provided by an embodiment of the present invention.

[0067] Markings in the figure: 100 - sound masking device; 110 - sound-absorbing structure; 120 - sound-absorbing clip; 200 - noise-shielding headphones; 210 - noise-reducing earmuffs; 211 - outer earmuffs; 212 - inner earmuffs; 213 - spring; 220 - arc-shaped connecting handle 220; bus 500; receiver 501; processor 502; transmitter 503; memory 504; bus interface 505. DETAILED DESCRIPTION

[0068] The present invention will be described in detail below with reference to the accompanying drawings.

[0069] Example 1

[0070] As shown in FIG1 , an embodiment of the present invention provides a sound masking device 100 . The sound masking device 100 includes a sound-absorbing structure 110 and a sound-absorbing clip 120 .

[0071] The silencer structure 110 is a gourd-shaped cavity structure, with a sound-absorbing port provided on the neck end of the silencer structure, a plurality of silencer holes penetrating the belly cavity wall provided on the belly cavity of the silencer structure, and silencer cotton provided on the outer wall of the silencer structure. The silencer structure 110 can expand or contract.

[0072] The noise reduction clip 120 is provided at the neck of the noise reduction structure 110 and is used to clamp or release the neck of the noise reduction structure 110 .

[0073] Optionally, the sound masking device 100 further includes an electronic silencing module, which is used to determine the type of sound and selectively silence the sound according to the sound type.

[0074] The sound masking device 100 absorbs sound into the belly cavity of the sound-absorbing structure 110 through the sound-absorbing port, diffuses the sound waves through the sound-absorbing hole, and silences the sound through the sound-silencing cotton. The sound-absorbing structure 110 can expand or contract. When the sound is loud, the sound-absorbing structure 110 can expand to expand the space of the belly cavity, increase the area for dissipating sound, and improve the sound-absorbing effect. In addition, a sound-absorbing clip 120 is provided. When sound-absorbing is not required, the sound-absorbing clip 120 can clamp the neck of the sound-absorbing structure 110 so that the sound-absorbing structure 110 does not eliminate sound. When sound-absorbing is required, the sound-absorbing clip 120 opens the neck of the sound-absorbing structure 110 so that the sound-absorbing structure 110 can absorb and eliminate sound.

[0075] Furthermore, the sound masking device 100 also includes an electronic mute module, which is used to determine the type of sound and selectively mute it based on the sound type. Specifically, the electronic mute module includes a sound acquisition module, a sound recognition module, a hazard information acquisition module, and a hazard sound replacement module. The sound acquisition module is electrically connected to the sound recognition module, which is in turn electrically connected to the hazard information acquisition module, which is in turn electrically connected to the hazard sound replacement module.

[0076] The sound acquisition module is used to obtain received sound, wherein the received sound is the external sound received by the misophonia noise masking earphones for a period of time.

[0077] The sound recognition module is configured to obtain sound information based on the received sound using a sound recognition model. The sound information includes the sound category, the sound duration, and semantic text. The sound category indicates whether the sound belongs to a living being or an object. The sound duration includes the sound start time and the sound end time. The semantic text indicates the words in the sound.

[0078] The danger information acquisition module is configured to obtain a danger information set. The danger information set includes a danger category set and a danger word set; the danger category set includes multiple danger categories. The danger word set includes multiple danger words; the danger categories represent the types of sounds that misophonia patients do not want to hear. The danger words are words that misophonia patients do not want to hear.

[0079] The danger sound replacement module is used to obtain the transmission sound based on the received sound, sound information and danger collection information.

[0080] By adopting the above solution, received sound is obtained, where the received sound is external sound received by misophonia noise-masking headphones over a period of time. Based on the received sound, sound information is obtained using a sound recognition model. The sound information includes the sound category, sound duration, and semantic text. The sound category indicates whether the sound belongs to a living being or an object. The sound duration includes the sound start time and the sound end time. The semantic text represents the words in the sound. A set of danger information is obtained. The danger information set includes a set of danger categories and a set of danger words. The danger category set includes multiple danger categories. The danger word set includes multiple danger words. The danger categories represent the types of sounds that misophonia patients do not want to hear. The danger words are words that misophonia patients do not want to hear. Based on the received sound, sound information, and danger set information, a transmitted sound is obtained. By combining headphones, hearing aids, and noise-masking earplugs, the noise-masking earplugs are used to directly block real sounds, allowing misophonia patients to reduce the effects of sound on the headphones. After receiving sounds that can cause stress reactions, the bone conduction headphones will convert them into other more acceptable sounds. Sometimes, multiple sounds may mix simultaneously, making the fusion of the two sounds difficult for a computer to distinguish. A separation network is used to extract the fused audio separately, generating spectrograms corresponding to the individual sounds and detecting them. By training the network, a sound recognition model is developed that accurately detects sound information. Weights are obtained by multiplying the spectrograms by an inverse matrix. By cyclically adding them and using labeled individual spectrograms to generate a training matrix, the individual spectrograms can be separated to produce distinct individual spectrograms. The phase difference between the two images is determined by grayscale difference, for example. Using a clustering algorithm, unnecessary noise is removed.

[0081] Continuing to refer to FIG1 , an embodiment of the present invention further provides a noise-canceling headphone 200 , comprising a noise-canceling earmuff 210 , wherein the noise-canceling earmuff 210 comprises an outer earmuff 211 , an inner earmuff 212 , and a sound masking device 100 ;

[0082] The inner ear cover 212 is detachably arranged inside the outer ear cover 211 via a spring 213. The outer ear cover 211 can cover the user's ears, and the inner ear cover 212 can be inserted into the user's external auditory canal.

[0083] The sound masking device 100 is connected to the outer ear cover 211 .

[0084] When noise reduction is required, the noise reduction clip 120 is opened so that the sound masking device 100 can absorb the sound, thereby achieving a noise reduction effect. That is, when the user does not want to hear unwanted sounds coming from the earphones, the sound masking device 100 can be opened to achieve noise reduction. In addition, the inner earmuff 212 extends into the opening of the external auditory canal, transmitting the sound to the user's ear, while the outer earmuff 211 covers the user's ear, isolating the external sound from the outside. On the one hand, it can reduce the impact of external noise on the user, and on the other hand, it can prevent the user's earphones from leaking sound and affecting the surrounding environment. In summary, the masked earphones 200 provided in this application can achieve noise reduction by at least three means, and its noise reduction effect is good.

[0085] Optionally, the sound masking device 100 is detachably connected to the outer earmuff. Specifically, the outer earmuff 211 includes a sound-absorbing opening extending through the outer earmuff wall. The cavity of the sound-absorbing structure 110 communicates with the cavity of the outer earmuff 211 via the sound-absorbing opening and the sound-absorbing opening. Specifically, the inner wall of the sound-absorbing opening is provided with internal threads, and the outer wall of the sound-absorbing opening is provided with external threads. The threaded assembly between the internal threads of the sound-absorbing opening and the external threads of the sound-absorbing opening enables the sound masking device 100 to be detachably connected to the outer earmuff 210.

[0086] Optionally, there are two noise-canceling earmuffs 210, and the earphones 200 further include an arc-shaped connecting handle 220, one end of which is fixedly connected to the outer earmuff 211 of one of the noise-canceling earmuffs 210, and the other end of which is fixedly connected to the outer earmuff 211 of the other noise-canceling earmuff 210.

[0087] Specifically, the noise-cancelling earphones 200 include a left earmuff and a right earmuff, and the structures of the left and right earmuffs are the same as those of the noise-cancelling earmuffs 210. One end of the arc-shaped connecting handle 220 is fixedly connected to the outer earmuff 211 of the left earmuff, and the other end of the arc-shaped connecting handle 220 is fixedly connected to the outer earmuff 211 of the right earmuff.

[0088] Optionally, the arc-shaped connecting handle 220 is retractable.

[0089] Optionally, the arc-shaped connecting handle 220 can be bent.

[0090] Optionally, the cross-section of the outer ear cover 211 is oval.

[0091] In summary, in the noise-cancelling headphones 200 described above, the inner earmuff 212 is detachably mounted inside the outer earmuff 211 via a spring 213. The outer earmuff 211 can cover the user's ears, and the inner earmuff 212 can be inserted into the user's external auditory canal. In this way, the spring 213 can securely hold the inner earmuff 212 in the user's external auditory canal, preventing it from falling off. At the same time, the spring 213 can help adjust the headphones for different users, i.e., the position of the inner earmuff 212 can be adjusted in depth, front and back, left and right, and other directions, so that the inner earmuff 212 can be positioned in a comfortable position for the user. In this embodiment of the present invention, the length of the spring 213 is greater than the depth of the outer earmuff 211's housing. Combined with the above-mentioned sound masking device 100, when noise reduction is required, the noise reduction clip 120 is opened so that the sound masking device 100 can absorb the sound, thereby achieving a noise reduction effect. That is, when the user does not want to hear unwanted sounds coming from the earphones, the sound masking device 100 can be opened to achieve noise reduction, and when the user does not need noise reduction, the sound masking device 100 can be closed. In addition, the noise-canceling earphones 200 include an outer earmuff 211 and an inner earmuff 212. The inner earmuff 212 extends into the opening of the external auditory canal to transmit the sound to the user's ear, which can improve the effective conduction of the sound to the user's ear and improve the sound effect of the noise-canceling earphones 200. At the same time, the outer earmuff 211 covers the user's ears and isolates the external sound from the outside. On the one hand, it can reduce the impact of external noise on the user, and on the other hand, it can prevent the user's earphones from leaking sound and affecting the surrounding environment. In summary, the masking earphones 200 provided in this application can achieve noise reduction by at least four means, and its noise reduction effect is good.

[0092] As shown in FIG2 , an embodiment of the present invention provides a method for masking noise, the method comprising:

[0093] S101: Obtain received sound, wherein the received sound is external sound received by the misophonia noise masking earphones.

[0094] The received sound is a sound acquired in a short period of time for quick response.

[0095] S102: Based on the received sound, sound information is obtained using a sound recognition model. The sound information includes the sound category, the sound start time, and semantic text. The sound category indicates whether the sound belongs to a living being or an object. The sound duration includes the sound start time and the sound end time. The semantic text indicates the words in the sound.

[0096] The semantic text is obtained when the sound category is voice. If the sound category is not voice, the value in the semantic text is set to 0. The sound start time is the time when a sound in a certain sound category begins in the received sound. The sound end time is the time when a sound in a certain sound category ends in the received sound.

[0097] S103: Obtain a danger information set; the danger information set includes a danger category set and a danger word set; the danger category set includes multiple danger categories; the danger word set includes multiple danger words; the danger category represents the type of sound that misophonia patients do not want to hear; the danger words are words that misophonia patients do not want to hear.

[0098] S104: Based on the received sound, the sound information and the danger collection information, a transmission sound is obtained.

[0099] Optionally, obtaining the transmission sound based on the received sound, the sound information, and the danger set information includes:

[0100] A plurality of alternative audio categories are obtained; the alternative audio categories are categories of audio that replace sensitive sounds.

[0101] Among them, if the original audio is the sound of a dog barking, the replacement audio is the sound audio of "dog barking".

[0102] If the sound category belongs to the dangerous category set or the semantic text belongs to the dangerous word set, traverse multiple alternative audio categories in turn, find the alternative audio with the same alternative audio category as the sound category in the sound information, and obtain the accurate alternative audio; the accurate alternative audio is the alternative information corresponding to the sound information.

[0103] The sound of the sound existence time in the received sound is muted to obtain a muted sound.

[0104] From the starting point of the muted sound, the accurate replacement audio is used to replace it and obtain the delivered sound.

[0105] The above method can mute sounds that the user is sensitive to. However, sounds affect our lives, and the user cannot judge whether the muted sound really does not exist or is muted. Therefore, an alternative audio is used to remind the user.

[0106] Optional training methods for the sound recognition model include:

[0107] A training set is obtained; the training set includes multiple training sounds and corresponding labeled data; the training sounds represent multiple sounds previously received by the misophonia noise-masking headphones. The labeled data includes multiple labeled sound categories, multiple labeled semantic texts, multiple labeled single sound spectrograms, multiple labeled sound start times, and multiple labeled sound end times. The labeled sound category represents the category of the sound.

[0108] The labeled sound category indicates the category of the sound. For example, in this embodiment, the labeled sound category is dog barking, and the category of the sound is 08.

[0109] The training sound is subjected to short-time Fourier transform to obtain a training spectrogram.

[0110] The training spectrogram is a spectrogram after noise removal and enhancement.

[0111] Based on the training spectrogram, overlapping sounds are separated to obtain a plurality of training single spectrograms.

[0112] Based on the training single sound spectrogram, training sound information is obtained. The training sound information includes the training sound category, the training sound existence time, and the training semantic pronunciation. The training sound existence time includes the training sound start time and the training sound end time.

[0113] Based on multiple training single sound spectrograms and corresponding multiple labeled single sound spectrograms, a single sound spectrum loss value is obtained.

[0114] The training sound category and the labeled sound category are subjected to a binary cross entropy loss function to obtain a sound category loss value.

[0115] The training semantic pronunciation and the annotated semantic text are subjected to a binary cross entropy loss function to obtain a semantic text loss value.

[0116] The training sound start time and the labeled sound start time are subjected to a binary cross entropy loss function to obtain a sound start time loss value.

[0117] The training sound end time and the labeled sound end time are subjected to a binary cross entropy loss function to obtain a sound end time loss value.

[0118] Obtain a total loss value; the total loss value is the sum of the sound category loss value, semantic text loss value, sound start time loss value and sound end time loss value.

[0119] The current training iteration number of the sound recognition model and the preset maximum iteration number of the sound recognition model training are obtained.

[0120] The preset maximum number of iterations for the sound recognition model training is 1500.

[0121] When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained sound recognition model.

[0122] Using the above method, the received audio signal is converted into a two-dimensional image representing the sound state through a short-time Fourier transform. The brightness of the two-dimensional image represents the loudness, or amplitude, of the sound. The presence of multiple waves on a vertical line at the same time, when they overlap, represents the frequency of the sound. However, sometimes multiple sounds may be mixed at the same time, and the fusion of the two phases makes it difficult for a computer to distinguish them. A separation network is used to extract the fused audio separately, generating spectrograms of the corresponding individual sounds and performing detection. By training the network, a sound recognition model capable of accurately detecting sound information is developed.

[0123] Optionally, the method of separating overlapping sounds based on the spectrogram through a separation network to obtain multiple single spectrograms includes:

[0124] The training spectrogram is multiplied by a first separation matrix to obtain a first single spectrogram. The first separation matrix is ​​a constant used to separate single spectrograms obtained through training from the spectrogram; the first single spectrogram is a spectrogram of a single sound separated from overlapping sounds.

[0125] The sound spectrogram and the first sound spectrogram are input into a times switch structure to obtain a first switch value. The first switch value is 1, indicating that the sound spectrogram can be further separated. The first switch value is 0, indicating that the sound spectrogram has been completely separated.

[0126] If the first switch value is 1, the spectrogram is multiplied by the second separation matrix to obtain a second single spectrogram.

[0127] The spectrogram, the first single spectrogram and the second single spectrogram are input into the times switch structure to obtain a second switch value.

[0128] The method for obtaining the second switch value is similar to the method for obtaining the first switch value, which is to compare the grayscale value of the spectrogram minus the grayscale value of the first spectrogram, and then subtract the grayscale difference of the second spectrogram from the obtained image.

[0129] The switch value in the switch structure is obtained multiple times until the switch value is 0 and the spectrogram separation is stopped, thereby obtaining multiple training single spectrograms; the multiple training spectrograms are multiple single spectrograms separated when the switch value is 1.

[0130] The separation process is shown in FIG3 .

[0131] Using the above method, two sounds superimposed on each other have different characteristics, but due to the nature of waves, they overlap and are difficult to separate. The spectrogram is repeatedly segmented and trained according to the number of segmentations. Weights are obtained by multiplying the spectrogram by the inverse matrix. This is then cyclically added, and the training matrix is ​​generated using labeled single spectrograms. This allows for the separation of single spectrograms, resulting in distinct single spectrograms.

[0132] Optionally, inputting the sound spectrogram and the first sound spectrogram into a times switch structure to obtain a switch value includes:

[0133] The spectrogram is grayscaled to obtain a grayscale spectrogram.

[0134] The first spectrogram is grayscaled to obtain a first grayscale spectrogram.

[0135] The grayscale value in the grayscale spectrogram is subtracted from the grayscale value in the first spectrogram to obtain a grayscale difference spectrogram.

[0136] The grayscale difference spectrogram is subjected to a clustering algorithm to obtain multiple grayscale value sets of clustered areas.

[0137] A switch value is obtained based on the multiple aggregation area grayscale value sets.

[0138] Through the above method, the phase difference between the two images is obtained by grayscale difference, etc. Other unnecessary noise parts are removed according to the clustering algorithm.

[0139] Optionally, obtaining the switch value based on the plurality of grayscale value sets of the clustered areas includes:

[0140] The optimal clustering region grayscale value set is obtained. The optimal clustering region grayscale value set is a set of clustering region grayscale value sets that is greater than other clustering region grayscale value sets.

[0141] Obtaining the optimal clustering area grayscale average value; the optimal clustering area grayscale average value is the average value of the grayscale values ​​in the optimal clustering area grayscale value set.

[0142] If the grayscale average value of the optimal aggregation area is greater than the grayscale threshold, the switch value is set to 1.

[0143] If the grayscale average value of the optimal aggregation area is less than the grayscale threshold, the switch value is set to 0.

[0144] In this embodiment, the grayscale threshold is 5.

[0145] By using the above method, the grayscale average value is compared with the grayscale threshold to control the similarity value of the two images, thereby obtaining the switch value to determine whether to continue separation.

[0146] Optionally, obtaining training sound information based on the training single sound spectrogram includes:

[0147] The single sound spectrogram is input into a convolutional neural network to extract features to obtain a training single sound feature graph.

[0148] In this embodiment, the network uses a DENSET network for discrimination.

[0149] The single sound feature map is input into the fully connected layer to obtain the training sound category.

[0150] Based on the training single sound feature graph, the existence time of the training sound is obtained through a neural network.

[0151] Among them, the features in the sound feature map are classified into the start time and the end time of the sound through a neural network, which is equivalent to obtaining the border value.

[0152] If the training sound category is human voice, the single sound feature map is input into the semantic recognition network to obtain the training semantic pronunciation.

[0153] Through the above method, the DenseNet architecture clearly distinguishes between the information added to the network and the information retained, preserving the temporal connection of the sound, and can better obtain feature maps containing sound characteristics.

[0154] Optionally, obtaining a single spectrum loss value based on a plurality of training single spectrum graphs and a corresponding plurality of labeled single spectrum graphs includes:

[0155] The training single sound spectrogram is rearranged to obtain a training single sound vector.

[0156] Among them, the pixel values ​​of each time in the RGB layer of the training single image are arranged in sequence into a one-dimensional training single sound vector.

[0157] The labeled single sound spectrogram is rearranged to obtain a labeled single sound vector.

[0158] The arrangement method of the labeled single-sound vectors is the same as the arrangement method of the training single-sound vectors.

[0159] The training single sound vector and the labeled single sound vector are subjected to a binary cross entropy loss function to obtain a single sound spectrum loss value.

[0160] By the above method, the single spectrograms are segmented from the spectrogram and arranged and stretched into vectors, so that the training spectrogram can be brought close to the separation matrix of the labeled spectrogram through the binary cross entropy function.

[0161] The above method silences sounds that users are sensitive to. However, since sounds affect our daily lives, users cannot tell whether the muted sound is truly absent or simply muted. Therefore, a substitute audio is used to alert the user. The received audio signal is converted into a two-dimensional image representing the sound state through a short-time Fourier transform. Sometimes, multiple sounds may be present simultaneously, and the fusion of the two sounds can make it difficult for the computer to distinguish them. A separation network is used to extract the fused audio separately, generating spectrograms corresponding to the individual sounds and detecting them. By training the network, a sound recognition model is developed that accurately detects sound information. Two sounds, when superimposed, have different characteristics, but due to the characteristics of the waves, they overlap and are difficult to separate. The spectrogram is segmented repeatedly, and training is performed based on the number of segmentations. Weights are obtained by multiplying the spectrogram by an inverse matrix. This weight is then cyclically added and combined with labeled single spectrograms to generate a training matrix, which allows the spectrograms to be separated, resulting in distinct single spectrograms. The phase difference between the two images is determined by grayscale differences, for example. An aggregation algorithm removes unnecessary noise. The DenseNet architecture clearly distinguishes between information added to the network and information retained, preserving temporal connections between sounds. This allows for better characterization of feature maps containing sound characteristics. Individual spectrograms are segmented from the spectrogram and arranged and stretched into vectors. Using the binary cross-entropy function, the training spectrograms are aligned with the separation matrix of the annotated spectrograms.

[0162] Example 2

[0163] Based on the above-mentioned sound noise masking method, an embodiment of the present invention further provides a sound noise masking device for executing the above-mentioned sound noise masking method. The sound noise masking device includes a sound acquisition module, a sound recognition module, a danger information acquisition module and a danger information acquisition module.

[0164] The sound acquisition module is used to obtain received sound, which is the external sound received by the misophonia noise masking headphones for a period of time.

[0165] The sound recognition module is used to receive sound and obtain sound information using a sound recognition model. The sound information includes the sound category, sound duration, and semantic text. The sound category indicates whether the sound belongs to a living being or an object. The sound duration includes the start and end time of the sound. The semantic text indicates the words in the sound.

[0166] The danger information acquisition module is configured to obtain a danger information set. The danger information set includes a danger category set and a danger word set. The danger category set includes multiple danger categories. The danger word set includes multiple danger words. The danger categories represent types of sounds that misophonia sufferers do not want to hear. The danger words are words that misophonia sufferers do not want to hear.

[0167] Danger sound replacement module: based on the received sound, sound information and danger collection information, the transmitted sound is obtained.

[0168] Optionally, obtaining the transmission sound based on the received sound, the sound information, and the danger set information includes:

[0169] Obtaining a plurality of alternative audio categories; wherein the alternative audio category is a category of audio that replaces sensitive sounds;

[0170] If the sound category belongs to the dangerous category set or the semantic text belongs to the dangerous word set, traverse multiple alternative audio categories in sequence to find an alternative audio of the same alternative audio category as the sound category in the sound information, and obtain an accurate alternative audio; the accurate alternative audio is the alternative information corresponding to the sound information;

[0171] The sound of the sound existence time in the received sound is muted to obtain a muted sound.

[0172] From the starting point of the muted sound, the accurate replacement audio is used to replace it and obtain the delivered sound.

[0173] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0174] An embodiment of the present invention also provides a masking headset, as shown in Figure 4, including a memory 504, a processor 502, and a computer program stored in the memory 504 and executable on the processor 502. When the processor 502 executes the program, the steps of any one of the methods of the sound noise masking device described above are implemented.

[0175] In FIG4 , a bus architecture (represented by bus 500) is shown. Bus 500 may include any number of interconnected buses and bridges. Bus 500 links various circuits, including one or more processors represented by processor 502 and memory represented by memory 504. Bus 500 may also link various other circuits, such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and, therefore, will not be described further herein. Bus interface 505 provides an interface between bus 500 and receiver 501 and transmitter 503. Receiver 501 and transmitter 503 may be the same component, namely a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 502 is responsible for managing bus 500 and general processing, while memory 504 may be used to store data used by processor 502 when performing operations.

[0176] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods and the data involved.

[0177] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general-purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing this type of system. In addition, the present invention is not directed to any specific programming language. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0178] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0179] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the invention.

[0180] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed herein may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0181] Furthermore, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0182] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the apparatus according to an embodiment of the present invention. The present invention can also be implemented as a device or device program (e.g., a computer program and a computer program product) for executing a part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0183] It should be noted that the above embodiments illustrate rather than limit the invention, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

Claims

1. A sound noise masking device, characterized in that: The device comprises: A sound acquisition module, used to obtain received sound; the received sound is the external sound received by the misophonia noise masking headphones for a period of time; A sound recognition module is used to obtain sound information based on the received sound through a sound recognition model; the sound information includes a sound category, a sound existence time and a semantic text; the sound category indicates that the sound belongs to a category of a creature or an object; the sound existence time includes a sound start time and a sound end time; the semantic text indicates a word in the sound; A danger information acquisition module, used to obtain a danger information set; the danger information set includes a danger category set and a danger word set; the danger category set includes multiple danger categories; the danger word set includes multiple danger words; the danger category indicates the type of sound that the misophonia patient does not want to hear; the danger word is the word that the misophonia patient does not want to hear; The danger sound replacement module is used to obtain the transmission sound based on the received sound, sound information and danger collection information.

2. A sound noise masking device according to claim 1, characterized in that: The method of obtaining the transmitted sound based on the received sound, the sound information and the danger set information includes: Obtaining a plurality of alternative audio categories; the alternative audio category is a category of audio that replaces sensitive sounds; If the sound category belongs to the dangerous category set or the semantic text belongs to the dangerous word set, traverse multiple alternative audio categories in sequence, find alternative audio of the same alternative audio category as the sound category in the sound information, and obtain accurate alternative audio; the accurate alternative audio is the alternative information corresponding to the sound information; Muting the sound of the sound existence time in the received sound to obtain a muted sound; From the start position of the muted sound, use the accurate replacement audio to replace it and get the delivered sound.

3. A sound noise shielding device according to claim 1, characterized in that: The training method of the sound recognition model includes: A training set is obtained; the training set includes a plurality of training sounds and a plurality of corresponding annotated data; the training sounds represent a plurality of sounds that have been received by the misophonia noise-masking headphones; the annotated data include a plurality of annotated sound categories, a plurality of annotated semantic texts, a plurality of annotated single sound spectrograms, a plurality of annotated sound start times, and a plurality of annotated sound end times; the annotated sound category represents a category of the sound; Performing short-time Fourier transform on the training sound to obtain a training spectrogram; Based on the training spectrogram, overlapping sounds are separated to obtain a plurality of training single spectrograms; Based on the training single sound spectrogram, training sound information is obtained; the training sound information includes training sound category, training sound existence time and training semantic pronunciation; the training sound existence time includes training sound start time and training sound end time; Based on multiple training single sound spectrograms and corresponding multiple labeled single sound spectrograms, a single sound spectrum loss value is obtained; The training sound category and the labeled sound category are subjected to a binary cross entropy loss function to obtain a sound category loss value; The training semantic pronunciation and the annotated semantic text are subjected to a binary cross entropy loss function to obtain a semantic text loss value; The training sound start time and the labeled sound start time are subjected to a binary cross entropy loss function to obtain a sound start time loss value; The training sound end time and the labeled sound end time are subjected to a binary cross entropy loss function to obtain a sound end time loss value; Obtain a total loss value; the total loss value is the sum of the sound category loss value, the semantic text loss value, the sound start time loss value and the sound end time loss value; Obtaining the current training iteration number of the sound recognition model and the preset maximum iteration number of the sound recognition model training; When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained sound recognition model.

4. A sound noise masking device according to claim 3, characterized in that: The method of separating overlapping sounds based on the training spectrogram to obtain a plurality of training single-sound spectrograms includes: The training spectrogram is multiplied by a first separation matrix to obtain a first single spectrogram; the first separation matrix is ​​a constant used to separate a single spectrogram obtained from a spectrogram through training; the first single spectrogram is a spectrogram of only one sound separated from overlapping sounds; Input the sound spectrogram and the first sound spectrogram into the times switch structure to obtain a first switch value; the first switch value is 1, indicating that the sound spectrogram can continue to be separated; the first switch value is 0, indicating that the sound spectrogram has been separated; If the first switch value is 1, multiply the spectrogram by the second separation matrix to obtain a second single spectrogram; Input the spectrogram, the first single spectrogram and the second single spectrogram into a times switch structure to obtain a second switch value; The switch value in the switch structure is obtained multiple times until the switch value is 0 and the spectrogram separation is stopped to obtain multiple training single spectrograms; the multiple training spectrograms are multiple single spectrograms separated when the switch value is 1.

5. A sound noise shielding device according to claim 4, characterized in that: The step of inputting the sound spectrogram and the first sound spectrogram into a times switch structure to obtain a switch value includes: Gray-scale the spectrogram to obtain a gray-scale spectrogram; Gray-scale the first spectrogram to obtain a first gray-scale spectrogram; Subtracting the grayscale value in the grayscale spectrogram from the grayscale value in the first spectrogram to obtain a grayscale difference spectrogram; The grayscale difference spectrogram is subjected to a clustering algorithm to obtain a plurality of grayscale value sets of clustered areas; Based on the plurality of aggregation area grayscale value sets, a switch value is obtained.

6. A sound noise shielding device according to claim 5, characterized in that: The step of obtaining a switch value based on the plurality of grayscale value sets of the clustered areas comprises: An optimal clustering region grayscale value set is obtained; the optimal clustering region grayscale value set is a set of clustering region grayscale value sets that is greater than other clustering region grayscale value sets; Obtaining an optimal clustering region grayscale average value; the optimal clustering region grayscale average value is an average value of grayscale values ​​in an optimal clustering region grayscale value set; If the grayscale average value of the optimal gathering area is greater than the grayscale threshold, the switch value is set to 1; If the grayscale average value of the optimal gathering area is less than the grayscale threshold, the switch value is set to 0.

7. A sound noise shielding device according to claim 4, characterized in that: The obtaining of training sound information based on the training single sound spectrogram comprises: Input the single sound spectrogram into a convolutional neural network to extract features to obtain a training single sound feature graph; Input the single sound feature map into a fully connected layer to obtain a training sound category; Based on the training single sound feature graph, obtaining the existence time of the training sound through a neural network; If the training sound category is human voice, the single sound feature map is input into the semantic recognition network to obtain the training semantic pronunciation.

8. A sound noise shielding device according to claim 3, characterized in that: The method of obtaining a single sound spectrum loss value based on a plurality of training single sound spectrograms and a plurality of corresponding labeled single sound spectrograms includes: Rearranging the training single sound spectrogram to obtain a training single sound vector; Rearranging the labeled single-sound spectrogram to obtain a labeled single-sound vector; The training single sound vector and the labeled single sound vector are subjected to a binary cross entropy loss function to obtain a single sound spectrum loss value.

9. A sound noise shielding device according to claim 1, characterized in that: The sound masking device also includes a sound-absorbing structure and a sound-absorbing clip; The muffler structure is a gourd-shaped cavity structure, a sound absorbing port is provided on the neck end of the muffler structure, a plurality of muffler holes penetrating the cavity wall are provided on the cavity of the muffler structure, muffler cotton is provided on the outer wall of the muffler structure, and the muffler structure can expand or contract; The silencer clip is arranged on the neck of the silencer structure, and is used to clamp or release the neck of the silencer structure.

10. A noise-canceling headset, characterized in that: Comprising a noise reduction earmuff, the noise reduction earmuff comprising an outer earmuff, an inner earmuff and a sound masking device as described in any one of claims 1 to 9; The inner earmuff is detachably arranged in the outer earmuff through a spring, the outer earmuff can cover the ear of the user, and the inner earmuff can be inserted into the opening of the external auditory canal of the user; The sound masking device is connected to the outer ear cover.

Citation Information

Patent Citations

  • Systems and methods for tracking and presenting tinnitus therapy data

    CN105813608A

  • Built-in earplug type noise reduction earmuff

    CN106028224A

  • Active noise reduction method based on key sound recognition, electronic equipment and storage medium

    CN112767908A

  • Sound signal processing method and device, equipment and storage medium

    CN115295011A

  • Tinnitus rehabilitation device and method

    US20040141624A1