A scenario-based key information reminder method, system and device

Abstract

The present invention provides a key information reminder method, system and embedded audio playback device. Compared with the prior art, the embedded audio playback device can independently complete the real-time detection, reminder, recording and playback of continuous voice key information based on the scene, which is easy to use, convenient and private. The key information reminder system and reminder method are pre-trained to obtain a keyword recognition model that is highly compatible with the application scene based on the actual needs of the current application scene and the keywords or training samples customized by the user. Therefore, it can effectively improve the accuracy of identifying key information in the continuous voice stream, and timely output reminders and save the information that should be focused on in the current application scene and the information that the user is interested in, and the user experience is excellent.

Description

Technical Field

[0001] The present invention relates to the technical field of embedded devices, and in particular to a scene-based key information reminder system, method and embedded audio playback device. Background Art

[0002] At present, the Internet and mobile communication networks have entered thousands of households and spread across every corner of people's lives. Various remote audio and video applications based on these remote communication platforms, such as online conferencing, online teaching, online business negotiations, and online sales, have also become increasingly popular with the high maturity of related technologies and products such as computer network technology, audio and video processing technology, and embedded devices with system-on-chip SoC as the core. These remote audio and video applications used with embedded devices such as mobile phones, headphones, tablets, and speakers completely break the geographical restrictions, allowing people in different places to achieve real-time voice and video communication and interaction at any time, providing great convenience for people's production and life. For example, in the current situation of the surging epidemic, students can continue to take classes at home through online teaching platforms. Students often participate in online classes through headphones and walk around freely within a certain range while wearing headphones during the teaching process. But the disadvantage is that due to the lack of classroom atmosphere during online teaching, teachers cannot observe the listening status of each child in time, so they rely heavily on the self-discipline of students. Once students are distracted or playing privately, no one can give timely reminders and corrections, and the course content taught by the teacher is also missed. This situation actually exists in a similar way during online video conferencing. For example, due to personal interruptions or answering calls during the meeting, key voice information of the meeting is missed. Usually, there is no key information reminder function for the content of the speaker on the other end in the online class and video conferencing software on the mobile phone or computer. Even if there is, the local user may not be next to the mobile phone or computer. Therefore, it is very necessary to directly implement the key information reminder function in the accessory devices of the mobile phone or computer that are closest to the local user, such as headphones or speakers, so that the local user's thinking can be pulled back to the online class or video conference as soon as possible.

[0003] In recent years, speech recognition technology has been increasingly used in speech monitoring and identifying important information. In particular, with the strong support of Moore's Law and big data, speech recognition based on artificial intelligence technology has entered the deep learning stage from shallow recognition. Speech recognition technology based on deep learning theory and neural network models can output recognition results with higher accuracy, so it is widely used in many fields such as intelligent voice wake-up, intelligent voice control, and intelligent voice dialogue.

[0004] However, after in-depth research, the inventors found that if artificial intelligence speech recognition technology is used to implement key voice information reminder functions in current remote audio and video applications, there are many technical bottlenecks, such as:

[0005] First, in artificial intelligence speech recognition technology, the speech recognition model is the key to ensuring recognition accuracy. In various current intelligent voice wake-up, intelligent voice control, and intelligent voice dialogue technology applications, a universal version of the speech recognition model is often used, that is, the device / application provider pre-trains the speech recognition model, and the judgment criteria for important information and the selection of training samples are all determined by the device / application provider. If this universal version of the speech recognition model is simply used in remote audio and video applications, it will be difficult to adapt to various application scenarios, and may even lead to a poor user experience due to the inability to guarantee recognition accuracy.

[0006] Secondly, the implementation of artificial intelligence speech recognition technology, especially deep learning technology, requires a lot of high-precision calculations, which rely on the strong support of hardware systems in terms of memory, computing overhead and power consumption. Therefore, most of these technologies are currently used on large-scale dedicated computing platforms with high cost, high power consumption and high performance, such as GPU and FPGA, while independent keyword recognition that does not rely on mobile phones or the cloud is very rare in various low-power, low-performance embedded devices (such as headphones, portable speakers, phone watches, conference terminal devices, etc., which are attached to mobile phones or computers) used by ordinary consumers. Or only simple isolated words or fixed keyword sets, limited sentence pattern recognition and other technologies are used to realize some simple and low-level speech recognition functions, such as simple voice wake-up, smart home voice control, etc., but key voice information reminder functions cannot be realized in complex and continuous voice streams. For example, some well-known intelligent voice assistants on the market all upload the voice stream collected by the embedded device to the mobile phone or the cloud for recognition, which usually can only realize the voice recognition of a single sentence; and uploading the collected voice stream to the cloud or remote device for recognition and then returning the result usually leads to poor user experience due to long delays, and the user's privacy is difficult to be protected. The reason for this is that the hardware computing power and power consumption of embedded devices are limited, which makes it difficult to provide sufficient support for the current large-vocabulary continuous speech recognition technology.

[0007] Thirdly, the current voice recognition in the consumer field is to perform certain interactions after recognizing keywords or full speech of the locally input voice stream, but lacks the function of recognizing interesting keywords in the voice from the other direction / from the remote end and providing reminders in specific scenarios.

[0008] Therefore, it is necessary to propose a scene-based key information reminder technology to solve at least one of the above-mentioned technical defects. Summary of the invention

[0009] In view of this, the present invention proposes a key information reminder method, device, system and embedded audio playback device, which can effectively remind users to pay attention to key information.

[0010] To achieve the above object, as a first aspect of the present invention, an embedded audio playback device is provided, comprising a speaker and a communication unit, and further comprising a control unit, a storage unit, a voice recognition unit and a reminder unit.

[0011] The communication unit receives an audio stream from a remote end;

[0012] The speech recognition unit includes a keyword recognition model unit, and the keyword recognition model unit is used to store a scene-based keyword recognition model;

[0013] The keywords are associated with application scenarios, and include a group of words that need to be focused on in the application scenario, and one or more of the words are pre-specified by the user;

[0014] The speech recognition unit extracts a speech signal from the audio stream, and uses the scene-based keyword recognition model to detect in real time whether the speech signal contains the keyword;

[0015] The control unit is used to start recording the received audio stream when the voice signal contains keywords, and control the reminder unit to output key information reminders;

[0016] The storage unit is used to store the recorded audio stream;

[0017] The speaker is used to play the audio stream, or to play back the recorded audio stream in response to a playback instruction.

[0018] Preferably, the scenario-based keyword recognition model may be obtained by training with a deep learning algorithm based on a training sample library containing speech samples for the keyword and / or speech samples of a specific person for the keyword in advance;

[0019] The control unit may also be configured to download the scenario-based keyword recognition model from a remote end via the communication unit.

[0020] Preferably, the speech recognition unit may also include a speech preprocessing unit for preprocessing the input audio stream to eliminate noise, background human voice, and music sound, and extract the speech signal;

[0021] Preferably, the speech recognition unit may also include a neural network processing unit, which is used to perform data processing on the speech signal or the speech signal processed by the speech preprocessing unit based on the keyword recognition model using a deep learning algorithm, thereby inferring and judging the words appearing in the speech signal to determine whether it contains keyword words.

[0022] Preferably, the reminder unit may be one or more of an indicator light module, a vibrator module, a text message generation module, a voice message generation module, and a music message generation module.

[0023] Furthermore, it may also include an input unit for receiving a recording stop instruction and a playback instruction input by a user;

[0024] When the voice signal contains a keyword, the control unit may start to continuously compress and encode the received audio stream and store it locally;

[0025] The control unit may stop recording when receiving a recording stop instruction or the continuous recording time exceeds a first predetermined time length;

[0026] The control unit may play the locally stored recorded audio stream when receiving a local audio playback instruction;

[0027] Furthermore, the control unit may be further configured to send a recording start instruction to the remote end when the voice signal contains a keyword, so as to enable the remote end to start continuously recording the sent audio stream, and to send a recording stop instruction to the remote end when the continuous recording time does not exceed a second predetermined time and a stop recording instruction is received;

[0028] When receiving the remote audio playback instruction, the control unit may send a playback request to the remote end, and receive and play the recorded audio stream stored in the remote end.

[0029] Preferably, the embedded audio playback device is a headset or a speaker with a call function.

[0030] As a second aspect of the present invention, a key information reminder system is provided, comprising an embedded audio playback device and a remote device.

[0031] The remote device receives a keyword vocabulary defined by a user and / or a speech sample of a specific person provided by the user and containing at least the keyword, so as to obtain a scenario-based keyword recognition model; the keyword is associated with an application scenario, and includes a group of words that need to be focused on in the application scenario;

[0032] The scenario-based keyword recognition model is obtained by training in advance based on a training sample library containing speech samples for the keyword and / or speech samples of a specific person for the keyword;

[0033] The embedded audio playback device communicates with the remote device, receives an audio stream from the remote device, and plays the audio stream;

[0034] The embedded audio playback device also obtains a voice signal from the audio stream, and uses a scene-based keyword recognition model to perform voice recognition on the voice signal, and detects in real time whether the voice signal contains keywords;

[0035] When the voice signal contains a keyword, the embedded audio playback device generates a key information reminder and starts recording the received audio stream;

[0036] The embedded audio playback device plays the recorded audio stream in response to the playback instruction.

[0037] Furthermore, cloud servers may also be included.

[0038] The remote device communicates with the cloud server and sends the keyword and / or the voice sample of the specific person to the cloud server;

[0039] The cloud server uses the received keywords and / or the speech samples of the specific person to expand its standard sample library to form a training sample library, and based on the training sample library, uses a deep learning algorithm to train to obtain the scenario-based keyword recognition model;

[0040] The remote device receives the scenario-based keyword recognition model from the cloud server, and downloads the scenario-based keyword recognition model to the embedded audio playback device.

[0041] Preferably, the remote device uses the keyword vocabulary input by the user and / or the voice sample of a specific person provided by the user that at least contains the keyword to expand the standard sample library to form a training sample library, and based on the training sample library, uses a deep learning algorithm to train to obtain the scenario-based keyword recognition model;

[0042] The remote device downloads the scenario-based keyword recognition model to the embedded audio playback device.

[0043] As a third aspect of the present invention, a key information reminder method is provided, wherein:

[0044] Receive user-defined keyword vocabulary and / or a user-provided voice sample of a specific person containing at least the keyword; the keyword is associated with an application scenario and includes a group of vocabulary that needs to be focused on in the application scenario;

[0045] Based on a training sample library containing speech samples for the keyword and / or speech samples of a specific person for the keyword, training to obtain the scenario-based keyword recognition model;

[0046] When receiving and playing an audio stream, obtaining a voice signal from the audio stream;

[0047] Using the scenario-based keyword recognition model to perform speech recognition on the speech signal, and detecting in real time whether the speech signal contains keywords;

[0048] When the voice signal contains keywords, a key information reminder is generated, and the received audio stream is started to be recorded;

[0049] In response to the playback instruction, the recorded audio stream is played.

[0050] Preferably, a wide range of speech samples are collected in advance to form a standard sample library;

[0051] Acquire a voice sample containing at least the keyword according to the keyword;

[0052] The speech samples containing the keywords and / or the speech samples of the specific person are expanded to the standard sample library to form a training sample library, and the scene-based keyword recognition model is obtained by training with a deep learning algorithm based on the training sample library.

[0053] Furthermore, the step of obtaining a voice signal from the audio stream may also include a preprocessing step of eliminating noise, music, and background human voices;

[0054] Preferably, the use of the scene-based keyword recognition model to perform speech recognition on the speech signal or the preprocessed speech signal, and to detect in real time whether the speech signal contains keywords, may specifically include: constructing a deep learning neural network based on the keyword recognition model, and continuously inputting the speech signal into the deep learning neural network for data processing, so as to infer and judge the words appearing in the speech signal to determine whether it contains keyword words.

[0055] Preferably, the recording of the received audio stream may specifically include: when the voice signal contains a keyword, starting to continuously compress and encode the received audio stream and storing it locally;

[0056] When a recording stop instruction is received or the continuous recording time exceeds a first predetermined time, the local recording is stopped;

[0057] The playing of the recorded audio stream in response to the playback instruction specifically includes: playing the locally stored recorded audio stream in response to the playback local audio instruction.

[0058] Preferably, the recording of the received audio stream may specifically include: when the voice signal contains a keyword, sending a recording start instruction to the remote end, so that the remote end starts to continuously record the sent audio stream and performs remote storage;

[0059] When the continuous recording time does not exceed the second predetermined time and a stop recording instruction is received, a recording stop instruction is sent to the remote end, and the remote end stops recording;

[0060] The playing of the recorded audio stream in response to the playback instruction specifically includes: sending a playback request to the remote end in response to the playback remote audio instruction, and receiving and playing the recorded audio stream stored in the remote end.

[0061] Preferably, the key information reminder may be a combination of one or more forms of visual reminder, tactile reminder and auditory reminder;

[0062] The visual reminder includes light effect reminder and remote text message reminder;

[0063] The tactile reminder includes a vibration reminder;

[0064] The auditory reminder includes voice reminder, music reminder

[0065] The beneficial effects of the present invention are as follows: relative to the prior art, the embedded audio playback device provided by the present invention can independently complete the real-time detection, reminder, recording and playback of continuous voice key information based on the scene, and is easy to use, convenient and private. The key information reminder system and reminder method provided by the present invention are pre-trained to obtain a keyword recognition model that is highly compatible with the application scene based on the actual needs of the current application scene and the keywords or training samples customized by the user, so that the accuracy of identifying key information in the continuous voice stream can be effectively improved, and the information that should be focused on in the current application scene and the information that the user is interested in can be output and saved in a timely manner, and the user experience is excellent. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0067] Figure 1 This is a flowchart of a key information reminder method according to Embodiment 1 of the present invention;

[0068] Figure 2 is a circuit principle block diagram of an embedded audio playback device according to Embodiment 2 of the present invention;

[0069] Figure 3 This is a system architecture diagram of the key information reminder system of Example 3 of the present invention. DETAILED DESCRIPTION

[0070] In the more than 40 years of application of Moore's Law, semiconductor chip design technology and manufacturing process levels have been rapidly improved, chip computing power has been greatly improved, on-chip storage capacity has been greatly improved, and at the same time power consumption has been continuously reduced, which makes it possible for artificial intelligence technology to be widely used in small, low-power embedded devices. The present invention is a technical improvement proposed to address the defect in the prior art that people easily miss important information from the other end when using remote audio and video applications. Specifically, on embedded devices, for voice information, scene-based artificial intelligence voice recognition technology is used to identify interesting information from the other end in real time, and timely output reminders and save key audio streams. The present invention can be applied to different application scenarios to meet the personalized needs of different users, and therefore can effectively solve the defects of the prior art. The "real-time" mentioned in this article means that the embedded audio playback device has sufficient computing power to identify keywords in the audio stream played at the original speed.

[0071] The technical solution of the present invention is further described exemplarily below through the accompanying drawings and embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0072] Embodiment 1:

[0073] like Figure 1 As shown, according to the core idea of ​​the present invention, this embodiment provides a key information reminder method, wherein:

[0074] Step 100, initialization step.

[0075] This step is the processing flow before entering the key information reminder, which is mainly used to check and update the software and hardware environment configuration, parameter preparation, program preparation, etc. required for the key information reminder. It may include the step of establishing a communication connection between the local device and the remote device using wireless communication or wired communication, and may also include the step of obtaining a scenario-based keyword recognition model.

[0076] It should be noted that the "local" and "remote" referred to in this article are relative concepts, where "local" refers to the party or end that receives the audio stream and generates key information reminders, while "remote" refers to the other party or end that is independent of the "local" but communicates with the "local" directly or indirectly through one or more media in a wired or wireless manner, and sends the audio stream to the "local". In addition, it should be noted that the "remote" referred to in this article is not the same concept as the "opposite end" commonly used in the field to describe voice calls. The "opposite end" refers to the other initiator of the audio stream, while the "remote end" refers to the initial recipient of the call audio stream after it is initiated by the opposite end. Correspondingly, "local" refers to the final recipient of the call audio stream.

[0077] As a specific implementation, "local" can be an audio playback device based on an embedded system (abbreviated as: embedded audio playback device). The "embedded system" referred to herein is a special-purpose computer system embedded in an object system, which is application-centric, based on computer technology, and has tailorable software and hardware, and is suitable for application systems with strict requirements on functions, reliability, cost, volume, and power consumption. "Embedded device" refers to a device that contains an embedded system internally, which is generally based on the ARM kernel and architecture or other low-power kernels and architectures, and is used to achieve specific functions and applications. It is a device relative to a general-purpose PC with multiple functions, and can specifically be headphones, speakers, phone watches, conference terminal equipment, etc. "Remote" can be an end-user computer system, a network server or server system, a mobile computing device, a consumer electronic device or other appropriate electronic device or any combination or part thereof, such as a mobile phone, a tablet computer, a computer, a smart TV, etc.

[0078] Remote audio and video applications can be applied to a variety of scenarios, and the amount of voice information transmitted is huge. The categories of key information vary from person to person and from scenario to scenario. For example, people often participate in video conferences or online classes through headphones or speakers, which are accessories of mobile phones or computers. In video conferences, users may be most concerned about the content related to themselves in the meeting, such as their department, their boss, and the business related to themselves. Therefore, the keywords used to identify key information should be department name, boss's name, their own name, business name, task assignment, delivery deadline, etc.; in online classes, students may be most concerned about the knowledge points taught by the teacher, so the keywords used to identify key information should be key points, difficult points, test points, summary, review, etc.; in the customer service center, customer service personnel may be most concerned about the complaints mentioned by customers, so the keywords need to include complaints, suggestions, quality, service attitude, etc. If a full-text speech recognition model is used in these different scenarios and the recognition accuracy is to be guaranteed, the speech recognition model must be trained based on massive speech samples. However, this is usually difficult to achieve in mobile phone or computer accessories. On the one hand, it is difficult to obtain massive voice samples. On the other hand, training based on massive voice samples places very high demands on computer hardware. The high implementation cost limits the promotion and application of technology in mobile phone or computer accessories.

[0079] Therefore, in this embodiment, the step of obtaining the scenario-based keyword recognition model, especially obtaining the keyword recognition model in an attached embedded device of a mobile phone or computer, is used to adjust and update the keyword recognition model according to the actual application scenario, so that the keyword recognition model is more suitable for the current scenario and meets user needs. The keywords are associated with the application scenario and include a group of words that need to be focused on in the application scenario. Different application scenarios can correspond to different keywords. Users can set and specify one or more words in the keywords according to actual needs.

[0080] The step of obtaining the scenario-based keyword recognition model may specifically include: receiving a keyword vocabulary customized by a user, and / or receiving a speech sample of a specific person provided by a user that at least contains the keyword; using the keyword and the speech sample of the specific person to expand a standard sample library to form a training sample library, and obtaining a scenario-based keyword recognition model through training based on the training sample library. The standard sample library may be a training sample set formed based on a wide range of pre-collected speech samples.

[0081] The steps of receiving a user-defined keyword vocabulary and / or receiving a user-provided voice sample of a specific person containing at least the keyword are usually performed remotely based on a richer user interface on a mobile phone or computer.

[0082] As an optional implementation, the user can set a custom keyword set in advance through the remote end according to his or her preferences, needs, and usage scenarios; the provider of the audio stream can also generate a default keyword set based on various factors such as usage scenarios, audio stream content, and user usage habits. The remote end can also display several default keyword words in advance for the user to select, add, or delete, so as to form a keyword word set associated with the application scenario.

[0083] In order to match the hardware environment of the embedded device, an upper limit of the number of words in the keyword may be set, such as 30 groups of words.

[0084] In addition, in view of the fact that various factors such as the speaker's gender, age, physiological characteristics of pronunciation, dialect, non-native pronunciation, emotion when speaking, and environmental noise may affect the accuracy of recognition in speech recognition, for example, the pronunciation of the same word "key point" is very different between Sichuanese and Cantonese. Therefore, in this embodiment, a speech sample with a specific accent containing at least the key words provided by the user can also be obtained and used to expand the standard sample library. For example, students can provide some recordings of teachers teaching, and employees can provide a recording of the boss in a meeting, etc.

[0085] After obtaining the keywords set by the user, the keywords are used to select from the existing massive standard voice sample library to form a training sample library; after receiving the voice sample of a specific person provided by the user that at least contains the keywords, the voice sample of the specific person is also used to expand the training sample library. This embodiment obtains a scenario-based keyword recognition model based on the sample library containing these voice samples closely related to the application scenario, which can effectively improve the recognition accuracy.

[0086] The process of training to obtain the keyword recognition model in this embodiment can be implemented by using the Hidden Markov Model (HMM), Dynamic Topic Models (DTM), and various classical artificial intelligence speech recognition algorithms derived from such technologies that have been successfully used for speech recognition and text recognition, or by using algorithms based on deep learning and various related algorithms in the future. Deep learning is one of the important fields in machine learning research. Its motivation is to establish and simulate a neural network for human brain analysis and learning, and to interpret data such as images, sounds, and texts by imitating the mechanism of the human brain. The core of deep learning is to learn more useful features by building a machine learning model with multiple hidden layers and a large amount of training data, thereby ultimately improving the accuracy of classification or prediction. At present, in computer vision and natural language, the mainstream deep learning algorithms are convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory network (LSTM), deep full sequence convolutional neural network (DFCNN), etc. In terms of implementation, this embodiment may adopt various applicable deep learning algorithms including but not limited to these existing or future ones.

[0087] As a preferred implementation, this embodiment adopts a continuous speech keyword recognition technology based on a deep learning algorithm. For example, after obtaining the training sample library, a deep learning algorithm such as a convolutional neural network (CNN) algorithm or a recurrent neural network (RNN) algorithm is used to obtain a scene-based keyword recognition model based on the training sample library.

[0088] The training process of the keyword recognition model using the deep learning algorithm can be completed remotely or in the cloud. It should be noted that the "cloud" mentioned in this article refers to a cloud computing server or a cloud computing backend server with powerful processing and storage capabilities. As a preferred implementation, the training process is completed in the cloud so as to make full use of the hardware resources and powerful computing power of the cloud. Specifically, it includes: after the user inputs the keyword vocabulary or uploads the voice sample of a specific person containing the keyword at the remote end, the remote end sends the keyword and / or voice sample to the cloud, so that the cloud can obtain the voice sample including the keyword in various ways such as from the Internet, and expand the voice sample and the voice sample of the specific person into its standard sample library to form a training sample library, and then use the training sample library to train to obtain a scene-based keyword recognition model.

[0089] Furthermore, after the training is completed, the scenario-based keyword recognition model is received remotely from the cloud.

[0090] The process of completing the training remotely can refer to the above-mentioned cloud training process, which will not be repeated here.

[0091] The initialization step may also include a step of updating the local keyword recognition model, which specifically includes: the remote end downloads the scene-based keyword recognition model to the local end. The remote end may actively send an update request to the local end, or the remote end may initiate the download execution in response to the local update request.

[0092] After completing the initialization steps, you can enter the following key information real-time detection and reminder process.

[0093] Step 110, when receiving and playing an audio stream, obtaining a voice signal from the audio stream;

[0094] The key information reminder process of this embodiment is to identify the key information contained in the voice information and make a reminder while receiving and playing the audio stream in the remote audio and video application in the remote audio and video application.

[0095] As a preferred implementation, this step also performs background sound elimination on the audio stream when acquiring the voice signal in the audio stream, removes background noise, background human voices, music sounds, etc., and extracts a foreground voice signal with a high signal-to-noise ratio to improve the signal-to-noise ratio, thereby improving the success rate of voice recognition.

[0096] Step 120, using the scenario-based keyword recognition model to perform speech recognition on the speech signal, and detecting in real time whether the speech signal contains preset keywords;

[0097] When detecting whether the speech signal contains keywords, as long as a group of keyword words are detected, it can be considered that the speech signal contains keywords.

[0098] As a preferred implementation, continuous speech keyword recognition technology based on a deep learning algorithm is used to perform scene-based key information recognition, specifically including constructing a deep learning neural network based on the keyword recognition model, inputting the continuous speech signal to be recognized into the deep learning neural network for data processing, so as to infer and judge the words appearing in the speech signal to determine whether it contains keyword words.

[0099] In this embodiment, a continuous speech keyword recognition technology based on a deep learning algorithm is used to perform scene-based key information recognition. Compared with the large-vocabulary continuous speech recognition in the prior art, it is not necessary to recognize all the text, but only to detect whether one or more groups of keywords set by the user appear in the continuous speech stream. On the one hand, real-time detection can be performed on the continuous speech stream. On the other hand, it has low requirements on the hardware's computing power, storage space and power consumption, and can be applied to small and low-power embedded systems. At the same time, scene-based recognition can effectively improve the recognition accuracy and enhance the user experience of speech recognition.

[0100] When it is detected that the voice signal does not contain the keyword, the process returns to step 110 to continuously detect the subsequently acquired audio stream.

[0101] When the voice signal contains at least one set of keywords, executing steps 130 and 140;

[0102] Step 130, generating a key information reminder.

[0103] The key information reminder may include a visual reminder, a tactile reminder, and an auditory reminder;

[0104] The visual reminder includes light effect reminder, text message reminder, such as LED indicator flashing or displaying specific light effect, flashing pattern on the remote screen, remote text message (such as notification message of mobile phone application APP), etc.;

[0105] The tactile reminder includes a vibration reminder, such as a ringing according to a predetermined pattern;

[0106] The auditory reminder includes a voice reminder and a music reminder, such as using a predetermined voice content or music for reminder.

[0107] In specific implementation, one or several of the above reminder methods can be selected according to the actual application scenario. For example, only light effect reminder or music reminder can be set, or a message can be sent to the associated computer application APP at the same time as the vibration reminder to achieve a double reminder effect.

[0108] Step 140, start recording the received audio stream;

[0109] In this embodiment, when it is determined that the voice information of the current audio stream contains keywords, in order to help the user miss as few important contents as possible, while generating a reminder, the recording of the received audio stream is also started.

[0110] When recording the audio stream, the keyword itself can be used as the recording starting point, or the audio stream received after the keyword appears can be used as the recording starting point, or a segment of the audio stream that has been rolling compressed and encoded in the current audio stream when the keyword appears can be moved forward by a fixed time from the keyword as the starting point as the recording starting point. That is, the recorded audio stream can include the audio stream when the keyword appears, or it may not include it, and it may also include the audio stream before the keyword appears.

[0111] The recorded audio stream will be compressed and encoded and stored locally for local playback. The audio stream is continuously recorded until a recording stop command is received or the continuous recording time exceeds a first predetermined time, then the recording stops. The first predetermined time takes into account the limited capacity of the local storage medium and can be set to be relatively short, such as 1 to 2 minutes. Usually, important content will appear in the voice information in a short time after the keywords appear. Therefore, the shorter first predetermined time, although brief, may save the most important voice content, making it easier for users to quickly understand important information when playing back the recording.

[0112] As a preferred mode, when the voice signal contains keywords, a recording start instruction can also be sent to the remote end, which can enable the remote end to start continuously recording the audio stream sent by it and store it remotely; after sending the recording start instruction, the timing can also be started. When the calculated remote end continuous recording time does not exceed the second predetermined time and the recording stop instruction is received, the recording stop instruction is sent to the remote end, so that the remote end can terminate the recording at any time within the second predetermined time due to receiving the recording stop instruction, thereby increasing the controllability of the recording time. When the continuous recording time of the remote end exceeds the second predetermined time, the recording can be automatically stopped.

[0113] In order to help users fully grasp important information and reduce information omissions as much as possible, the second predetermined time can be set to be greater than or equal to the first predetermined time, that is, the second predetermined time is relatively long, such as 2 to 5 minutes, so that a longer audio stream containing key information can be saved for user playback.

[0114] Of course, if the local storage space is large enough, the first predetermined duration can also be set to be greater than or equal to the second predetermined duration. In this way, the local storage can store recordings of sufficiently long duration, while the remote end can retain recordings of shorter duration, so that the user or other person can play back the recordings remotely to quickly understand key information.

[0115] In addition, as an optional implementation, when recording an audio stream, the remote end may also perform full-text speech recognition on the recorded audio stream to obtain corresponding text and store the text information.

[0116] Step 150: In response to the playback instruction, play the recorded audio stream.

[0117] In this step, in response to the playback local audio instruction, the locally recorded and stored audio stream can be played; or in response to the playback remote audio instruction, a playback request can be sent to the remote end, and the recorded audio stream stored remotely can be received and played.

[0118] As an optional implementation, when the recorded audio streams are stored locally, they can be stored in the order of the recording start time; correspondingly, when they are played back, they can be played in the order of the recording start time.

[0119] It should be noted that step 104 is a step executed based on the obtained playback designation, so it does not have to be implemented after step 103, or it can be implemented to detect the playback instruction at any time during use to play back the recording.

[0120] In a typical application scenario, the key information reminder method of this embodiment can be applied in a call center system. Usually, the operators of a call center have to answer hundreds of voice calls every day, and the work intensity is very high. However, due to differences in verbal expression ability, accent problems, or even emotional problems, it is often difficult for the other party to clearly express their main purpose of the call in a short time. If the operator cannot concentrate highly, it is easy to miss the other party's important information, or even misunderstand the other party's meaning, which leads to adverse consequences. Using the method of this embodiment, when the operator answers a call, the headset that can remind the key information automatically identifies whether the voice information of the other party contains keywords such as "alarm", "complaint", "liar", and promptly reminds the operator to pay attention to the key information; and the headset can also record the key information, or notify the remote end (such as the call center management platform, call transfer platform, etc.) connected to it to record the key information. In this way, the operator can understand the key information more accurately and comprehensively through the playback function, and deepen the understanding of the other party's call intention. It can be seen that the key information reminder method of this embodiment can not only remind the operator in a timely and effective manner, but also help him review the call content, which not only reduces information loss, but also greatly reduces the operator's work pressure.

[0121] Example 2

[0122] refer to Figure 2 According to the core concept of the present invention, this embodiment provides an embedded audio playback device, including a communication unit, a speaker, a control unit, a storage unit, a voice recognition unit and a reminder unit.

[0123] The storage unit is used to store data, programs, etc. related to the operation of the device.

[0124] The communication unit may be a wired communication unit or a wireless communication unit, and may include both a wired communication module and a wireless communication module. Specifically, the communication unit may be implemented as a Bluetooth communication unit, a WIFi communication unit, an Internet network interface, an audio-specific wired transmission interface, a USB interface, a micro USB interface, a mini USB interface, a Type-C interface, a Lightning interface, and other known or future communication units that can be used in this embodiment.

[0125] The communication unit receives an audio stream from a remote end;

[0126] The speech recognition unit is used to extract a speech signal from the audio stream, and use a scene-based keyword recognition model to detect in real time whether the speech signal contains a keyword;

[0127] The control unit is the control center of the device, which uses various interfaces and lines to connect other units in the device, and monitors and schedules each unit as a whole to realize various functions of the device, especially when the voice signal contains keywords, it starts recording the received audio stream and controls the reminder unit to output key information reminders;

[0128] In this embodiment, the keyword is associated with an application scenario, which includes a group of words that need to be focused on in the application scenario, and one or more of the words are pre-specified by the user;

[0129] The speech recognition unit includes a keyword recognition model unit, and the keyword recognition model unit is used to store the scenario-based keyword recognition model. The scenario-based keyword recognition model is obtained by training based on a training sample library containing speech samples for the keyword and / or speech samples of a specific person for the keyword; as a preferred embodiment, the scenario-based keyword recognition model is obtained by training using a deep learning algorithm, and the speech recognition unit can use the keyword recognition model to perform continuous speech keyword recognition to detect in real time whether the speech signal contains keywords;

[0130] The speech recognition unit may also include a speech preprocessing unit for preprocessing the input audio stream to eliminate noise, music, background human voice, etc., and extract a speech signal with a high signal-to-noise ratio;

[0131] The speech recognition unit may also include a neural network processing unit for performing data processing on the speech signal using a deep learning algorithm based on the keyword recognition model, thereby inferring and judging the words appearing in the speech signal to determine whether the words contain keyword words. The neural network processing unit may be an embedded neural network processor (neural-network process units, NPU), a dedicated neural network processing array processing unit, a DSP, an embedded processor, or other processing modules that can be used to process massive multimedia data in a neural network.

[0132] In this embodiment, the keyword recognition model is trained externally and downloaded into the device before use. Therefore, the control unit is also used to download the scenario-based keyword recognition model from a remote end via the communication unit.

[0133] The reminder unit is one or more of an indicator light module, a vibrator module, a text message generation module, a voice message generation module, and a music message generation module. The indicator light module may be an LED indicator light, which may output a reminder by flashing light or displaying a specific graphic; the vibrator module may generate a vibration of a predetermined frequency; the text message generation module may generate a text message according to a predetermined message format, such as a text message containing a currently recognized keyword; the voice message generation module may generate a voice message according to a predetermined voice message format, such as a voice message containing a currently recognized keyword; the voice message generation module may select a segment from pre-stored voice data according to a preset method as a voice message, such as a "di di" or "ding dong" tone.

[0134] The speaker is used to play the audio stream, or play back the recorded audio stream, or play the voice message or sound message, etc. It should be understood that in some specific embodiments, the speaker can cooperate with the control unit and the storage unit to replace the function of the reminder unit, for example, only using the sound reminder method.

[0135] The embedded audio playback device further includes an input unit for receiving various control instructions input by a user, for example, receiving a playback instruction, a stop reminder instruction, a recording stop instruction, etc. input by a user.

[0136] The input unit may be any mechanical or voice input module such as a touch panel, a button, or a voice command input module.

[0137] The storage unit is used to store the recorded audio stream;

[0138] In an optional implementation, when the voice signal contains a keyword, the control unit starts to continuously compress and encode the received audio stream and store it locally; when the control unit receives a recording stop instruction or the continuous recording time exceeds a first predetermined time, the control unit stops recording;

[0139] The control unit plays the locally stored recorded audio stream when receiving the instruction to play back the local audio;

[0140] The control unit is further configured to send a recording start instruction to the remote end when the voice signal contains a keyword, so as to enable the remote end to start continuously recording the sent audio stream, and to send a recording stop instruction to the remote end when the continuous recording time does not exceed a second predetermined time and a stop recording instruction is received;

[0141] When receiving the remote audio playback instruction, the control unit sends a playback request to the remote end, and receives and plays the recorded audio stream stored in the remote end.

[0142] In addition, the embedded audio playback device may also include a power supply unit, which is used to provide the power required for the device to work. It can be a power supply circuit module powered by a button battery or a rechargeable battery, or a power supply management module that powers the device through an external input power supply, or a circuit module that draws power based on a wired communication interface.

[0143] Obviously, the embedded audio playback device of this embodiment can be used to implement part or all of the methods, processes or steps in the key information reminder method described in Example 1. The description of the same or similar parts as those in Example 1 will not be repeated in this embodiment.

[0144] The embedded audio playback device can be specifically implemented as a head-mounted audio playback device, such as various wired earphone devices, wireless earphone devices, etc., and can also be implemented as various portable speakers; it can also be specifically implemented as a telephone watch, a portable game console, a portable multimedia player, and other mobile phone or computer accessory devices. For example, in a typical application scenario, the embedded audio playback device is a speaker with a call function. An LED indicator light is provided on the shell of the speaker, and a scene-based keyword recognition model is pre-loaded inside the speaker, which can perform real-time continuous detection of the voice information currently played by the speaker. When the current voice information contains keywords, the LED indicator light starts to flash to remind the user. The speaker has an intelligent voice control function, and the user can issue a control command by voice to control the speaker to execute functions such as turning off the LED indicator light, stopping recording, and replaying. The detailed process of the speaker for realizing key information reminders can refer to the aforementioned embodiment 1 and part of the description of this embodiment, which will not be repeated here.

[0145] Example 3

[0146] According to the core idea of ​​the present invention, this embodiment provides a key information reminder system, including an embedded audio playback device and a remote device,

[0147] The remote device receives a keyword vocabulary defined by a user and / or a speech sample of a specific person provided by the user and containing at least the keyword, so as to obtain a scenario-based keyword recognition model; the keyword is associated with an application scenario, and includes a group of words that need to be focused on in the application scenario;

[0148] The scenario-based keyword recognition model is obtained by training in advance based on a training sample library containing speech samples for the keyword and / or speech samples of a specific person for the keyword;

[0149] The embedded audio playback device communicates with the remote device, receives the audio stream from the remote device, and plays it; the communication can be any applicable communication form, such as wired (e.g. Ethernet, USB, lightning, optical fiber) communication or wireless (e.g. WiFi, Bluetooth, IR) communication.

[0150] The embedded audio playback device also obtains a voice signal from the audio stream, and uses a scene-based keyword recognition model to perform voice recognition on the voice signal, and detects in real time whether the voice signal contains keywords;

[0151] When the voice signal contains a keyword, the embedded audio playback device generates a key information reminder and starts recording the received audio stream;

[0152] The embedded audio playback device plays the recorded audio stream in response to the playback instruction.

[0153] As an optional manner, the key recognition model is trained on the remote device, and the remote device uses the user-defined keyword vocabulary and / or the voice sample of a specific person provided by the user that at least contains the keyword to expand its standard sample library to form a training sample library, and obtains a scene-based keyword recognition model based on the training sample library;

[0154] The remote device downloads the scenario-based keyword recognition model to the embedded audio playback device.

[0155] As another optional implementation, the keyword recognition model is trained in the cloud, and the system further includes a cloud server;

[0156] The remote device communicates with the cloud server and sends the keyword and / or the voice sample of the specific person to the cloud server;

[0157] The cloud server uses the received keywords and the speech sample of the specific person to expand its standard sample library, and obtains a scene-based keyword recognition model based on the training sample library;

[0158] The remote device receives the scenario-based keyword recognition model from the cloud server, and downloads the scenario-based keyword recognition model to the embedded audio playback device.

[0159] Obviously, the key information reminder system provided in this embodiment can be used to implement some or all of the methods, processes or steps in the key information reminder method described in Example 1. The embedded audio playback device described in Example 2 can also be used to implement the key information reminder system of this embodiment. Similar technical details can be referred to the description of the aforementioned embodiment, which will not be repeated here.

[0160] A typical application scenario is taken as an example below to describe the core idea of ​​the embodiment of the present invention more clearly and in detail.

[0161] refer to Figure 3 In this application scenario, the key information reminder system includes a video playback device (such as a tablet computer) 300, headphones 310 and a cloud server 320.

[0162] The earphone 310 can be a headset, an in-ear earphone or an earhook earphone; it can be a wired earphone or a wireless earphone; it can have only one earphone 311 or can have left and right earphones 311; the left and right earphones 311 can be connected or separated.

[0163] The earphone 310 communicates with the video playback device 300 by wire or wirelessly, thereby receiving an audio stream from the video playback device 300. The video playback device 300 may be a user's personal computer, tablet computer, smart TV, mobile phone, etc. The user watches video programs through the video playback device 300. Figure 3 Shown is a student watching an online class on a tablet computer.

[0164] The video playback device 300 may also access the cloud server 320 based on a network, and the network may be a local area network, a wide area network, a cellular network, or a combination thereof.

[0165] The earphone 310 is provided with an LED indicator 312 and buttons 313 to 316. The LED indicator 312 can emit a flashing red light; the button 313 is a volume increase button, the button 314 is a play / pause button, the button 315 is a stop reminder / stop recording / playback button, and the button 316 is a volume reduction button. The button 315 can be set to execute the three functions of stop reminder, stop recording, and start playback at the same time by pressing it once, or it can be set to execute stop reminder and recording at the same time by pressing it once, and start playback by pressing it twice in a row. The specific setting can be based on the actual implementation environment, and the present invention does not impose specific restrictions on this.

[0166] The LED indicator light 312 may also be disposed on an external microphone (not shown) of the headset 310. When the user wears the headset, the external microphone may be adjusted to a position in front of the user's lips, so that the LED indicator light 312 can be more easily seen by the user as a light reminder.

[0167] In addition, a vibrator (not shown) is also provided in the earphone 310. The vibrator can be implemented by using existing or future applicable technologies, and the present invention does not make any specific restrictions. For example, it can be an eccentric motor with a cam.

[0168] The cloud server 320 can generate a keyword recognition model based on the aforementioned deep learning algorithm training. In specific implementation, the cloud server 320 can pre-collect a wide range of voice samples, and perform vocabulary tagging and other processing on the voice samples to form a standard sample library.

[0169] In this application scenario, the key information reminder system implements the following process for key information reminders:

[0170] Step 1: Initialization step.

[0171] Before starting the key information reminder process, perform the initialization steps to check and update the software and hardware environment configuration and various parameter settings required for the operation and communication of each device and equipment in the system.

[0172] This includes setting keywords and obtaining a new keyword recognition model. Specifically:

[0173] The user sets keywords through the video playback device 300. For example, before taking an online class, a student can enter words such as "key points", "exams", "summary" and his or her own name as keywords. Through the user's independent settings, personalized keywords that meet the current application scenario can be formed.

[0174] In order to match the hardware power consumption and computing power of the headset 310, the upper limit of the number of keyword words is set to 20.

[0175] When a new word is input into the keyword of the video playback device 300 , the video playback device 300 accesses the cloud server 320 , sends a request to update the keyword recognition model to the cloud server 320 , and sends the keyword to the cloud server 320 .

[0176] After receiving the keywords, the cloud server 320 can compare the keyword vocabulary therein with the existing keyword vocabulary on the cloud server 320. When all the vocabulary in the keywords sent by the video playback device 300 are included in the existing keywords on the cloud server 320, the existing standard sample library is directly used as the training sample library, and the deep learning algorithm is used to train the keywords to obtain a new scenario-based keyword recognition model; when some of the vocabulary in the keywords are not included in the existing keywords on the cloud server 320, voice samples containing the part of the vocabulary are obtained from the Internet, and the standard sample library is expanded to form a training sample library, and then a new keyword recognition model is generated by training.

[0177] The user can also upload a specific person's voice sample containing one or more words in the keyword through the video playback device 300, such as a student uploading a teacher's voice audio material to the video playback device 300. The video playback device 300 uploads the specific person's voice sample to the cloud server 320 to expand the standard sample library of the cloud server 320, so that the cloud server 320 can train a new keyword recognition model based on the training sample library of the specific person's voice sample containing at least the keyword.

[0178] In response to the update request of the video playback device 300 , the cloud server 320 sends the trained scene-based keyword recognition model to the video playback device 300 .

[0179] After receiving the keyword recognition model from the cloud server 320 , the video playback device 300 downloads the keyword recognition model to the headset 310 , so that the headset 310 updates its locally stored keyword recognition model.

[0180] It should be noted that the process of setting keywords and obtaining a new keyword recognition model can be completed in the initialization step, or can be completed at various appropriate times during system operation. The specific time can be determined based on actual conditions, and the present invention does not impose any limitation on this.

[0181] Step 2: the earphone 310 receives the audio stream.

[0182] After the system initialization is completed, the user can start receiving and playing the audio stream from the video playback device 300 through the button 314 on the headset 310. For example, the student can watch the online course through the headset 310 and the tablet computer 300 at this time.

[0183] In step three, the earphone 310 obtains the voice signal in the audio stream, performs voice recognition on the voice signal, and uses the scene-based keyword recognition model to detect in real time whether the voice signal contains preset keywords.

[0184] The headset 310 has a built-in speech recognition unit, which may be an embedded neural network processor for constructing a neural network based on the keyword recognition model and using a deep learning algorithm for data processing to perform real-time keyword recognition on continuously input voice signals.

[0185] The audio stream of the online course may include various sound signals such as music and voice. The earphone 310 extracts the voice signal and uses a scene-based keyword recognition model and deep learning algorithm to detect whether the voice signal contains preset keywords. For example, if the student pre-sets the keyword "summary", when the online teacher says "Let's summarize the main content of this class", the current voice signal can be detected and recognized to contain keywords; and if the student also uses his or her name or student number as a keyword, when the online teacher calls on him or her, the earphone 310 can play a good role in assisting reminders.

[0186] When the keyword is not recognized, the earphone 310 continues to receive and play the audio stream, and does not enter the execution of the following steps. It should be understood that when the system performs key information reminders, the process of the earphone 310 receiving and playing the audio stream may not be affected.

[0187] Step 4: the earphone 310 generates a key information reminder and records an audio stream.

[0188] When the headset 310 detects and recognizes that the current voice signal contains preset keywords, it will start vibrating its vibrator. The user can stop the headset 310 from vibrating by pressing button 315. If the vibration exceeds a predetermined vibration time, such as 10 seconds, and the user does not stop the vibration, the vibration can be automatically stopped, and the LED indicator 312 starts to emit a flashing red light. The red light can continue to flash for a long flashing time, or flash until the user stops it by pressing button 315. When the headset 310 detects that the current state of the LED indicator 312 is the working state (flashing red light) before generating a new vibration, no new vibration is generated, but the current working state of the LED indicator 312 is maintained. In this way, if the student is still wearing the headset at this time, the key information can be paid attention to by vibration; if he has taken off the headset, the reminder purpose can be achieved by light effect.

[0189] While generating the key information reminder, the headset 310 also starts recording the received audio stream. Specifically:

[0190] The recorded audio stream is stored locally within the first predetermined time. The first predetermined time should be less than or equal to the maximum duration of the audio stream that can be stored in the headset 310. The first predetermined time can be a preset value. For example, if the maximum duration of the audio stream that can be stored in the headset 310 is 2 minutes, the first predetermined time can be 2 minutes, or the first predetermined time can be 30 seconds, so that the headset 310 can store up to 4 audio streams with a maximum duration of 30 seconds.

[0191] When the earphone 310 starts recording the received audio stream, it also sends a recording start instruction and the detected keyword vocabulary to the video playback device 300 .

[0192] After receiving the recording start instruction sent by the earphone 310, the video playback device 300 starts recording the sent audio stream.

[0193] Step 5: The video playback device 300 converts the recorded voice signal into text information and stores it.

[0194] The video playback device 300 can obtain the voice signal in the recorded audio stream, and convert the full text into text using various speech-to-text conversion methods in the prior art and store it. When storing, the received keywords, words, and recordings detected by the earphone 310 can also be stored in association, so that the user can select and view them later.

[0195] Step 6: Stop recording.

[0196] When the user inputs a recording stop command through the button 315, or when the duration of continuous recording exceeds the first predetermined duration but still no recording stop command issued by the user is received, the headset 310 will automatically stop recording the audio stream.

[0197] When the user inputs a recording stop command through the key 315, or when the continuous recording time exceeds the second predetermined time but still no recording stop command is received from the user, the video playback device 300 will automatically stop recording the audio stream.

[0198] Step 7: Play back the recording.

[0199] In this embodiment, the user can play back the recording on the headset 310 or on the video playback device 300 .

[0200] For example, when a student starts the local playback function by pressing the button 315 twice in succession, the earphone 310 will play the locally stored recorded audio stream while playing the audio stream from the video playback device 300. During playback, the two audio streams can be mixed before playback, or one of the two earphones 311 can play one audio stream and the other earphone can play another audio stream.

[0201] Alternatively, when the student starts the remote playback function by pressing the button 315 three times in succession, the earphone 310 sends a playback request instruction to the video playback device 300 , and after receiving the playback request instruction, the video playback device 300 sends the recorded audio stream to the earphone 310 .

[0202] In addition, students can also directly input playback instructions on the video playback device 300 to play the recorded audio stream stored in the video playback device 300 .

[0203] The student can also specify to play the recorded audio stream on the video playback device 300 .

[0204] Step 8: Check the text message.

[0205] In this step, students can check the text information corresponding to the recorded audio stream in the video playback device 300, so that students can review and take notes based on the text information.

[0206] It can be seen from the description of the above embodiments and typical application scenarios that the key information reminder method, system and embedded audio playback device provided by the embodiments of the present invention realize real-time detection, reminder and playback of key information of continuous voice on a small, low-power embedded device. It is easy to use, simple to operate, and has a wide range of applications. It can effectively remind, save and review key information, reduce the loss of users missing key information, and increase user satisfaction with remote audio and video applications.

[0207] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0208] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented by hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0209] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An embedded audio playback device, comprising a speaker and a communication unit, characterized in that: It also includes a control unit, a storage unit, a voice recognition unit and a reminder unit. The communication unit receives an audio stream from a remote end; The control unit is used to download a scenario-based keyword recognition model from a remote end through the communication unit, wherein the scenario-based keyword recognition model is obtained by training in advance based on a training sample library containing voice samples for the keyword and / or voice samples of a specific person for the keyword; the keyword is associated with an application scenario, which contains a group of words that need to be focused on in the application scenario, and one or more of the words are pre-specified by a user; The speech recognition unit includes a keyword recognition model unit, and the keyword recognition model unit is used to store the scene-based keyword recognition model; The speech recognition unit extracts a speech signal from the audio stream, and uses the scene-based keyword recognition model to perform artificial intelligence speech recognition on the speech signal, and detects in real time whether the speech signal contains the keyword; The control unit is used to start recording the received audio stream when the voice signal contains keywords, and control the reminder unit to output key information reminders; The storage unit is used to store the recorded audio stream; The speaker is used to play the audio stream, or to play back the recorded audio stream in response to a playback instruction.

2. The embedded audio player as claimed in claim 1, characterized in that: The scenario-based keyword recognition model is pre-trained using a deep learning algorithm based on a training sample library containing voice samples for the keyword and / or voice samples of a specific person for the keyword.

3. The embedded audio player as claimed in claim 2, characterized in that: The speech recognition unit also includes a speech preprocessing unit for preprocessing the input audio stream to eliminate noise, background human voice, and music sound, and extract the speech signal; The speech recognition unit also includes a neural network processing unit, which is used to perform data processing on the speech signal or the speech signal processed by the speech preprocessing unit based on the keyword recognition model using a deep learning algorithm, so as to infer and judge the words appearing in the speech signal to determine whether the word contains keyword words.

4. The embedded audio player as claimed in claim 1, characterized in that: The reminder unit is one or more of an indicator light module, a vibrator module, a text message generation module, a voice message generation module, and a music message generation module.

5. The embedded audio player as claimed in claim 1, characterized in that: It also includes an input unit for receiving a recording stop instruction and a playback instruction input by a user; When the voice signal contains a keyword, the control unit starts to continuously compress and encode the received audio stream and store it locally; The control unit stops recording when receiving a recording stop instruction or the continuous recording time exceeds a first predetermined time period; The control unit plays the locally stored recorded audio stream when receiving the instruction to play back the local audio; The control unit is further configured to send a recording start instruction to the remote end when the voice signal contains a keyword, so as to enable the remote end to start continuously recording the sent audio stream, and send a recording stop instruction to the remote end when the continuous recording time does not exceed a second predetermined time and a stop recording instruction is received; When receiving the remote audio playback instruction, the control unit sends a playback request to the remote end, and receives and plays the recorded audio stream stored in the remote end.

6. The embedded audio playback device as claimed in any one of claims 1 to 5, characterized in that: The embedded audio playback device is a headset or a speaker with a call function.

7. A key information reminder system, characterized in that: Including embedded audio playback device and remote device, The remote device receives a keyword vocabulary defined by a user and / or a speech sample of a specific person provided by the user and containing at least the keyword, so as to obtain a scenario-based keyword recognition model; the keyword is associated with an application scenario, and includes a group of words that need to be focused on in the application scenario; The scenario-based keyword recognition model is obtained by training in advance based on a training sample library containing speech samples for the keyword and / or speech samples of a specific person for the keyword; The embedded audio playback device communicates with the remote device, downloads the scene-based keyword recognition model from the remote device, and receives and plays the audio stream from the remote device; The embedded audio playback device also obtains a voice signal from the audio stream, and uses a scene-based keyword recognition model to perform artificial intelligence voice recognition on the voice signal, and detects in real time whether the voice signal contains keywords; When the voice signal contains a keyword, the embedded audio playback device generates a key information reminder and starts recording the received audio stream; The embedded audio playback device plays the recorded audio stream in response to the playback instruction.

8. The key information reminder system as claimed in claim 7, characterized in that: Also includes cloud servers, The remote device communicates with the cloud server and sends the keyword and / or the voice sample of the specific person to the cloud server; The cloud server uses the received keywords and / or the speech samples of the specific person to expand its standard sample library to form a training sample library, and based on the training sample library, uses a deep learning algorithm to train to obtain the scenario-based keyword recognition model; The remote device receives the scenario-based keyword recognition model from the cloud server, and downloads the scenario-based keyword recognition model to the embedded audio playback device.

9. The key information reminder system as claimed in claim 7, characterized in that: The remote device uses the keyword vocabulary input by the user and / or the speech sample of a specific person provided by the user that at least contains the keyword to expand the standard sample library to form a training sample library, and based on the training sample library, uses a deep learning algorithm to train to obtain the scenario-based keyword recognition model; The remote device downloads the scenario-based keyword recognition model to the embedded audio playback device.

10. A key information reminder method, characterized in that: Receive user-defined keyword vocabulary and / or a user-provided voice sample of a specific person containing at least the keyword; the keyword is associated with an application scenario and includes a group of vocabulary that needs to be focused on in the application scenario; Based on a training sample library containing speech samples for the keyword and / or speech samples of a specific person for the keyword, training to obtain a scenario-based keyword recognition model; When the embedded audio playback device receives and plays the audio stream, obtaining a voice signal from the audio stream; The embedded audio playback device uses the scenario-based keyword recognition model downloaded from the remote end to perform artificial intelligence speech recognition on the voice signal, and detects in real time whether the voice signal contains keywords; When the voice signal contains a keyword, the embedded audio playback device generates a key information reminder and starts recording the received audio stream; The embedded audio playback device plays the recorded audio stream in response to the playback instruction.

11. The key information reminder method according to claim 10, characterized in that: Collect a wide range of voice samples in advance to form a standard sample library; Acquire a voice sample containing at least the keyword according to the keyword; The speech samples containing the keywords and / or the speech samples of the specific person are expanded to the standard sample library to form a training sample library, and the scene-based keyword recognition model is obtained by training with a deep learning algorithm based on the training sample library.

12. The key information reminder method according to claim 10, characterized in that: The step of obtaining a speech signal from the audio stream also includes a preprocessing step of eliminating noise, music, and background human voices; The scene-based keyword recognition model is used to perform speech recognition on the speech signal or the preprocessed speech signal, and real-time detection is performed on whether the speech signal contains keywords, specifically including: constructing a deep learning neural network based on the keyword recognition model, continuously inputting the speech signal into the deep learning neural network for data processing, so as to infer and judge the words appearing in the speech signal, and determine whether the words contain keyword words.

13. The key information reminder method according to claim 10, characterized in that: The recording of the received audio stream specifically includes: when the voice signal contains a keyword, starting to continuously compress and encode the received audio stream and storing it locally; When a recording stop instruction is received or the continuous recording time exceeds a first predetermined time, the local recording is stopped; The playing of the recorded audio stream in response to the playback instruction specifically includes: playing the locally stored recorded audio stream in response to the playback local audio instruction.

14. The key information reminder method according to claim 10, characterized in that: The recording of the received audio stream specifically includes: when the voice signal contains a keyword, sending a recording start instruction to the remote end, so that the remote end starts to continuously record the sent audio stream and performs remote storage; When the continuous recording time does not exceed the second predetermined time and a stop recording instruction is received, a recording stop instruction is sent to the remote end, and the remote end stops recording; The playing of the recorded audio stream in response to the playback instruction specifically includes: sending a playback request to the remote end in response to the playback remote audio instruction, and receiving and playing the recorded audio stream stored in the remote end.

15. The key information reminder method according to claim 10, characterized in that: The key information reminder is a combination of one or more forms of visual reminder, tactile reminder and auditory reminder; The visual reminder includes light effect reminder and remote text message reminder; The tactile reminder includes a vibration reminder; The auditory reminder includes voice reminder and music reminder.

Images