A call alarm method and system based on ESP32S3

Abstract

This invention provides a call alarm method and system based on ESP32S3. The method includes: pre-setting voice command words; acquiring voice signals through the microphone of ESP32S3; creating a self-built server; converting the voice signals into text data based on the DAMO Academy model on the self-built server, and then converting the text data into trigger pinyin data; returning the trigger pinyin data to ESP32S3 through the self-built server; storing the trigger pinyin data in a local command library; acquiring the voice signals of local users and converting them into corresponding pinyin data; matching the pinyin data with the trigger pinyin data in the local command library; if successful, triggering an alarm prompt and initiating a request to create a call room to the self-built server; creating the call room and inviting associated users to enter the call room, enabling real-time audio transmission between the associated users and ESP32S3. This invention can create multi-terminal call rooms via voice.

Description

Technical Field

[0001] This invention relates to the field of multi-terminal communication technology, and in particular to a call alarm method and system based on ESP32S3. Background Technology

[0002] In scenarios such as home-based elderly care and industrial workshops, when emergency communication with external personnel is required, the ESP32S3 chip is generally used to establish communication.

[0003] In existing technologies, the ESP32S3 chip has limited computing power and only supports local speech-to-text (SR model), lacking built-in speech-to-pinyin functionality. This prevents direct integration with voice command recognition modules like Multinet that rely on pinyin input, impacting the addition and matching of custom voice commands. Current mainstream voice call solutions suffer from the following drawbacks: ① High long-term usage costs; ② Most only support two clients simultaneously (i.e., two-way communication), failing to meet the needs of multi-user collaborative scenarios (e.g., needing to notify technicians and administrators on multiple devices during an alarm). Traditional alarm call devices suffer from limited functionality: they only support "one-click alarm" or "one-way voice broadcasting," lacking real-time voice interaction capabilities—unable to trigger alarms via voice commands or communicate details with multiple personnel in real-time after an alarm is triggered. For example, an elderly person cannot directly communicate with their children or medical staff after triggering an alarm, and a worker cannot simultaneously share fault information with multiple technicians after triggering an alarm. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of this invention is to provide a call alarm method based on ESP32S3, which aims to solve the technical problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0006] A call alarm method based on ESP32S3 includes the following steps:

[0007] The system uses a preset voice command word and captures the voice signal emitted by the local user speaking the voice command word through the microphone of the ESP32S3.

[0008] Create a self-built server, convert the voice signal into text data based on the DAMO Academy model in the self-built server, convert the text data into trigger pinyin data, and return the trigger pinyin data to the ESP32S3 through the self-built server;

[0009] The triggered pinyin data is stored in the local command library to complete the addition of the command word;

[0010] The microphone of the ESP32S3 collects the voice signal of the local user in real time and converts the voice signal into Pinyin data accordingly.

[0011] The real-time obtained Pinyin data is matched with the trigger Pinyin data in the local command library. If the match is successful, an alarm is triggered and a request to create a call room is sent to the self-built server.

[0012] The self-built server creates a call room and invites associated users into the call room so that the associated users can transmit audio with the ESP32S3 in real time.

[0013] According to one aspect of the above technical solution, the ESP32S3 is connected to a microphone via an I2S interface and the parameters are set as follows: 16kHz sampling rate, 16-bit depth, and mono.

[0014] According to one aspect of the above technical solution, the specific steps of acquiring the voice signal emitted by the local user speaking the voice command through the microphone of the ESP32S3 include:

[0015] The ESP32S3 acquires the audio of the voice command spoken by the local user via I2S.

[0016] VAD detects and filters out silent segments in the audio;

[0017] The audio that exceeds a preset time will be treated as PCM data;

[0018] The PCM data is encapsulated together with the unique ID of the ESP32S3 into a custom data packet.

[0019] According to one aspect of the above technical solution, after encapsulating the PCM data together with the unique ID of the ESP32S3 into a custom data packet, the method further includes the following steps:

[0020] To enable protocol adaptation between the ESP32S3 and the self-built server, the following is specifically included:

[0021] Transmission protocol: The custom data packet is sent to the self-built server via HTTP POST, with the request header carrying a key identifier;

[0022] Device-ID: Verify whether the unique ID of the ESP32S3 matches the unique ID in the custom data packet;

[0023] Content-Type: application / octet-stream: declares the binary audio format;

[0024] X-Damo-Format: pcm: Informs the self-built server that it needs to call the PCM format interface of the DAMO Academy model;

[0025] Resume interrupted transmission: If the network is interrupted, the self-built server caches the audio segments that were not sent in the custom data packets and automatically resumes transmission after the connection is restored.

[0026] According to one aspect of the above technical solution, the steps of converting the speech signal into text data based on the DAMO Academy model in the self-built server, converting the text data into trigger pinyin data, and returning the trigger pinyin data to the ESP32S3 through the self-built server include:

[0027] The self-built server initializes the DAMO Academy model through AK / SK authentication, specifying the region and interface version;

[0028] The self-built server transmits the following key parameters to the DAMO Academy model:

[0029] SpeechData: Custom data packets sent by the ESP32S3;

[0030] SampleRate=16000, Channel=1, Format="pcm";

[0031] EnablePinyin=True;

[0032] DeviceId: The unique ID of the ESP32S3;

[0033] The DAMO Academy model uses a built-in algorithm to identify the audio in the custom data packet as text data, and then converts the text data into preliminary pinyin data;

[0034] The text data and the preliminary pinyin data are scored with confidence, and data with confidence scores higher than the confidence threshold are retained;

[0035] The initial pinyin data with tones is converted into trigger pinyin data without tones using a preset tone mapping table;

[0036] Filter out non-pinyin characters and merge consecutive spaces in the triggered pinyin data;

[0037] The self-built server packages the trigger pinyin data, the corresponding text data, and the confidence score into JSON format, encrypts them with AES, and returns them to the ESP32S3.

[0038] According to one aspect of the above technical solution, the step of storing the triggering pinyin data in the local command library to complete the addition of the command word includes:

[0039] The JSON-formatted data is decrypted using a preset key, the confidence score is checked, and data with a confidence score less than the confidence threshold is filtered out.

[0040] The trigger pinyin data is split into words by spaces and added to the local command library to complete the addition of command words;

[0041] In the local command library, the words are configured with their corresponding instructions.

[0042] According to one aspect of the above technical solution, the step of matching the real-time obtained pinyin data with the trigger pinyin data in the local command library further includes the following steps:

[0043] If the match fails, the ESP32S3 will display a message via its speaker indicating that the command word is not recognized and to be re-entered.

[0044] According to one aspect of the above technical solution, the self-built server creates a call room and invites associated users into the call room, so that the associated users can transmit audio with the ESP32S3 in real time. Specific steps include:

[0045] The self-built server pushes room invitations to the bound devices of other associated users;

[0046] The ESP32S3 and other associated user devices transmit audio in real time via WebSocket.

[0047] The present invention also provides a call alarm system based on ESP32S3, comprising:

[0048] Setting module: used to preset voice command words, and to collect the voice signal emitted by the local user when the voice command words are spoken through the microphone of ESP32S3;

[0049] Conversion module: Used to create a self-built server, in which the voice signal is converted into text data based on the DAMO Academy model, and the text data is converted into trigger pinyin data, and the trigger pinyin data is returned to the ESP32S3 through the self-built server;

[0050] Storage module: used to store the trigger pinyin data to the local command library to complete the addition of command words;

[0051] Acquisition module: Used to acquire the local user's voice signal in real time through the microphone of the ESP32S3, and convert the voice signal into Pinyin data accordingly;

[0052] Matching module: used to match the real-time obtained pinyin data with the trigger pinyin data in the local command library. If the match is successful, an alarm is triggered and a request to create a call room is sent to the self-built server.

[0053] Transmission module: Used by the self-built server to create a call room and invite associated users into the call room, so that the associated users can transmit audio with the ESP32S3 in real time.

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0055] ESP32S3 speech-to-pinyin solution: Existing technologies such as traditional alarm devices and Volcano Engine do not have a solution and require additional high-performance chips, resulting in high costs; while this invention connects to the DAMO Academy model on the server side, requiring only lightweight local processing, thus reducing hardware costs;

[0056] Voice call platform dependence: Existing technologies such as traditional alarm devices and Volcano Engine rely on third-party platforms, resulting in high usage costs; the self-built server of this invention is independent and controllable, saving costs;

[0057] Caller limit: Existing technologies such as traditional alarm devices and Volcano Engine devices do not support calls, and third-party platforms are limited to 2 devices; this invention can support ≥3 devices for multi-person calls, meeting the needs of collaborative scenarios;

[0058] Functional integration: Existing technologies such as traditional alarm devices and Volcano Engine separate alarm and call functions (i.e., a manual call is required after an alarm is triggered); this invention integrates "voice command to trigger alarm + multi-terminal real-time call + voice to add command words", making the process integrated. Attached Figure Description

[0059] Figure 1 This is a flowchart of the call alarm method based on ESP32S3 in the first embodiment of the present invention;

[0060] Figure 2 This is a structural block diagram of the call alarm system based on ESP32S3 in the second embodiment of the present invention;

[0061] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0062] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0063] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0065] Please see Figure 1 The image shows a call alarm method based on ESP32S3 in the first embodiment of the present invention, which includes the following steps:

[0066] S10, preset voice command words, and collect the voice signal emitted by the local user saying the voice command words through the microphone of ESP32S3;

[0067] S20, create a self-built server, convert the voice signal into text data based on the DAMO Academy model in the self-built server, convert the text data into trigger pinyin data, and return the trigger pinyin data to the ESP32S3 through the self-built server;

[0068] S30, the trigger pinyin data is stored in the local command library to complete the addition of the command word;

[0069] S40, the microphone of the ESP32S3 collects the voice signal of the local user in real time and converts the voice signal into Pinyin data accordingly;

[0070] S50, the real-time obtained pinyin data is matched with the trigger pinyin data in the local command library. If the match is successful, an alarm is triggered and a request to create a call room is sent to the self-built server.

[0071] S60, the self-built server creates a call room and invites associated users into the call room so that the associated users can transmit audio with the ESP32S3 in real time.

[0072] It is understandable that, compared with the prior art, the beneficial effects of the present invention are as follows:

[0073] ESP32S3 speech-to-pinyin solution: Existing technologies such as traditional alarm devices and Volcano Engine do not have a solution and require additional high-performance chips, resulting in high costs; while this invention connects to the DAMO Academy model on the server side, requiring only lightweight local processing, thus reducing hardware costs;

[0074] Voice call platform dependence: Existing technologies such as traditional alarm devices and Volcano Engine rely on third-party platforms, resulting in high usage costs; the self-built server of this invention is independent and controllable, saving costs;

[0075] Caller limit: Existing technologies such as traditional alarm devices and Volcano Engine devices do not support calls, and third-party platforms are limited to 2 devices; this invention can support ≥3 devices for multi-person calls, meeting the needs of collaborative scenarios;

[0076] Functional integration: Existing technologies such as traditional alarm devices and Volcano Engine separate alarm and call functions (i.e., a manual call is required after an alarm is triggered); this invention integrates "voice command to trigger alarm + multi-terminal real-time call + voice to add command words", making the process integrated.

[0077] Specifically, in this embodiment, the ESP32S3 is connected to a microphone via an I2S interface, and the parameters are set as follows: 16kHz sampling rate, 16-bit depth, and mono.

[0078] Understandably, this parameter is configured strictly according to the DAMO Academy API requirements to ensure that the subsequently acquired PCM audio can be directly called via the API without format conversion.

[0079] The specific steps of acquiring the voice signal emitted by the local user speaking the voice command through the microphone of the ESP32S3 include:

[0080] The ESP32S3 acquires the audio of the voice command spoken by the local user via I2S.

[0081] VAD detects and filters out silent segments in the audio; VAD is used for speech activity detection, retaining only valid speech with energy exceeding a threshold (i.e., filtering out ambient noise) and reducing invalid API calls;

[0082] The audio exceeding the preset time is treated as PCM data; the purpose of this step is to filter out shorter audio clips.

[0083] The PCM data is encapsulated together with the unique ID of the ESP32S3 into a custom data packet; for example, if the unique ID of the device is "esp32s3_001", it is encapsulated together with the PCM data into a custom data packet (including an audio length field) to facilitate the DAMO Academy service in distinguishing the sources of multiple devices.

[0084] Furthermore, the step of encapsulating the PCM data together with the unique ID of the ESP32S3 into a custom data packet also includes the following steps:

[0085] To enable protocol adaptation between the ESP32S3 and the self-built server, the following is specifically included:

[0086] Transmission protocol: The custom data packet is sent to the self-built server via HTTP POST, with the request header carrying a key identifier;

[0087] Device-ID: Verify whether the unique ID of the ESP32S3 matches the unique ID in the custom data packet; this step is for double verification.

[0088] Content-Type: application / octet-stream: declares the binary audio format;

[0089] X-Damo-Format: pcm: Informs the self-built server that it needs to call the PCM format interface of the DAMO Academy model;

[0090] Resume interrupted transmission: If the network is interrupted, the self-built server caches the audio segments that were not sent in the custom data packets, and automatically resumes transmission after the connection is restored, avoiding audio loss due to network fluctuations.

[0091] Furthermore, the specific steps of converting the speech signal into text data based on the DAMO Academy model in the self-built server, converting the text data into trigger pinyin data, and returning the trigger pinyin data to the ESP32S3 through the self-built server include:

[0092] The self-built server initializes the DAMO Academy model through AK / SK authentication, specifying the region (such as "cn-shanghai") and interface version to adapt to the latest speech recognition function;

[0093] The self-built server transmits the following key parameters to the DAMO Academy model:

[0094] SpeechData: Custom data packets sent by the ESP32S3; specifically, PCM binary data;

[0095] SampleRate=16000, Channel=1, Format="pcm"; strictly aligned with the terminal acquisition parameters;

[0096] EnablePinyin=True; Enables the Pinyin return function, exclusively supported by the DAMO Academy model.

[0097] DeviceId: The unique ID of ESP32S3; used for device management and log tracing in the Dharma Academy model background;

[0098] The Dharma Academy model uses a built-in algorithm to recognize the audio in the custom data packet as text data (such as "help"), and converts the text data into preliminary pinyin data (such as "jiù míng");

[0099] Perform confidence scoring on the text data and the preliminary pinyin data, and retain the data with a confidence score higher than the confidence threshold; the confidence score is 0 - 1, such as 0.92, which is used to filter out low-quality recognition, and only retain the results with a confidence level ≥ 0.8, eliminating misidentifications caused by noise and blurred speech;

[0100] Convert the preliminary pinyin data with tones into trigger pinyin data in a tone-less format through a preset tone mapping table; for example, "jiù míng" is converted into a tone-less format such as "jiu ming", which is achieved through the preset tone mapping table ā→a, á→a, etc.;

[0101] Filter out non-pinyin characters (such as punctuation marks and numbers) in the trigger pinyin data and merge consecutive spaces; ensure that the output format is exactly the same as the terminal instruction library, such as "jiu ming kuai lai";

[0102] Package the trigger pinyin data, the corresponding text data, and the confidence score into a JSON format through the self-built server, such as {"status":"success","pinyin":"jiu ming","text":"救命","confidence":0.92}, encrypt it with AES and return it to the ESP32S3 to prevent the pinyin instructions from being eavesdropped or tampered with, and the terminal decrypts it with the preset key.

[0103] Further, storing the trigger pinyin data in the local command library to complete the addition of command words, the specific steps include:

[0104] Decrypt the JSON format data with the preset key, check the confidence score, and filter out the data with a confidence score less than the confidence threshold; the purpose of this step is to check the confidence after the terminal decrypts, and only process the results with a confidence level ≥ 0.8, providing double protection to avoid accidental triggering;

[0105] Split the trigger pinyin data into words by spaces (such as "jiu ming" → ["jiu", "ming"]), and add them to the local command library to complete the addition of command words;

[0106] In the local command library, the words are configured with their corresponding instructions, such as ["jiu", "ming"] corresponding to "alarm";

[0107] In actual use, the ESP32S3's microphone collects the local user's voice signal in real time. The same steps are used to match the obtained Pinyin data with the data in the local command library to determine whether the match is successful. If successful, an alarm is triggered (buzzer + light) and a request to create a call room is sent to the self-built server.

[0108] Furthermore, the step of matching the real-time obtained pinyin data with the trigger pinyin data in the local command library further includes the following steps:

[0109] If the match fails, the ESP32S3 will display a message via its speaker indicating that the command word is not recognized and to be re-entered.

[0110] Furthermore, the self-built server creates a call room and invites associated users into the call room so that the associated users can transmit audio with the ESP32S3 in real time. Specific steps include:

[0111] The self-built server pushes room invitations to the bound devices of other associated users;

[0112] The ESP32S3 and other associated user devices transmit audio in real time via WebSocket.

[0113] Taking the scenario of elderly people living alone as an example, the local users are the elderly, the associated users are the elderly's relatives, and the associated users' bound devices include multiple terminals such as mobile phones, computers, and tablets.

[0114] When either a local user or a connected user leaves the room, the self-built server will destroy the room and the alarm will stop.

[0115] In summary, the ESP32S3-based call alarm method in the above embodiments of the present invention is cost-effective: it requires no third-party voice platform, has low hardware requirements for self-built servers (a regular cloud server suffices), and the ESP32S3 chip offers high cost-performance, making it suitable for batch deployment, such as in workshops with multiple devices or campus dormitories. It is also highly flexible in function: supporting custom voice commands such as "Call XX" or "XX Alarm," and can be quickly expanded through the server-side DAMO Academy model to adapt to different scenarios such as homes, industries, and campuses. Furthermore, it is highly user-friendly: no manual operation is required; simply pressing the alarm button or entering a room number triggers the alarm and call; non-professional users such as the elderly and workers can quickly learn how to use it.

[0116] Please refer to Figure 2The image shows a call alarm system based on ESP32S3 according to a second embodiment of the present invention, comprising:

[0117] Setting module 11: Used to preset voice command words, and to collect the voice signal emitted by the local user when the voice command words are spoken through the microphone of ESP32S3;

[0118] The ESP32S3 is connected to a microphone via an I2S interface, and the parameters are set as follows: 16kHz sampling rate, 16-bit depth, and mono.

[0119] The setting module 11 is specifically used for: ESP32S3 to collect the audio emitted by the local user speaking the voice command words via I2S;

[0120] VAD detects and filters out silent segments in the audio;

[0121] The audio that exceeds a preset time will be treated as PCM data;

[0122] The PCM data is encapsulated together with the unique ID of the ESP32S3 into a custom data packet;

[0123] The process of encapsulating the PCM data together with the unique ID of the ESP32S3 into a custom data packet further includes the following steps:

[0124] To enable protocol adaptation between the ESP32S3 and the self-built server, the following is specifically included:

[0125] Transmission protocol: The custom data packet is sent to the self-built server via HTTP POST, with the request header carrying a key identifier;

[0126] Device-ID: Verify whether the unique ID of the ESP32S3 matches the unique ID in the custom data packet;

[0127] Content-Type: application / octet-stream: declares the binary audio format;

[0128] X-Damo-Format: pcm: Informs the self-built server that it needs to call the PCM format interface of the DAMO Academy model;

[0129] Resume interrupted transmission: If the network is interrupted, the self-built server caches the audio segments that were not sent in the custom data packets and automatically resumes transmission after the connection is restored;

[0130] Conversion module 12: Used to create a self-built server, in which the voice signal is converted into text data based on the DAMO Academy model, and the text data is converted into trigger pinyin data, and the trigger pinyin data is returned to the ESP32S3 through the self-built server;

[0131] The conversion module 12 is specifically used for:

[0132] The self-built server initializes the DAMO Academy model through AK / SK authentication, specifying the region and interface version;

[0133] The self-built server transmits the following key parameters to the DAMO Academy model:

[0134] SpeechData: Custom data packets sent by the ESP32S3;

[0135] SampleRate=16000, Channel=1, Format="pcm";

[0136] EnablePinyin=True;

[0137] DeviceId: The unique ID of the ESP32S3;

[0138] The DAMO Academy model uses a built-in algorithm to identify the audio in the custom data packet as text data, and then converts the text data into preliminary pinyin data;

[0139] The text data and the preliminary pinyin data are scored with confidence, and data with confidence scores higher than the confidence threshold are retained;

[0140] The initial pinyin data with tones is converted into trigger pinyin data without tones using a preset tone mapping table;

[0141] Filter out non-pinyin characters and merge consecutive spaces in the triggered pinyin data;

[0142] The self-built server packages the trigger pinyin data, the corresponding text data, and the confidence score into JSON format, encrypts them with AES, and returns them to the ESP32S3.

[0143] Storage module 13: Used to store the trigger pinyin data to the local command library to complete the addition of command words;

[0144] The storage module 13 is specifically used for:

[0145] The JSON-formatted data is decrypted using a preset key, the confidence score is checked, and data with a confidence score less than the confidence threshold is filtered out.

[0146] The trigger pinyin data is split into words by spaces and added to the local command library to complete the addition of command words;

[0147] In the local command library, the words are configured with their corresponding instructions;

[0148] Acquisition module 14: Used for the microphone of the ESP32S3 to acquire the voice signal of the local user in real time, and convert the voice signal into Pinyin data accordingly;

[0149] Matching module 15: Used to match the real-time obtained Pinyin data with the trigger Pinyin data in the local command library. If the match is successful, an alarm prompt is triggered and a request to create a call room is sent to the self-built server. If the match is unsuccessful, the speaker of the ESP32S3 will prompt that the command word is not recognized and please re-enter it.

[0150] Transmission module 16: Used for the self-built server to create a call room and invite associated users into the call room, so that the associated users can transmit audio with the ESP32S3 in real time;

[0151] The transmission module 16 is specifically used for:

[0152] The self-built server pushes room invitations to the bound devices of other associated users;

[0153] The ESP32S3 and other associated user devices transmit audio in real time via WebSocket.

[0154] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0155] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. An ESP32S3-based call alarm method, characterized in that, The method comprises the following steps: Pre-set voice command words, collect the voice signal emitted by the local user speaking the voice command words through the microphone of the ESP32S3; Create a self-built server, convert the voice signal into text data based on the DAMO model in the self-built server, convert the text data into trigger pinyin data, and return the trigger pinyin data to the ESP32S3 through the self-built server; Store the trigger pinyin data in the local command library to complete the addition of command words; The microphone of the ESP32S3 collects the voice signal of the local user in real time, and converts the voice signal into pinyin data; Match the real-time obtained pinyin data with the trigger pinyin data in the local command library, if the matching is successful, trigger an alarm prompt, and initiate a create call room request to the self-built server; The self-built server creates a call room and invites associated users into the call room to enable the associated users to transmit audio in real time with the ESP32S3; The microphone of the ESP32S3 collects the voice signal emitted by the local user speaking the voice command words, and the specific steps include: The ESP32S3 collects the audio emitted by the local user speaking the voice command words through I2S; VAD detects the mute segment of the audio and filters it; The audio exceeding the preset time is taken as PCM data; The PCM data is packaged together with the unique ID of the ESP32S3 as a custom data packet; After the PCM data is packaged together with the unique ID of the ESP32S3 as a custom data packet, the following steps are further included: Make the ESP32S3 and the self-built server adapt to the protocol, specifically including: Transmission protocol: send the custom data packet to the self-built server through HTTP POST, and carry the key identification in the request header; Device-ID: check whether the unique ID of the ESP32S3 is consistent with the unique ID in the custom data packet; Content-Type: application / octet-stream: declare the binary audio format; X-Damo-Format: pcm: inform the self-built server to call the PCM format interface of the DAMO model; Resume transmission: if the network is interrupted, the self-built server caches the audio segments not sent in the custom data packet, and automatically resumes transmission after the connection is restored; In the self-built server, the voice signal is converted into text data based on the DAMO model, the text data is converted into trigger pinyin data, and the trigger pinyin data is returned to the ESP32S3 through the self-built server, and the specific steps include: The self-built server initializes the DAMO model through AK / SK authentication, specifies the region and interface version; The self-built server inputs the following key parameters to the DAMO model: SpeechData: custom data packet sent by the ESP32S3; SampleRate=16000, Channel=1, Format="pcm"; EnablePinyin=True; DeviceId: the unique ID of the ESP32S3; The DAMO Institute model uses built-in algorithms to identify the audio in the custom data packet as text data and converts the text data into preliminary pinyin data; The text data and the preliminary pinyin data are scored for confidence, and data with a confidence score higher than a confidence threshold is retained; The preliminary pinyin data with tones is converted to trigger pinyin data without tones through a preset tone mapping table; Non-pinyin characters in the trigger pinyin data are filtered out, and consecutive spaces are merged; The trigger pinyin data, the corresponding text data, and the confidence score are packaged into JSON format by the self-built server and returned to the ESP32S3 after AES encryption.

2. The ESP32S3-based call alerting method of claim 1, wherein, The ESP32S3 connects the microphone through the I2S interface and sets the parameters to 16kHz sampling rate, 16-bit depth, and single channel.

3. The ESP32S3-based call alerting method of claim 1, wherein, The trigger pinyin data is stored in the local command library to complete command word addition, including the following steps: Decrypt the JSON format data using a pre-set key, check the confidence score, and filter data with a confidence score less than the confidence threshold; Split the trigger pinyin data by spaces into words and add them to the local command library to complete command word addition; In the local command library, configure the words with corresponding instructions.

4. The ESP32S3-based call alerting method of claim 1, wherein, The real-time obtained pinyin data is matched with the trigger pinyin data in the local command library, and then includes the following steps: If the match is unsuccessful, the ESP32S3 speaker prompts that the command word has not been recognized and asks for re-input.

5. The ESP32S3-based call alerting method of claim 1, wherein, The self-built server creates a call room and invites associated users into the call room to enable real-time audio transmission between the associated users and the ESP32S3, including the following steps: The self-built server pushes room invitations to the binding devices of other associated users; The ESP32S3 and the binding devices of other associated users transmit audio in real time through WebSocket.

6. A call alarm system based on ESP32S3, characterized in that, Including: Setting module: used to preset voice command words, and collect voice signals emitted by local users speaking the voice command words through the microphone of the ESP32S3; The microphone of the ESP32S3 collects voice signals emitted by local users speaking the voice command words, including the following steps: The ESP32S3 collects audio emitted by local users speaking the voice command words through I2S; VAD detects the silent segments of the audio and filters them out; The audio that exceeds the preset time is taken as PCM data; The PCM data is packaged into a custom data packet together with the unique ID of the ESP32S3; The PCM data is packaged into a custom data packet together with the unique ID of the ESP32S3, and then includes the following steps: Make the ESP32S3 and the self-built server protocol-adapted, including: Transmission protocol: Send the custom data packet to the self-built server through HTTP POST, and carry the key identification in the request header; Device-ID: Check whether the unique ID of ESP32S3 is consistent with the unique ID in the custom data packet; Content-Type: application / octet-stream: Declare binary audio format; X-Damo-Format: pcm: Inform the self-built server to call the PCM format interface of the DAMO model; Resuming transmission: If the network is interrupted, the self-built server caches the audio segments in the custom data packet that have not been sent, and automatically resumes transmission after the connection is restored; Conversion module: used for creating a self-built server, converting the voice signal into text data based on the DAMO model in the self-built server, converting the text data into trigger pinyin data, and returning the trigger pinyin data to the ESP32S3 through the self-built server; The specific steps of converting the voice signal into text data based on the DAMO model in the self-built server, converting the text data into trigger pinyin data, and returning the trigger pinyin data to the ESP32S3 include: The self-built server initializes the DAMO model through AK / SK authentication, specifies the region and interface version; The self-built server passes the following key parameters to the DAMO model: SpeechData: Custom data packet sent by ESP32S3; SampleRate=16000, Channel=1, Format="pcm"; EnablePinyin=True; DeviceId: Unique ID of ESP32S3; The DAMO model uses built-in algorithms to recognize the audio in the custom data packet as text data, and converts the text data into preliminary pinyin data; Perform confidence scoring on the text data and the preliminary pinyin data, and retain data with a confidence score higher than the confidence threshold; Convert the preliminary pinyin data with tones into trigger pinyin data without tones through a preset tone mapping table; Filter out non-pinyin characters in the trigger pinyin data and merge consecutive spaces; Pack the trigger pinyin data, the corresponding text data, and the confidence score into JSON format through the self-built server, encrypt them through AES, and return them to the ESP32S3; Storage module: used for storing the trigger pinyin data in the local command library to complete command word addition; Acquisition module: used for the microphone of the ESP32S3 to collect the voice signal of the local user in real time, and convert the voice signal into pinyin data; Matching module: used for matching the real-time obtained pinyin data with the trigger pinyin data in the local command library, if the matching is successful, triggering an alarm prompt, and initiating a create call room request to the self-built server; A transmission module is configured to create a call room by the self-built server and invite associated users into the call room, so that the associated users transmit audio in real time with the ESP32S3.

Images