Intercom Recording and Encoding / Decoding System and Method Based on Node.js Middleware

Abstract

This application relates to a two-way audio recording and encoding / decoding system and method based on Node.js middleware. The method includes: a client obtaining access to a sound sensor; the sound sensor acquiring raw PCM audio data in single-channel mode; the Node.js middleware merging and compressing the raw PCM audio data, and further compressing, encapsulating, and sorting it to obtain a processed audio packet, which is then sent to a server; the server receiving the processed audio packet, verifying its conformity, and accepting compliant audio packets; when the server sends the processed audio packet to the Node.js middleware for reception, the middleware decompresses and restores the processed audio packet to obtain a PCM audio packet, sorts and merges the PCM audio packet, sends it to the client via a socket connection, and plays it through the client's player. This application improves the stability of audio data transmission, thereby improving call quality and stability.

Description

Technical Field

[0001] This application relates to the field of Node.js middleware communication technology, and in particular to a Node.js middleware-based intercom recording encoding and decoding system and method. Background Technology

[0002] A walkie-talkie is a two-way mobile communication tool that allows communication without any network support or call charges. Walkie-talkies have a wide range of applications, primarily in public security, civil aviation, transportation, water conservancy, railways, manufacturing, construction, and service industries, for communication and command and dispatch among group members to improve communication efficiency and rapid response capabilities in handling emergencies. With walkie-talkies entering the civilian market, people are increasingly using them for travel and shopping.

[0003] With the development of the Internet, traditional walkie-talkies can no longer meet people's needs. The communication range of traditional walkie-talkies is limited, and the communication quality cannot be guaranteed when the communication exceeds a certain distance. Summary of the Invention

[0004] To improve the quality and stability of walkie-talkie calls, this application provides a walkie-talkie recording and encoding / decoding system and method based on Node.js middleware.

[0005] The first aspect of this application provides a two-way audio recording and decoding system based on Node.js middleware. The system includes: a data acquisition unit, a data processing unit, and a data receiving unit; the data acquisition unit runs on a client, and the data processing unit and the data receiving unit both run on Node.js middleware; the client establishes a socket connection with the Node.js middleware to send audio data to the Node.js middleware, and the Node.js middleware processes the audio data and sends it to a server.

[0006] By adopting the above technical solution, the audio data collected by the client is sent to the Node.js middleware. The Node.js middleware processes the audio data and continuously sends the processed audio data to the server, ensuring reliable and stable audio data transmission and improving the quality and stability of the call.

[0007] Optionally, the data acquisition unit consists of a sound sensor, used to acquire the audio data to the client via the sound sensor.

[0008] By adopting the above technical solution, the data acquisition unit on the client side consists of a sound sensor, which can be used to collect audio data.

[0009] Optionally, the data processing unit includes a merging and compression module, a packet decoding and compression module, an encapsulation and sorting module, and a sending module, wherein the merging and compression module, the packet decoding and compression module, the encapsulation and sorting module, and the sending module are respectively connected to each other.

[0010] The merging and compression module is used to calculate the offset of the original PCM audio data collected by the data acquisition unit, and to merge and compress the original PCM audio data to obtain merged and compressed audio data.

[0011] The packet decoding and compression module is used to convert the merged and compressed audio data into audio data with a specified sampling rate and number of samples; divide the converted audio data into PCM audio packets according to a preset packet division algorithm; and further convert the divided PCM audio packets into Opus format audio packets according to the Opus encoding algorithm.

[0012] The encapsulation and sorting module is used to encapsulate the Opus format audio packet using RTP, add an 8-byte data packet header to obtain the encapsulated audio packet, wherein the 8-byte data packet header consists of 4 bytes of data length and 4 bytes of data type; and sort the encapsulated audio packet according to a fixed size and through a packet splitting mechanism to obtain the RTP format audio packet.

[0013] The sending module is used to send the RTP format audio packets to the server at fixed time intervals using a timing algorithm.

[0014] By adopting the above technical solution, the data processing unit on the Node.js middleware performs a series of data processing steps on the original audio data, including merging and compressing, packet decoding and compression, and encapsulation and sorting. This allows the audio data to be compressed without affecting the audio quality, thereby reducing the load on the server.

[0015] Optionally, the data receiving unit comprises a decoding and decompression module, a data splicing module, and a playback module, wherein the decoding and decompression module, the data splicing module, and the playback module are sequentially connected for data transmission.

[0016] The decoding and decompression module is used to remove the 8-byte data packet header and the 12-byte RTP header from the RTP format audio packet to restore the RTP format audio packet to the Opus format audio packet, and decompress the Opus format audio packet into the PCM original audio packet using a preset Opus decoding algorithm.

[0017] The data splicing module is used to merge the original PCM audio packets sequentially according to the audio packet sequence number to obtain the merged audio packets.

[0018] The playback module is used to play the merged audio package through a player via a client.

[0019] By adopting the above technical solution, the received audio data is decoded, decompressed, and merged by the data receiving unit on the Node.js middleware to obtain a merged audio package, which is then played by the playback module.

[0020] A second aspect of this application provides a method for intercom recording and decoding based on Node.js middleware. This method is applied to an intercom recording and decoding system based on Node.js middleware, the system comprising a client, Node.js middleware, and a server, wherein the client, Node.js middleware, and server are sequentially connected for data transmission. The method includes:

[0021] The client obtains access to the sound sensor; it calls the sound sensor to acquire audio streams in single-channel mode, obtaining the acquired raw PCM audio data.

[0022] The Node.js middleware merges and compresses the original PCM audio data through the data processing unit to obtain compressed audio data, and further compresses, encapsulates and sorts the compressed audio data to obtain a processed audio package. The Node.js middleware then sends the processed audio package to the server.

[0023] The server receives the processed audio packet, checks whether the processed audio packet conforms to the specification, and if it conforms to the specification, it receives the packet; otherwise, it disconnects the connection.

[0024] When the server sends the processed audio package to the Node.js middleware for reception, the data receiving unit receives the processed audio package, the data processing unit decompresses and restores the processed audio package to obtain a PCM audio package, sorts and merges the PCM audio package, sends it to other clients through a socket connection, and plays it through the client's player.

[0025] By adopting the above technical solution, the audio data collected by the client is sent to the Node.js middleware. After the Node.js middleware encodes and decodes the audio data, it obtains audio packets and continuously sends audio packets to the server, ensuring reliable and stable audio data transmission and improving the quality and stability of the call.

[0026] Optionally, the Node.js middleware, through the data processing unit, merges and compresses the original PCM audio data to obtain compressed audio data, including: the Node.js middleware compresses the original PCM audio data into PCM audio data with a specified sampling rate and sampling bit depth through the data processing unit, and divides the PCM audio data into fixed-size audio packets through a packet splitting mechanism to obtain compressed audio data.

[0027] By adopting the above technical solution, the collected raw PCM audio data can be converted into PCM audio data with a specified sampling rate and sampling bit depth. This can compress the audio data without affecting the audio quality and reduce the load on the server.

[0028] Optionally, the step of further compressing, encapsulating, and sorting the compressed audio data to obtain a processed audio package includes: the Node.js middleware compresses the compressed audio data using Opus encoding, converts the PCM audio data into an Opus-format audio package, and encapsulates and sorts the Opus-format audio package using RTP to obtain the processed audio package.

[0029] By adopting the above technical solution, the compressed audio data is further encoded and compressed in the Node.js middleware using Opus to convert it into a high-fidelity Opus format audio package with a smaller data size. Opus is a high-fidelity voice encoding format suitable for transmission over the network, and it has better fidelity compared to other encoding formats.

[0030] Optionally, the step of encapsulating and sorting the Opus format audio packet via RTP includes: the Node.js middleware adds an 8-byte data packet header to the Opus format audio packet via RTP, and encapsulates and sorts the packets according to a fixed size using a packet-splitting mechanism to obtain the processed audio packet, wherein the 8 bytes consist of 4 bytes of data length and 4 bytes of data type.

[0031] By adopting the above technical solution, the Opus format audio packets are divided and packaged into packets of fixed size using a packetization mechanism, so as to process the audio packets into a standard format and transmit them in real time.

[0032] Optionally, sending the processed audio package to the server includes: the Node.js middleware splits the processed audio package into packets and continuously sends the processed audio package to the server at preset time intervals.

[0033] By adopting the above technical solution, the processed audio packets are divided into sub-packets and continuously sent at preset time intervals, which can achieve real-time transmission and reduce transmission latency.

[0034] Optionally, the data processing unit decompresses and restores the processed audio packet to obtain a PCM audio packet, including: the data processing unit removes the packet header and RTP header from the processed audio packet, restores it to an Opus format audio packet, and converts it into a PCM audio packet with a specified sampling rate and number of samples according to the Opus decoding algorithm to obtain a PCM audio packet.

[0035] By adopting the above technical solution, the encoded and compressed audio package is restored to the Opus format audio package, and then converted into a PCM audio package according to the Opus decoding algorithm, so that the client can play the PCM audio package.

[0036] A third aspect of this application provides a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method steps described above.

[0037] A fourth aspect of this application provides an electronic device comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the method steps described above.

[0038] In summary, this application includes at least one of the following beneficial technical effects:

[0039] 1. The audio data collected by the client is sent to the Node.js middleware. After the Node.js middleware encodes and decodes the audio data, it obtains audio packets and continuously sends audio packets to the server to ensure reliable and stable audio data transmission and improve the quality and stability of the call.

[0040] 2. By using the data processing unit on the Node.js middleware to perform a series of data processing steps on the original audio data, such as merging and compressing, splitting and decoding, and encapsulating and sorting, the audio data is compressed without affecting the audio quality, thus reducing the load on the server.

[0041] 3. By dividing the processed audio packets into smaller packets and sending them continuously at preset time intervals, real-time transmission can be achieved, reducing transmission latency. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic diagram of the structure of a walkie-talkie recording and decoding system based on Node.js middleware, provided in an embodiment of this application.

[0044] Figure 2 This is a schematic diagram of a data processing unit of a Node.js middleware-based intercom recording and decoding system provided in an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of a data receiving unit of a Node.js middleware-based intercom recording and decoding system provided in an embodiment of this application;

[0046] Figure 4 This is a flowchart illustrating a walkie-talkie recording and decoding method based on Node.js middleware provided in an embodiment of this application.

[0047] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0048] Explanation of reference numerals in the attached figures: 1000, electronic device; 1001, processor; 1002, communication bus; 1003, user interface; 1004, network interface; 1005, memory. Detailed Implementation

[0049] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0050] In the description of the embodiments in this application, words such as "illustrative," "for example," or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "illustrative," "for example," or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "illustrative," "for example," or "for example" is intended to present the relevant concepts in a specific manner.

[0051] In the description of the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, or A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0052] The present application will now be described in detail with reference to specific embodiments.

[0053] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of a Node.js middleware-based intercom recording and decoding system provided in an embodiment of this application.

[0054] In one embodiment, a Node.js middleware-based intercom recording and encoding / decoding system is proposed. The system includes a data acquisition unit, a data processing unit, and a data receiving unit. The data acquisition unit runs on the client side, while the data processing unit and the data receiving unit both run on the Node.js middleware.

[0055] The data acquisition unit consists of a sound sensor, which collects audio data and transmits the collected audio data to the client. The sound sensor works in conjunction with the player in the client. After the client and the Node.js middleware are successfully connected, the user clicks the walkie-talkie button, at which point the sound sensor begins to record and collect audio data in real time and transmits the collected audio data in real time to achieve real-time communication.

[0056] Please see Figure 2 , Figure 2 This is a schematic diagram of a data processing unit of a Node.js middleware-based intercom recording and decoding system provided in an embodiment of this application.

[0057] The data processing unit includes a merging and compression module, a packet decoding and compression module, an encapsulation and sorting module, and a sending module, wherein the merging and compression module, the packet decoding and compression module, the encapsulation and sorting module, and the sending module are connected to each other in sequence.

[0058] The merging and compression module is used to calculate the offset of the original PCM audio data acquired by the data acquisition unit, and then merge and compress the original PCM audio data to obtain merged and compressed audio data.

[0059] The packet decoding and compression module is used to convert merged and compressed audio data into audio data with a specified sampling rate and number of samples; it divides the converted audio data into PCM audio packets according to a preset packet division algorithm, and further converts the divided PCM audio packets into Opus format audio packets according to the Opus encoding algorithm.

[0060] The encapsulation and sorting module is used to encapsulate Opus format audio packets via RTP, adding an 8-byte data packet header to obtain the encapsulated audio packet, where the 8 bytes consist of 4 bytes of data length and 4 bytes of data type; the encapsulated audio packets are then sorted into packets of fixed size using a packet-splitting mechanism.

[0061] The sending module is used to send the segmented audio data to the server in RTP format at fixed time intervals using a timing algorithm.

[0062] The audio data is collected by the aforementioned data acquisition unit and processed by the data processing unit, thus realizing the function of transmitting voice data to the server.

[0063] Please see Figure 3 , Figure 3 This is a schematic diagram of a data receiving unit of a Node.js middleware-based intercom recording and decoding system provided in an embodiment of this application.

[0064] The data receiving unit consists of a decoding and decompression module, a data splicing module, and a playback module, wherein the decoding and decompression module, the data splicing module, and the playback module are connected to each other in sequence.

[0065] The decoding and decompression module is used to remove the 8-byte data packet header and the 12-byte RTP header from the RTP format audio packet, so as to restore the RTP format audio packet to the Opus format audio packet. The Opus format audio packet is decompressed into the original PCM audio packet by using a preset Opus decoding algorithm.

[0066] The data splicing module is used to merge the original PCM audio packets sequentially according to their audio packet numbers to obtain the merged audio packets.

[0067] The playback module is used to play the merged audio package through the client via a player.

[0068] By decoding, compressing, and splicing audio data through the aforementioned data receiving unit, the function of real-time intercom audio data playback using Node.js is realized.

[0069] In one embodiment, see Figure 4 This paper presents a flowchart illustrating a method for intercom recording and encoding / decoding based on Node.js middleware. This method is primarily applied to an intercom recording and encoding / decoding system based on Node.js middleware, which includes a client, a Node.js middleware, and a server. The client, middleware, and server sequentially establish data connections. The specific method includes:

[0070] Step 10: The client obtains access to the sound sensor; the sound sensor is invoked to acquire audio streams in single-channel mode, obtaining the raw PCM audio data.

[0071] In this embodiment of the application, the client is understood to be a program that corresponds to the server and provides local services to the client.

[0072] The data acquisition unit on the client consists of a sound sensor, which is used to collect audio data and transmit it to the client.

[0073] Furthermore, before the client collects or plays audio data, it's necessary to ensure a socket connection is established between the client and the Node.js middleware, and that the client's connection address is cached in the Node.js middleware. If the client fails to establish a connection with the Node.js middleware, an error message will be displayed. Therefore, establishing a connection between the client and the Node.js middleware is a prerequisite for the real-time intercom recording and encoding / decoding operation of the Node.js middleware.

[0074] For example, the data acquisition unit runs on the client. After the client successfully connects to the Node.js middleware, the data acquisition unit obtains the access permission of the sound sensor through the getUserMedia method, determines whether access to the sound sensor is allowed, and displays an error message if access is not allowed; if access is allowed, the sound sensor is called to acquire the audio stream in single-channel mode to obtain the acquired raw PCM audio data.

[0075] Step 20: The Node.js middleware merges and compresses the original PCM audio data through the data processing unit to obtain compressed audio data. The compressed audio data is then further compressed, encapsulated, and sorted to obtain the processed audio package. The Node.js middleware sends the processed audio package to the server.

[0076] In this embodiment, Node.js middleware can be understood as encapsulating request processing functionality within a program. The Node.js middleware resides on the server's operating system and manages computer resources and network communication. Essentially, Node.js middleware performs filtering and other processing on data before it enters the actual data processing stage.

[0077] Optionally, based on the above embodiments, as an optional embodiment, the Node.js middleware merges and compresses the original PCM audio data through the data processing unit to obtain compressed audio data, and further compresses, encapsulates, and sorts the compressed audio data to obtain a processed audio package. The step of the Node.js middleware sending the processed audio package to the server also includes the following steps:

[0078] Step 201: The Node.js middleware compresses the original PCM audio data into PCM audio data with a specified sampling rate and bit depth. The PCM audio data is then divided into fixed-size audio packets using a packet splitting mechanism to obtain compressed audio data.

[0079] For example, the Node.js middleware obtains the raw PCM audio data through the `audioContext` and `audioprocess` methods. This raw PCM audio data is a 32-bit floating-point PCM audio data stream with a sampling rate of 48000, which consumes a significant amount of resources. The raw PCM audio stream is a two-dimensional array. By calculating offsets, the two-dimensional PCM audio data is converted into a single unit, achieving the purpose of merging and compression. The Node.js middleware first converts the audio data into PCM audio data with a sampling rate of 8000 and a sample count of 16-bit integers. Then, according to a preset packet splitting algorithm, the converted audio data is divided into compressed audio data of length 480, each consisting of 16-bit integers. Converting the acquired raw PCM audio into audio data with a specified sampling rate and sample count is to compress the audio data without affecting audio quality, thereby reducing the load on the server.

[0080] Step 202: The Node.js middleware compresses the compressed audio data using Opus encoding and converts the PCM audio data into Opus format audio packets.

[0081] For example, the Node.js middleware further compresses the compressed audio data using Opus encoding, converting the PCM audio data into a smaller, high-fidelity Opus format audio packet. Opus is a high-fidelity voice encoding format suitable for transmission over networks, offering better fidelity compared to other encoding formats. This embodiment employs, but is not limited to, the Opus encoding algorithm for further compression; other encoding algorithms may also be used in another feasible embodiment.

[0082] Step 203: The Node.js middleware adds an 8-byte header to the Opus format audio packet via RTP, and then segments and sorts the packets according to a fixed size using a packetization mechanism to obtain the processed audio packet.

[0083] The Node.js middleware adds an 8-byte header to Opus format audio packets via RTP. This 8-byte header consists of 4 bytes of data length and 4 bytes of data type. The packets are then divided and sorted according to a fixed size using a packetization mechanism to obtain the processed audio packets.

[0084] Step 204: The Node.js middleware sends the processed audio package to the server.

[0085] For example, the Node.js middleware continuously sends the processed audio packets to the server at preset time intervals, which can achieve real-time transmission and reduce transmission latency.

[0086] Step 30: The server receives the processed audio packet and checks whether the processed audio packet conforms to the specifications.

[0087] For example, after receiving the processed audio packet, the server determines whether the processed audio packet conforms to a preset standard. In this embodiment, the preset standard may be that the data packet is 4 bytes long, the data packet type is 4 bytes long, and the RTP header is 12 bytes long. If the processed audio packet conforms to the preset standard, it is received, and the server forwards the audio packet conforming to the preset standard to the Node.js middleware; if the processed audio packet does not conform to the preset standard, the connection is closed. In this embodiment, multiple Node.js middleware can be set up to establish connections with the server and receive audio packets, enabling one-to-one communication and multi-device listening functionality.

[0088] Step 40: The server sends the compliant audio package to the Node.js middleware. The Node.js middleware decompresses and restores the compliant audio package, sorts and merges it, and sends the merged audio package to the client for playback.

[0089] The data processing unit of the Node.js middleware decodes and decompresses the compliant audio packets. It removes the 8-byte data header and the 12-byte RTP header, restoring the compliant audio packets to Opus format. The Opus format audio packets, after removing the headers and RTP headers, are then decompressed using the Opus decoding algorithm and converted into fixed-size PCM audio packets with a sampling rate of 8000 and a bit depth of 16-bit integers. The PCM audio packets are then merged sequentially according to their sequence numbers, and the merged audio packets are sent to the client so that the client's player can play them until playback is complete.

[0090] The implementation principle of this application is as follows: The client establishes a socket connection with the Node.js middleware, uses the `getUserMedia` and `AudioContext` methods to obtain access permissions to the sound sensor, and then calls the sound sensor to collect raw PCM audio data in single-channel mode. The Node.js middleware merges and compresses the raw PCM audio data through the data processing unit, converting it into PCM audio data with a specified sampling rate and bit depth. It then divides the PCM audio data into fixed-size audio packets using a packetization mechanism, converts the PCM audio data into Opus format audio packets using Opus encoding compression, and finally encapsulates and sorts the Opus-compressed audio packets using RTP, continuously sending them to the server at preset time intervals. The server receives the data processed by the data processing unit, checks whether the processed audio packets conform to the specifications, and if they do, receives and forwards them; otherwise, it disconnects. When the server transmits the RTP audio packet to the Node.js middleware for reception, it removes the header and RTP header from the compliant audio packet, restoring it to the Opus format audio packet. Then, it uses the Opus decoding algorithm to convert it into a PCM audio packet with a specified sampling rate and number of samples, and then sends it to the client. The client sorts and merges the converted PCM audio packets through data concatenation, and plays them through the client's player.

[0091] It should be noted that the system provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0092] This application also provides a computer storage medium that can store multiple instructions, which are adapted to be loaded and executed by a processor as described in the above-described embodiment of a Node.js middleware-based intercom recording and decoding method. The specific execution process can be found in [reference needed]. Figure 1 The specific details of the illustrated embodiments will not be elaborated here.

[0093] Please see Figure 5 This document provides a schematic diagram of the structure of an electronic device as described in an embodiment of this application. The electronic device is a client, a Node.js middleware, or a server, such as... Figure 5 As shown, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.

[0094] The communication bus 1002 is used to realize the connection and communication between these components.

[0095] The user interface 1003 may include a display screen and a camera. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.

[0096] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0097] The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts within the server 1000 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1001 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 1001.

[0098] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 5 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application based on a Node.js middleware intercom recording and decoding method.

[0099] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0100] exist Figure 5 In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain the user input data; while the processor 1001 can be used to call an application stored in the memory 1005 that is a talkback recording encoding and decoding method based on Node.js middleware. When executed by one or more processors, the electronic device executes one or more of the methods described in the above embodiments.

[0101] An electronic device readable storage medium is provided, characterized in that the electronic device readable storage medium stores instructions. When executed by one or more processors, these instructions cause the electronic device to perform one or more of the methods described in the above embodiments.

[0102] Those skilled in the art will clearly understand that the technical solutions of this application can be implemented using software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform specific functions. Hardware may include, for example, a Field-Programmable Gate Array (FPGA), an Integrated Circuit (IC), etc.

[0103] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0104] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0106] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0107] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0108] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0109] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0110] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A two-way audio recording and decoding system based on Node.js middleware, characterized in that, The system includes: a data acquisition unit, a data processing unit, and a data receiving unit; the data acquisition unit runs on a client, and the data processing unit and the data receiving unit both run on a Node.js middleware. The client establishes a socket connection with the Node.js middleware to send audio data to the Node.js middleware; The Node.js middleware is used to process the audio data and continuously send it to the server. The data processing unit includes a merging and compression module, a packet decoding and compression module, an encapsulation and sorting module, and a sending module, wherein the merging and compression module, the packet decoding and compression module, the encapsulation and sorting module, and the sending module are connected to each other in sequence. The merging and compression module is used to calculate the offset of the original PCM audio data collected by the data acquisition unit, and to merge and compress the original PCM audio data to obtain merged and compressed audio data. The packet decoding and compression module is used to convert the merged and compressed audio data into audio data with a specified sampling rate and number of samples; divide the converted audio data into PCM audio packets according to a preset packet division algorithm; and further convert the divided PCM audio packets into Opus format audio packets according to the Opus encoding algorithm. The encapsulation and sorting module is used to encapsulate the Opus format audio packet using RTP, add a data packet header of length 8 bytes to obtain the encapsulated audio packet, wherein the 8 bytes consist of 4 bytes of data length and 4 bytes of data type; and sort the encapsulated audio packet according to a fixed size and through a packet splitting mechanism to obtain the RTP format audio packet. The sending module is used to send the RTP format audio packets to the server at fixed time intervals using a timing algorithm.

2. The intercom recording and encoding / decoding system based on Node.js middleware according to claim 1, characterized in that, The data acquisition unit consists of a sound sensor, which is used to acquire the audio data to the client through the sound sensor.

3. The intercom recording and encoding / decoding system based on Node.js middleware according to claim 1, characterized in that, The data receiving unit comprises a decoding and decompression module, a data splicing module, and a playback module, wherein the decoding and decompression module, the data splicing module, and the playback module are sequentially connected for data transmission. The decoding and decompression module is used to remove the 8-byte data packet header and the 12-byte RTP header from the RTP format audio packet to restore the RTP format audio packet to the Opus format audio packet, and decompress the Opus format audio packet into PCM raw audio data through a preset Opus decoding algorithm. The data splicing module is used to merge the original PCM audio packets sequentially according to the audio packet sequence number to obtain the merged audio packets. The playback module is used to play the merged audio package through a player via a client.

4. A method for intercom recording and encoding / decoding based on Node.js middleware, characterized in that, The method is applied to the intercom recording and encoding / decoding system based on Node.js middleware as described in claim 1, wherein the system includes a client, a Node.js middleware, and a server, and the client, Node.js middleware, and server are sequentially connected to each other for data transmission. The client obtains access to the sound sensor and calls the sound sensor to collect audio streams, obtaining the collected raw PCM audio data. The Node.js middleware merges and compresses the original PCM audio data through the data processing unit to obtain compressed audio data, and further compresses, encapsulates and sorts the compressed audio data to obtain a processed audio package. The Node.js middleware then sends the processed audio package to the server. The server receives the processed audio packet and checks whether the processed audio packet conforms to the specification. If it conforms to the specification, it receives the packet; otherwise, it disconnects the connection. When the server sends the processed audio package to the Node.js middleware for reception, the data receiving unit receives the processed audio package, the data processing unit decompresses and restores the processed audio package to obtain a PCM audio package, sorts and merges the PCM audio package, sends it to the client through a socket connection, and plays it through the client's player.

5. The method for intercom recording and encoding / decoding based on Node.js middleware according to claim 4, characterized in that, The Node.js middleware, through the data processing unit, merges and compresses the original PCM audio data to obtain compressed audio data, including: The Node.js middleware uses the data processing unit to compress the original PCM audio data into PCM audio data with a specified sampling rate and bit depth. The PCM audio data is then divided into fixed-size audio packets using a packet splitting mechanism to obtain compressed audio data.

6. The method for intercom recording and encoding / decoding based on Node.js middleware according to claim 5, characterized in that, The step of further compressing, encapsulating, and sorting the compressed audio data to obtain a processed audio packet includes: The Node.js middleware compresses the compressed audio data using Opus encoding, converts the PCM audio data into Opus format audio packets, and encapsulates and sorts the Opus format audio packets using RTP to obtain the processed audio packets.

7. The method for intercom recording and encoding / decoding based on Node.js middleware according to claim 6, characterized in that, The process of encapsulating and sorting the Opus format audio packets via RTP includes: The Node.js middleware adds an 8-byte header to the Opus format audio packet via RTP, and then segments and sorts the packets according to a fixed size using a packetization mechanism to obtain the processed audio packet. The 8-byte header consists of 4 bytes of data length and 4 bytes of data type.

8. The method for intercom recording and encoding / decoding based on Node.js middleware according to claim 4, characterized in that, The Node.js middleware sends the processed audio package to the server, including: The Node.js middleware divides the processed audio package into packets and continuously sends the processed audio packets to the server at preset time intervals.

9. A method for intercom recording and decoding based on Node.js middleware according to claim 4, characterized in that, The data processing unit decompresses and restores the processed audio package to obtain a PCM audio package, including: The data processing unit removes the packet header and RTP header from the processed audio packet, restoring it to an Opus format audio packet. Then, according to the Opus decoding algorithm, it converts the Opus format audio packet into a PCM audio packet with a specified sampling rate and number of samples.

Images