NVR recording method, device and medium for two-way intercom audio and video

Abstract

The present invention provides an NVR recording method for two-way intercom audio and video, comprising: a local intercom and an NVR device negotiate a format and port; the local intercom and a remote intercom negotiate a format and port; the remote intercom collects data, encodes and encapsulates it, and sends it to the local intercom; the local intercom receives and parses the data, enters a decoder for decoding, and then transmits it to a remote audio device and a remote camera; the local camera and the local audio device collect audio and video data; images are synthesized based on the local video data and the remote video data, and audio mixing is performed based on the local audio data and the remote audio data; the synthesized video data and the mixed audio data are transferred to a video encoder and an audio encoder for encoding, packaged into video packets and audio packets, and sent to the NVR device via a transmission port. The present invention also provides an electronic device and a computer-readable storage medium for implementing two-way audio and video recording.

Description

Technical Field

[0001] The present invention relates to the technical field of building intercoms, and in particular to an NVR recording method, device and medium for recording two-way intercom audio and video. Background Art

[0002] With the development of technology and the gradual popularization of digital intercom technology, video intercom is becoming the industry standard. It is necessary not only to hear the sound but also to see the video. In the IPC (IP CAMERA) industry, the video and sound of the IPC end can be recorded through the IPC + NVR (Network Video Recorder) method. However, in the field of building intercom, the introduction of IPC-related technologies RTSP (Real Time Streaming Protocol) + ONVIF (Open Network Video Interface) means that the NVR can only record the audio and video of the local intercom (stairway or door station). When two-way intercom is in progress, the audio and video of the remote intercom (indoor station, app terminal, analog phone or property management machine) cannot be recorded.

[0003] like Figure 1 As shown, there are multiple local intercoms (stair entrance machines or door machines) in a community. Each local intercom may conduct audio calls, video calls, or two-way video calls with remote intercoms (such as apps, analog phones, indoor machines, and property management machines, etc.). In traditional technical implementation, audio and video storage is considered to be placed on the indoor machine or the cloud. Recording audio and video on the indoor machine will lose audio and video data of other devices (apps, analog phones, and property management machines) except the indoor machine. Recording audio and video on the cloud will increase the load on the cloud server. Each call must be forwarded through the cloud, and the 24-hour audio and video of the local intercom needs to be transmitted to the cloud in real time. This has upstream requirements for the bandwidth of the cloud server. For the overall cloud solution, cost and network bandwidth will become a bottleneck problem. Summary of the Invention

[0004] To solve the above problems, the present invention provides an NVR recording method for two-way intercom audio and video. Without changing the technical architecture of the NVR, an audio and video system is designed on the local intercom to support two-way audio and video recording.

[0005] The present invention is achieved through the following technical solutions:

[0006] The present invention provides an NVR recording method for two-way intercom audio and video. The method requires providing a local intercom, an NVR device, and a remote intercom. The local intercom is equipped with a local camera, a remote camera, a local audio device, a remote audio device, an audio encoder, an audio decoder, a video encoder, and a video decoder. The method comprises the following steps:

[0007] Step 1: The local intercom and the NVR device negotiate the audio and video format used during recording and the audio and video sending port;

[0008] Step 2: The local intercom and the remote intercom negotiate the audio and video formats and the audio and video transceiver ports of both parties during the intercom;

[0009] Step 3: The remote intercom collects the remote audio and video data, encodes and encapsulates it, and then sends the encapsulated audio and video data to the local intercom according to the negotiated audio and video format and the audio and video transceiver port; the local intercom receives and parses the encapsulated audio and video data, classifies the parsed audio and video data, and enters the corresponding audio decoder and video decoder for decoding processing, obtains decoded audio data and video data, and then transmits them to the remote audio device and remote camera respectively;

[0010] Step 4: The local camera and local audio device collect local audio and video data;

[0011] Step 5: Performing image synthesis based on the local video data collected by the local camera and the remote video data obtained by the remote camera, and performing audio mixing based on the local audio data collected by the local audio device and the remote audio data obtained by the remote audio device;

[0012] Step 6: The synthesized video data and the mixed audio data are transferred to the corresponding video encoder and audio encoder for encoding processing, and then packaged into video packets and audio packets, and sent to the NVR device for recording according to the negotiated audio and video format and audio and video sending port.

[0013] Furthermore, the local intercom is also provided with an ONVIF device, and the step 1 specifically includes:

[0014] Step 11: The NVR device broadcasts through the ONVIF discovery protocol to search for ONVIF devices;

[0015] Step 12: After receiving the ONVIF discovery protocol, the ONVIF device responds with the ONVIF Web Service address to the NVR device;

[0016] Step 13: After receiving the ONVIF Web Service address, the NVR device sends a query RTSP address request to the ONVIF device;

[0017] Step 14: After receiving the query RTSP address request, ONVIF responds with the corresponding RTSP address to the NVR device;

[0018] Step 15: The NVR device initiates an RTSP request to the local intercom according to the RTSP address;

[0019] Step 16: The local intercom negotiates with the NVR device according to the RTSP request on the audio and video format and the audio and video sending port to be used during recording.

[0020] Furthermore, the step 2 specifically includes:

[0021] The local intercom negotiates with the remote intercom via the SIP protocol to determine the audio and video formats and the audio and video transceiver ports of both parties during the intercom.

[0022] Furthermore, the step 3 specifically includes:

[0023] The remote intercom collects audio data, encodes the audio data into audio compressed data in G711A / U, G722, G729 or OPUS format through the audio encoder in the remote intercom, and then packages the audio compressed data into an audio RTP packet. The audio RTP packet is sent to the local intercom according to the negotiated audio and video format and the audio and video transceiver port; the local intercom receives and parses the audio RTP packet to obtain audio compressed data, and sends it to the audio decoder for decoding. After decoding, the audio data is transmitted to the remote audio device and cached in the data pool;

[0024] At the same time, the remote intercom collects video data, encodes the video data into video compression data in H264 or H265 format through the video encoder in the remote intercom, and then packages the video compression data into a video RTP packet. The video RTP packet is sent to the local intercom according to the negotiated audio and video format and the audio and video transceiver port; the local intercom receives and parses the video RTP packet to obtain video compression data, and sends it to the video decoder for decoding. After decoding, the video data is obtained, passed to the remote camera, and cached in the data pool.

[0025] Furthermore, performing image synthesis based on the local video data collected by the local camera and the remote video data obtained by the remote camera in step 5 specifically includes:

[0026] When the local video data collected by the local camera is obtained, query whether there is remote video data in the remote camera. If so, obtain the remote video data and perform scaling operations with the local video data to synthesize the remote video and local video into one picture according to the custom ratio; if not, fill the entire picture with the local video.

[0027] Furthermore, in step 5, mixing is performed based on the local audio data collected by the local audio device and the remote audio data acquired by the remote audio device, specifically including:

[0028] Determine whether the remote audio device has remote audio data. If not, directly transfer the local audio data to the audio encoder. If present, retrieve the remote audio data within time T from the remote audio device and mix it with the local audio data within time T obtained by the current local audio device.

[0029] Furthermore, the step 6 specifically includes:

[0030] The synthesized video data is transferred to the video encoder for encoding to generate video compression data in H264, H265 or JPEG format. The video compression data is then packaged into video RTP packets and sent to the NVR device for video recording according to the negotiated audio and video format and audio and video sending port;

[0031] At the same time, the mixed audio data is transferred to the audio encoder for encoding to generate audio compression data in G711A / U, G722, G729 or OPUS format, and then the audio compression data is packaged into audio RTP packets and sent to the NVR device for audio recording according to the negotiated audio and video format and audio and video sending port.

[0032] Furthermore, the local intercom is a stair entrance machine or a door machine, and the remote intercom is an indoor machine, an APP, an analog phone or a property management machine.

[0033] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, an NVR recording method for two-way intercom audio and video of an intercom as described above is implemented.

[0034] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the NVR recording method for two-way intercom audio and video as described above.

[0035] The beneficial effects of the present invention are:

[0036] A new set of audio and video interfaces is implemented on the local intercom side, which is compatible with the traditional NVR protocol (RTSP+ONVIF). When used with traditional NVR devices, the real-time video of the local intercom and the audio and video data of both parties during the intercom can be recorded on the NVR device in real time, realizing two-way audio and video recording. From the perspective of the community, without adding additional costs, it can provide more monitoring perspectives for the community, and can monitor the audio and video of both parties during the intercom, providing favorable support for the security of the community. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 It is the basic framework diagram of the existing technology.

[0039] Figure 2 The present invention provides a flowchart of an NVR recording method for two-way intercom audio and video.

[0040] Figure 3 It is a specific flow chart of step 1 in the present invention.

[0041] Figure 4 This is a specific flow chart of the audio data transmission from a remote intercom to a local intercom in the present invention.

[0042] Figure 5 This is a specific flow chart of the video data of a remote intercom transmitted to a local intercom in the present invention.

[0043] Figure 6 It is a schematic diagram of the judgment process of image synthesis in the present invention.

[0044] Figure 7 It is one of the image synthesis style graphs provided by the embodiment of the present invention.

[0045] Figure 8 This is the second image synthesis style diagram provided by an embodiment of the present invention.

[0046] Figure 9 This is a framework diagram of an NVR recording system for two-way intercom audio and video provided by an embodiment of the present invention.

[0047] Figure 10 is a schematic diagram of an electronic device provided by an embodiment of the present invention.

[0048] Figure 11 It is a schematic diagram of a computer-readable storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention for which protection is sought, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0050] like Figure 2 As shown, an embodiment of the present invention provides an NVR recording method for two-way intercom audio and video. The method requires providing a local intercom, an NVR device, and a remote intercom. The local intercom is a stair entrance machine or a door machine, and the remote intercom is an indoor machine, an APP, an analog phone, or a property management machine. The local intercom is equipped with a local camera, a remote camera, a local audio device, a remote audio device, an audio encoder, an audio decoder, a video encoder, a video decoder, and an ONVIF device. The method includes the following steps:

[0051] Step 1: The local intercom and the NVR device negotiate the audio and video format and the audio and video transmission port used during recording. The purpose of setting the format and port in advance is to have a specific transmission channel for the audio and video data to be transmitted from the local intercom to the NVR device later, making the audio and video data transmission more standardized.

[0052] like Figure 3 As shown, in this embodiment, this step specifically includes:

[0053] Step 11: The NVR device broadcasts through the ONVIF discovery protocol to search for ONVIF devices;

[0054] Step 12: After receiving the ONVIF discovery protocol, the ONVIF device responds with the ONVIF Web Service address to the NVR device;

[0055] Step 13: After receiving the ONVIF Web Service address, the NVR device sends a query RTSP address request to the ONVIF device;

[0056] Step 14: After receiving the query RTSP address request, ONVIF responds with the corresponding RTSP address to the NVR device;

[0057] Step 15: The NVR device initiates an RTSP request to the local intercom according to the RTSP address;

[0058] Step 16: The local intercom negotiates with the NVR device according to the RTSP request on the audio and video format and the audio and video sending port used during recording;

[0059] The NVR device finds the ONVIF device through the ONVIF discovery protocol, and queries the RTSP address of the ONVIF device by querying the RTSP address request, and then implements a new set of audio and video interfaces through the RTSP protocol and ONVIF protocol on the local intercom end, which is compatible with the traditional NVR protocol (RTSP+ONVIF) and does not require the development of new protocols, making operation simple and reducing costs.

[0060] Step 2: The local intercom and the remote intercom negotiate the audio and video formats and the audio and video transceiver ports for both parties during the intercom. The purpose of setting the format and port in advance is to provide a specific transmission channel for the subsequent two-way transmission of audio and video data from the remote intercom to the local intercom and from the local intercom to the remote intercom, making the audio and video data transmission more standardized.

[0061] In this embodiment, the step specifically includes:

[0062] The local intercom negotiates with the remote intercom through the SIP protocol to determine the audio and video formats and the audio and video sending and receiving ports of both parties during the intercom; the format and port settings are set using SIP (Session Initialization Protocol), which is flexible, easy to implement, and easy to expand.

[0063] Step 3: The remote intercom collects the remote audio and video data, encodes and encapsulates it, and then sends the encapsulated audio and video data to the local intercom according to the negotiated audio and video format and the audio and video transceiver port; the local intercom receives and parses the encapsulated audio and video data, classifies the parsed audio and video data, and enters the corresponding audio decoder and video decoder for decoding processing, obtaining decoded audio data and video data, and then transmits them to the remote audio device and remote camera respectively; this is the process of transmitting the audio and video data from the remote intercom to the local intercom, and storing and displaying them on the local intercom;

[0064] like Figure 4 As shown, in this embodiment, this step specifically includes:

[0065] Step 31: The remote intercom collects audio data;

[0066] Step 32: The audio data is encoded into audio compression data in G711A / U, G722, G729 or OPUS format by an audio encoder in the remote intercom.

[0067] Step 33: Pack the compressed audio data into an audio RTP packet; send the audio RTP packet to the local intercom according to the negotiated audio and video format and the audio and video transceiver port;

[0068] Step 34: The local intercom receives and parses the audio RTP packet to obtain compressed audio data.

[0069] Step 35: Send the compressed audio data to the audio decoder for decoding. After decoding, the audio data is transmitted to the remote audio device and cached in the data pool. Steps 31-35 are the process of transmitting the audio data collected by the remote intercom to the local intercom, and storing and displaying it on the local intercom. This completes the function of the local intercom receiving the audio data of the remote intercom.

[0070] At the same time, if Figure 5 As shown:

[0071] Step 36: The remote intercom collects video data;

[0072] Step 37: The video data is encoded into video compression data in H264 or H265 format by a video encoder in the remote intercom;

[0073] Step 38: Pack the compressed video data into a video RTP packet, and send the video RTP packet to the local intercom according to the negotiated audio and video format and the audio and video transceiver port;

[0074] Step 39: The local intercom receives and parses the video RTP packet to obtain video compressed data;

[0075] Step 310: Send the compressed video data to the video decoder for decoding. After decoding, the obtained video data is transmitted to the remote camera and cached in the data pool. Steps 36-310 are the process of transmitting the video data collected by the remote intercom to the local intercom, and storing and displaying it on the local intercom. This completes the function of the local intercom receiving the video data of the remote intercom.

[0076] Steps 31-35 and steps 36-310 are performed simultaneously without any particular order.

[0077] Step 4: The local camera and local audio device collect local audio and video data;

[0078] Step 5: Perform image synthesis based on the local video data collected by the local camera and the remote video data obtained by the remote camera. At the same time, perform audio mixing based on the local audio data collected by the local audio device and the remote audio data obtained by the remote audio device. This is the process of combining the local and remote audio and video data. The data is fused together in order to better output it to the NVR device for recording.

[0079] In this embodiment, if Figure 6-8 As shown, the image synthesis process in this step specifically includes:

[0080] When the local video data collected by the local camera is obtained, the remote camera is checked to see if there is remote video data. If so, the remote video data is obtained and scaled with the local video data to synthesize the remote video and the local video into one picture according to the custom ratio. Figure 7 The local video and the remote video are combined in a 1:1 ratio, such as Figure 8 The local video and the remote video are composited in an 8:1 ratio. If not, the local video fills the entire screen, which can be filled with a custom color. This is the process of image synthesis. The purpose of synthesis is to make the local video and the remote video more intuitive and better transmit them to the NVR device for recording.

[0081] The mixing process in this step specifically includes:

[0082] Determine whether the remote audio device has remote audio data. If not, directly transfer the local audio data to the audio encoder. If so, extract the remote audio data within time T from the remote audio device and mix it with the local audio data within time T obtained by the current local audio device. The time T is set by the user and can be 20ms. The amount of audio data collected within this time is relatively appropriate, which can be just right for mixing processing without causing too much processing pressure, and can also be better transmitted to the NVR device for recording.

[0083] Step 6: The synthesized video data and the mixed audio data are transferred to the corresponding video encoder and audio encoder for encoding processing, and then packaged into video packets and audio packets. They are sent to the NVR device for recording according to the negotiated audio and video format and audio and video sending port;

[0084] In this embodiment, the step specifically includes:

[0085] The synthesized video data is transferred to the video encoder for encoding to generate video compressed data in H264, H265 or JPEG format. The compressed video data is then packaged into video RTP packets and sent to the NVR device for video recording based on the negotiated audio and video format and audio and video transmission port. This is the process of transmitting local video data and remote video data to the NVR device. After encoding and RTP format encapsulation, it can be sent to the NVR device more quickly.

[0086] At the same time, the mixed audio data is transferred to the audio encoder for encoding to generate audio compression data in G711A / U, G722, G729 or OPUS format, and then the audio compression data is packaged into audio RTP packets. According to the negotiated audio and video format and the audio and video sending port, it is sent to the NVR device for audio recording. This is the process of transmitting local audio data and remote audio data to the NVR device. After encoding and RTP format encapsulation, it can be sent to the NVR device more quickly.

[0087] In the present invention, when the local intercom and the remote intercom are conducting a normal intercom, the data transmission process is as follows:

[0088] A: Remote intercom data to local intercom:

[0089] The remote intercom collects and encodes audio data, then encapsulates it into RTP data using the RTP packet format and sends the encapsulated RTP data to the local intercom. The local intercom parses the RTP data and sends it to the audio decoder for decoding. The decoded audio data is then transmitted to the remote audio device and cached in the data pool.

[0090] The remote intercom obtains the camera frame data, encodes it into H264 / H265 data, and sends the encapsulated RTP data to the local intercom via the RTP packet format. The local intercom parses the RTP data and sends it to the video decoder. The decoded video data is passed to the remote camera and cached in the data pool.

[0091] B: Local intercom data to remote intercom:

[0092] Open the local camera to obtain camera frame data, transfer the data directly to the video encoder, generate H264 / H265 / JPEG video compression data, then package it into video RTP packets and send it to the remote intercom;

[0093] The local audio device is opened to obtain audio data. The data is directly transferred to the audio encoder, which generates audio compression data such as G711A / U, G722, G729, and OPUS. The data is then packaged into audio RTP packets and sent to the remote intercom.

[0094] Through the transmission and interaction of data between the two parties, the two-way intercom function between the local intercom and the remote intercom can be realized.

[0095] The system framework diagram of the present invention is as follows: Figure 9 As shown, an NVR recording system for two-way intercom audio and video provided by an embodiment of the present invention includes: a local intercom, an NVR device, and a remote intercom; the local intercom is provided with a local camera, a remote camera, a local audio device, a remote audio device, an audio encoder, an audio decoder, a video encoder, a video decoder, an ONVIF device, a virtual camera module, a virtual audio module, an RTSP module, an RTP video encapsulation module, an RTP audio encapsulation module, an RTP parsing module, and a SIP module;

[0096] ONVIF device and RTSP module transmit bidirectionally with NVR device respectively to complete monitoring negotiation; SIP module transmits bidirectionally with remote intercom to complete intercom negotiation; audio and video data of remote intercom is sent to RTP parsing module for parsing, and then sent to audio decoder and video decoder for decoding, and then sent to corresponding remote audio device and remote camera; video data of local camera and remote camera is transmitted to virtual camera module for processing, and then sent to video encoder for encoding, and then sent to RTP video encapsulation module for encapsulation and sent to NVR device for recording; audio data of local audio device and remote audio device ... RTP parsing module for parsing, and then sent to RTP parsing module for parsing, and then sent to NVR device for recording; audio data of local audio device and remote audio device is transmitted to RTP parsing module for parsing, and then sent to RTP parsing module for parsing, and then sent to NVR device for parsing. After the data is transmitted to the virtual audio module for processing, it is sent to the audio encoder for encoding, and then to the RTP audio encapsulation module for encapsulation and sent to the NVR device for recording; in addition, the video data of the local camera is transmitted to the virtual camera module. At this time, the virtual camera module does not process it, but directly forwards it to the video encoder for encoding, and then passes it to the RTP video encapsulation module for encapsulation and sends it to the remote intercom to complete the video intercom; the audio data of the local audio device is transmitted to the virtual audio module. At this time, the virtual audio module does not process it, but forwards it to the audio encoder for encoding, and then passes it to the RTP audio encapsulation module for encapsulation and sends it to the remote intercom to complete the audio intercom.

[0097] like Figure 10 As shown, an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the NVR recording method of the two-way intercom audio and video of the above-mentioned intercom is implemented.

[0098] like Figure 11 As shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned NVR recording method for two-way intercom audio and video.

[0099] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for recording two-way audio and video of an intercom using an NVR, the method requiring a local intercom, an NVR device, and a remote intercom, characterized in that: The local intercom is provided with a local camera, a remote camera, a local audio device, a remote audio device, an audio encoder, an audio decoder, a video encoder and a video decoder, and the method comprises the following steps: Step 1: The local intercom and the NVR device negotiate the audio and video format and the audio and video transmission port used during recording; a new audio and video interface is implemented on the local intercom through the RTSP protocol and ONVIF protocol, which is compatible with the NVR protocol; Step 2: The local intercom and the remote intercom negotiate the audio and video formats and the audio and video transceiver ports of both parties during the intercom; Step 3: The remote intercom collects the remote audio and video data, encodes and encapsulates it, and then sends the encapsulated audio and video data to the local intercom according to the audio and video formats and audio and video transceiver ports agreed upon by the local intercom and the remote intercom; the local intercom receives and parses the encapsulated audio and video data, classifies the parsed audio and video data, and enters the corresponding audio decoder and video decoder for decoding processing, thereby obtaining decoded audio data and video data, which are then transmitted to the remote audio device and remote camera respectively; Step 4: The local camera and local audio device collect local audio and video data; Step 5: Performing image synthesis based on the local video data collected by the local camera and the remote video data obtained by the remote camera, and performing audio mixing based on the local audio data collected by the local audio device and the remote audio data obtained by the remote audio device; Step 6: The synthesized video data and the mixed audio data are transferred to the corresponding video encoder and audio encoder for encoding processing respectively, and then packaged into video packages and audio packages. The audio and video formats used during recording and the audio and video sending ports negotiated between the local intercom and the NVR device are sent to the NVR device for recording.

2. The NVR recording method of two-way intercom audio and video according to claim 1, characterized in that: The local intercom is also provided with an ONVIF device, and the step 1 specifically includes: Step 11: The NVR device broadcasts through the ONVIF discovery protocol to search for ONVIF devices; Step 12: After receiving the ONVIF discovery protocol, the ONVIF device responds with the ONVIF Web Service address to the NVR device; Step 13: After receiving the ONVIF Web Service address, the NVR device sends a query RTSP address request to the ONVIF device; Step 14: After receiving the query RTSP address request, ONVIF responds with the corresponding RTSP address to the NVR device; Step 15: The NVR device initiates an RTSP request to the local intercom according to the RTSP address; Step 16: The local intercom negotiates with the NVR device according to the RTSP request on the audio and video format and the audio and video sending port to be used during recording.

3. The NVR recording method of two-way intercom audio and video according to claim 1, characterized in that: The step 2 specifically includes: The local intercom negotiates with the remote intercom via the SIP protocol to determine the audio and video formats and the audio and video transceiver ports of both parties during the intercom.

4. The NVR recording method of two-way intercom audio and video according to claim 1, characterized in that: The step 3 specifically includes: The remote intercom collects audio data, encodes the audio data into audio compressed data in G711A / U, G722, G729 or OPUS format through the audio encoder in the remote intercom, and then packages the audio compressed data into an audio RTP packet. The audio RTP packet is sent to the local intercom according to the audio and video format of both parties and the audio and video transceiver ports negotiated by the local intercom and the remote intercom; the local intercom receives and parses the audio RTP packet to obtain audio compressed data, and sends it to the audio decoder for decoding. After decoding, the audio data is transmitted to the remote audio device and cached in the data pool; At the same time, the remote intercom collects video data, encodes the video data into video compression data in H264 or H265 format through the video encoder in the remote intercom, and then packages the video compression data into a video RTP packet. The video RTP packet is sent to the local intercom according to the audio and video format of both parties and the audio and video transceiver ports negotiated by the local intercom and the remote intercom; the local intercom receives the video RTP packet and parses it to obtain video compression data, and sends it to the video decoder for decoding. After decoding, the video data is obtained, transmitted to the remote camera, and cached in the data pool.

5. The NVR recording method of two-way intercom audio and video according to claim 1, characterized in that: In step 5, performing image synthesis based on the local video data collected by the local camera and the remote video data obtained by the remote camera specifically includes: When the local video data collected by the local camera is obtained, query whether there is remote video data in the remote camera. If so, obtain the remote video data and perform scaling operations with the local video data to synthesize the remote video and local video into one picture according to the custom ratio; if not, fill the entire picture with the local video.

6. The NVR recording method of two-way intercom audio and video according to claim 1, characterized in that: The step 5 of mixing the local audio data collected by the local audio device and the remote audio data acquired by the remote audio device specifically includes: Determine whether the remote audio device has remote audio data. If not, directly transfer the local audio data to the audio encoder. If present, retrieve the remote audio data within time T from the remote audio device and mix it with the local audio data within time T obtained by the current local audio device.

7. The NVR recording method of two-way intercom audio and video according to claim 1, characterized in that: The step 6 specifically includes: The synthesized video data is transferred to the video encoder for encoding to generate video compression data in H264, H265 or JPEG format, and then the video compression data is packaged into a video RTP packet, which is sent to the NVR device for video recording according to the audio and video format and audio and video sending port negotiated between the local intercom and the NVR device for recording; At the same time, the mixed audio data is transferred to the audio encoder for encoding to generate audio compression data in G711A / U, G722, G729 or OPUS format, and then the audio compression data is packaged into an audio RTP packet. According to the audio and video format used in recording and the audio and video sending port negotiated between the local intercom and the NVR device, it is sent to the NVR device for audio recording.

8. The NVR recording method of two-way intercom audio and video according to claim 1, characterized in that: The local intercom is a stair entrance machine or a door machine, and the remote intercom is an indoor machine, an APP, an analog phone or a property management machine.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the NVR recording method of two-way intercom audio and video of an intercom as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, an NVR recording method for two-way intercom audio and video of an intercom as described in any one of claims 1 to 8 is implemented.

Images