Mobile supervision record acquisition system and method based on audio and video real-time communication

Abstract

The invention provides a mobile supervision record acquisition system and method based on audio and video real-time communication, and relates to the technical field of supervision information acquisition, and the method comprises the steps that a command center generates a call request link, sends the call request link to a plurality of selected mobile devices, and establishes network connection with the command center; after the mobile equipment establishes connection with the command center, original audio data and original video data obtained by performing real-time audio and video acquisition on the site environment where the mobile equipment is located are packaged into a real-time transmission protocol data packet and uploaded to the command center; and the command center receives the real-time transmission protocol data packets uploaded by the connected mobile devices, decodes the real-time transmission protocol data packets to obtain corresponding original audio data and original video data, and renders the original audio data and the original video data on a screen of the command center for playing. The beneficial effects of the invention are that a supervisor opens a specific webpage through a mobile device to communicate with a command center in real time, and the method has the characteristics of real-time performance, high efficiency and convenience, and can greatly improve the supervision efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of supervision information collection, and in particular to a mobile supervision record collection system and method based on real-time audio and video communication. Background Art

[0002] With the rapid development of information technology, mobile supervision has gradually become an indispensable key means of modern supervision. In this process, real-time and accurate recording of the supervision scene plays a vital role in subsequent evidence preservation, case investigation and analysis. The traditional supervision recorder is a forensic technology equipment that integrates real-time audio and video recording, photography and video recording, which is carried by police officers when performing official duties. It integrates multiple functions such as video recording, photography, intercom, positioning, storage, etc. At the same time, it can digitally record the dynamic and static on-site conditions during the supervision process through 4G wireless real-time video transmission. The audio and video transmission of the supervision recorder is mainly realized through wireless transmission technology (such as 4G / 5G), and the collected audio and video data is transmitted to the back-end server or command center in real time. However, this method often has the disadvantages of high equipment maintenance cost, inconvenient carrying, high audio and video real-time communication delay, and low clarity. In addition, the hardware performance of the equipment is subject to the limitations of the equipment itself and cannot achieve the expected effect.

[0003] When encountering emergencies, the dispatch of specific personnel will also be greatly limited. Due to the suddenness and real-time nature of emergencies, when the command center wants to see the on-site scenes in time, if the supervisor does not carry a supervisory recorder, it will be impossible to collect the audio and video of the scene and transmit it to the command center in time. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a mobile supervision record collection system based on real-time audio and video communication, including a command center and mobile devices carried by multiple supervisors;

[0005] The mobile device comprises:

[0006] A device access module, used to receive a call request link sent by the command center and establish a network connection with the command center;

[0007] A data uploading module is used to encapsulate the original audio data and the original video data obtained by real-time audio and video acquisition of the on-site environment into a real-time transport protocol data packet and upload it to the command center after establishing a connection with the command center;

[0008] The command center includes:

[0009] A request sending module, used for generating the call request link and sending it to the selected multiple mobile devices;

[0010] The audio and video presentation module is used to receive the real-time transport protocol data packets uploaded by each connected mobile device and decode them to obtain the corresponding original audio data and original video data and render them on the screen of the command center for playback.

[0011] Preferably, the device access module includes:

[0012] The connection establishing unit is used to collect all candidate network paths between the mobile device and the command center after receiving and opening the call request link, and select the best candidate network path as the network path between the mobile device and the command center to establish a P2P connection.

[0013] Preferably, the data uploading module includes:

[0014] A device initialization unit, used to collect the original audio data at the scene through a microphone, and synchronously collect the original video data at the scene through a camera;

[0015] an audio processing unit, connected to the device initialization unit, configured to perform analog-to-digital conversion on the original audio data to obtain digital audio data, and then perform noise reduction and echo cancellation processing on the digital audio data and then encode the digital audio data to obtain encoded audio data;

[0016] a video processing unit, connected to the device initialization unit, configured to perform image preprocessing on the original video data to obtain processed video data, and then encode the processed video data to obtain encoded video data;

[0017] An encapsulation unit, connected to the audio processing unit and the video processing unit, for each of the connected devices to encapsulate the encoded audio data and the encoded video data collected at the same time into a real-time transport protocol data packet;

[0018] The data packet uploading unit is connected to the encapsulation unit and is used to upload each of the real-time transport protocol data packets to the command center through a network channel connected by P2P using the UDP protocol.

[0019] Preferably, the audio and video presentation module includes:

[0020] A decoding unit, configured to decapsulate the real-time transport protocol data packet to obtain the corresponding encoded audio data and the encoded video data, and then respectively decode the encoded audio data and the encoded video data to obtain the corresponding original audio data and the original video data;

[0021] The video playback unit is connected to the decoding unit and is used to synchronize the original audio data and the original video data obtained by decoding, and then render them on the screen of the command center for playback.

[0022] Preferably, the encapsulation unit is further used to associate the acquisition time corresponding to the original audio data and the original video data with the real-time transport protocol data packet as a timestamp, and to associate the corresponding sequence number during encapsulation with the real-time transport protocol data packet;

[0023] The audio and video presentation module further includes a sorting unit connected to the decoding unit, configured to add each received real-time transport protocol data packet to a queue to be decoded, and then sort each real-time transport protocol data packet in the queue to be decoded in a first-to-last order according to the timestamp and the sequence number associated with each real-time transport protocol data packet;

[0024] The decoding unit decapsulates the real-time transport protocol data packet located at the front of the queue to be decoded.

[0025] The present invention also provides a method for collecting mobile supervision records based on WebRTC, which is applied to the mobile supervision record collection system as described above, comprising:

[0026] Step S1, the command center generates the call request link and sends it to the selected multiple mobile devices;

[0027] Step S2, the mobile device receives the call request link sent by the command center and establishes a network connection with the command center;

[0028] Step S3, after the mobile device establishes a connection with the command center, the mobile device encapsulates the original audio data and the original video data obtained by real-time audio and video acquisition of the on-site environment into a real-time transport protocol data packet and uploads it to the command center;

[0029] Step S4, the command center receives the real-time transport protocol data packets uploaded by each of the connected mobile devices, decodes them to obtain the corresponding original audio data and original video data, and renders them on the screen of the command center for playback.

[0030] Preferably, step S2 includes collecting all candidate network paths between the mobile device and the command center after the mobile device receives and opens the call request link, and screening out the best candidate network path as the network path between the mobile device and the command center to establish a P2P connection.

[0031] Preferably, step S3 includes:

[0032] Step S31, the mobile device collects the original audio data at the scene through a microphone, and simultaneously collects the original video data at the scene through a camera;

[0033] Step S32, the mobile device performs analog-to-digital conversion on the original audio data to obtain digital audio data, and then performs noise reduction and echo cancellation processing on the digital audio data and then encodes the digital audio data to obtain encoded audio data;

[0034] Step S33, the mobile device performs image preprocessing on the original video data to obtain processed video data, and then encodes the processed video data to obtain encoded video data;

[0035] Step S34, each of the connected devices of the mobile device encapsulates the encoded audio data and the encoded video data collected at the same time into a real-time transport protocol data packet;

[0036] Step S35, the mobile device uploads each of the real-time transport protocol data packets to the command center via a network channel connected to P2P using the UDP protocol.

[0037] Preferably, step S4 comprises:

[0038] Step S41, the command center decapsulates the real-time transport protocol data packet to obtain the corresponding encoded audio data and the encoded video data, and then decodes the encoded audio data and the encoded video data to obtain the corresponding original audio data and the original video data;

[0039] In step S42, the command center synchronizes the decoded original audio data and the original video data, and then renders them on the screen of the command center for playback.

[0040] Preferably, in step S34, the mobile device further associates the acquisition time corresponding to the original audio data and the original video data as a timestamp with the real-time transport protocol data packet, and associates the corresponding sequence number during encapsulation with the real-time transport protocol data packet;

[0041] Then in step S4, before executing step S41, the following steps are also included:

[0042] Step S40, the command center adds each received real-time transport protocol data packet to a queue to be decoded, and then sorts each real-time transport protocol data packet in the queue to be decoded in a first-to-last order according to the timestamp and the sequence number associated with each real-time transport protocol data packet;

[0043] Then in step S41, the command center decapsulates the real-time transport protocol data packet located at the front of the queue to be decoded.

[0044] The above technical solution has the following advantages or beneficial effects: through real-time audio and video communication (WebRTC technology), a call request link based on the audio and video acquisition page of the H5 web page is generated, and the supervisor opens a specific web page through the call request link received by the mobile device (such as a smart phone or tablet) to conduct real-time audio and video calls or video chats, and communicate with other supervisors or the command center in real time. This method is real-time, efficient and convenient, and can greatly improve the efficiency of supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The present invention is a structural diagram of a mobile supervision record collection system based on real-time audio and video communication in a preferred embodiment of the present invention;

[0046] Figure 2 A flowchart of a method for collecting mobile supervision records based on real-time audio and video communication in a preferred embodiment of the present invention;

[0047] Figure 3 Schematic diagram of a sub-process of step S3 in a preferred embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of a sub-process of step S4 in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0049] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment, and other embodiments may also fall within the scope of the present invention as long as they conform to the gist of the present invention.

[0050] In a preferred embodiment of the present invention, based on the above problems existing in the prior art, a mobile supervision record collection system based on real-time audio and video communication is provided, which is characterized in that: Figure 1 As shown, it includes a command center and mobile devices carried by multiple supervisors;

[0051] The mobile device 10 includes:

[0052] The device access module 1 is used to receive the call request link sent by the command center and establish a network connection with the command center;

[0053] The data uploading module 2 is used to encapsulate the original audio data and original video data obtained by real-time audio and video acquisition of the on-site environment into a real-time transport protocol data packet and upload it to the command center after establishing a connection with the command center;

[0054] Command Center 20 includes:

[0055] A request sending module 3, used to generate a call request link and send it to a plurality of selected mobile devices;

[0056] The audio and video presentation module 4 is used to receive the real-time transport protocol data packets uploaded by each connected mobile device, decode them to obtain the corresponding original audio data and original video data, and render them on the screen of the command center for playback.

[0057] Specifically, a mobile supervision record collection system based on audio and video real-time communication (WebRTC technology) is provided in this embodiment, which replaces the solution of traditional mobile supervision recorders to collect audio and video transmission. By implementing an audio and video collection page based on an H5 web page through WebRTC technology, supervisors can open a specific web page through a text message link received by a mobile device (such as a smart phone or tablet computer) to conduct real-time audio and video calls or video chats, and communicate with other supervisors or command centers in real time. This method has the characteristics of real-time, high efficiency and convenience, which can greatly improve the efficiency of supervision. In addition, WebRTC (Web Real-Time Communication) is a real-time communication technology that allows network applications or sites to establish peer-to-peer (P2P, Peer-to-Peer) connections between browsers without the help of an intermediary, and realize the transmission of video streams and (or) audio streams or other arbitrary data. WebRTC's audio and video communication is based on the P2P architecture, and uses the ICE (Interactive Connectivity Establishment) protocol to establish a reliable communication connection in the P2P network. Through the ICE protocol, both parties can automatically discover and select the best network path to achieve real-time transmission of audio and video streams. In this process, WebRTC uses the RTP (Real-time Transport Protocol) protocol to transmit audio and video streams, and uses the SRTP (Secure Real-time Transport Protocol) encryption technology to ensure the security of audio and video data. It realizes functions such as media acquisition, encoding and decoding, network transmission and presentation on the browser side. WebRTC also uses protocols such as ICE (Interactive Connectivity Establishment), DTLS (Datagram Transport Layer Security) and SRTP (Secure Real-time Transport Protocol) to ensure the security and stability of communication.

[0058] Specifically, the operation process of the mobile supervision record collection system in this embodiment is as follows:

[0059] 1. Device access module startup:

[0060] The command center first sends a message containing a WebRTC call link to the mobile devices (such as mobile phones) of multiple supervisors via SMS, email or other means.

[0061] After receiving the information, the supervisor clicks the link to open the H5-based audio and video acquisition page on the mobile device.

[0062] The browser on the mobile device establishes a WebRTC P2P connection with the server of the command center and accesses the command center's system as a connected device.

[0063] 2. Real-time audio and video acquisition:

[0064] The connected device (the supervisor's mobile device) begins to use the mobile device's camera and microphone to collect real-time audio and video of the on-site environment.

[0065] 3. Data upload module processing:

[0066] The collected original audio data and original video data are encapsulated into real-time transport protocol (such as RTP) data packets.

[0067] Each packet is associated with a timestamp and sequence number to ensure the order and integrity of the data.

[0068] Connected devices upload these data packets to the command center in real time via P2P connections.

[0069] 4. Audio and video presentation module processing:

[0070] The command center receives the uploaded real-time transport protocol data packet.

[0071] According to the timestamps and sequence numbers in the data packets, these data packets are reordered and completed to ensure data continuity.

[0072] The sorted data packets are decoded and restored to the original audio data and video data.

[0073] The decoded audio and video data is rendered on the screen of the command center, enabling real-time playback and monitoring.

[0074] The device access module in this embodiment realizes a fast and convenient P2P connection between the command center and the supervisor's mobile device without installing additional applications or devices. Through the H5 web page, the compatibility and usability of the system are improved, and the difficulty of operation for supervisors is reduced. When there is no supervisory recorder, the supervisor's personal mobile device can also be used to collect audio and video of the emergency scene.

[0075] The data upload module in this embodiment encapsulates the audio and video data into a real-time transport protocol data packet, ensuring the real-time and integrity of the audio and video data. The data packets are managed by timestamps and sequence numbers, effectively solving the problems of disorder and packet loss during data transmission. The P2P transmission mechanism of WebRTC is used to reduce the burden on the server and improve the transmission efficiency and stability of the system.

[0076] The audio and video presentation module realizes the real-time decoding and rendering of audio and video data, ensuring that the command center can see the audio and video information of the supervision site in real time and clearly. By reordering and completing data packets, the continuity and integrity of audio and video information are ensured, and the monitoring effect is improved. It provides an intuitive screen playback method, which makes it easier for command center personnel to understand the situation of the supervision site in real time and make quick and accurate decisions.

[0077] In a preferred embodiment of the present invention, the device access module 1 includes:

[0078] The connection establishing unit 11 is used to collect all candidate network paths between the mobile device and the command center after receiving and opening the call request link, and select the best candidate network path as the network path between the mobile device and the command center to establish a P2P connection.

[0079] Specifically, in this embodiment, the device access module is used to establish a peer-to-peer (P2P) connection between browsers and initialize mobile devices. WebRTC uses the ICE (Interactive Connectivity Establishment) framework to establish a P2P connection and handle NAT (Network Address Translation) penetration issues to ensure secure and reliable data transmission.

[0080] Specifically, the ICE framework establishes a connection through the following steps:

[0081] Collect all paths: The ICE framework first collects all possible communication paths, including the local network interface, the public network address after NAT mapping, and the relay address provided by the TURN server.

[0082] Generate candidates: Based on the collected paths, the ICE framework generates candidates, which are candidate addresses for media transmission. These candidates consist of local and remote candidates for subsequent connectivity checks.

[0083] Connectivity check: The ICE framework performs connectivity checks on the generated candidates to determine which candidates can be connected. This is usually done by sending and receiving test packets.

[0084] Select the best path: Based on the results of the connectivity check, the ICE framework selects the best path for data transmission. This usually involves prioritizing the candidates and selecting the candidate with the highest priority as the network path between the mobile device and the command center to establish a P2P connection and access the command center as a connected device;

[0085] WebRTC uses the microphone for audio collection. On Android, WebRTC first checks whether the microphone permission has been obtained, and then starts to capture audio data from the microphone. This process involves audio sampling, quantization, encoding and other steps, and finally generates audio data packets that can be transmitted.

[0086] WebRTC captures video through the camera. Similarly, it first checks whether the camera permission has been obtained, and then starts capturing video frames from the camera. After the video frames are compressed and encoded, video data packets that can be transmitted are generated.

[0087] In a preferred embodiment of the present invention, the data uploading module 2 includes:

[0088] The device initialization unit 21 is used to obtain microphone and camera permissions for each mobile device connected to the command center, and collect original audio data of the scene through the microphone and original video data of the scene through the camera after successfully obtaining the permissions;

[0089] The audio processing unit 22 is connected to the device initialization unit 21 and is used to perform analog-to-digital conversion on the original audio data to obtain digital audio data, and then perform noise reduction and echo cancellation processing on the digital audio data and then encode it to obtain encoded audio data;

[0090] The video processing unit 23 is used for each connected device to perform image preprocessing on the original video data to obtain processed video data, and then encode the processed video data to obtain encoded video data;

[0091] The encapsulation unit 24 is connected to the audio processing unit 22 and the video processing unit 23, and is used for each connected device to encapsulate the encoded audio data and the encoded video data collected at the same time into a real-time transport protocol data packet;

[0092] The data packet uploading unit 24 is connected to the encapsulation unit 24 and is used to upload each real-time transport protocol data packet to the command center through the network channel of the P2P connection using the UDP protocol.

[0093] Specifically, in this embodiment, the original audio data captured by the microphone is converted from analog to digital (ADC), and then encoded into a format suitable for the current network transmission, such as AAC, Opus, etc., after processing such as noise reduction and echo cancellation. In order to improve transmission efficiency, a compression algorithm will be performed during the encoding process to reduce the amount of transmitted data.

[0094] Also, after capturing the original video frame through the camera, capturing the image signal and performing image preprocessing such as color correction, scaling, and cropping, the video frame is encoded into a compressed format by selecting an appropriate encoding format such as VP8, VP9, ​​H.264, and H.265. The video encoding process includes steps such as intra-frame prediction, inter-frame prediction, transformation, quantization, and entropy coding to reduce the size of the video data.

[0095] The encoded audio and video data are encapsulated into RTP (Real-time Transport Protocol) packets. Each RTP packet contains a timestamp and sequence number, which are used to reorder and synchronize audio and video data at the receiving end.

[0096] Furthermore, in addition to RTP packets, WebRTC also uses the RTCP (Real-time Transport Control Protocol) protocol to send control information, such as transmission quality reports, synchronization source identifiers (SSRC), etc. WebRTC uses the RTCP protocol to monitor transmission quality and provide feedback. The command center can estimate network delays, packet loss rates and other parameters through RTCP reports in order to make corresponding adjustments and optimizations. Based on the information reported by RTCP, WebRTC can perform congestion control, rate adjustment and other operations to optimize the quality of audio and video transmission.

[0097] In a preferred embodiment of the present invention, the encapsulation unit is further used to associate the acquisition time corresponding to the original audio data and the original video data with the real-time transport protocol data packet as a timestamp, and to associate the corresponding sequence number during encapsulation with the real-time transport protocol data packet;

[0098] The audio and video presentation module 4 includes:

[0099] The sorting unit 41 is connected to the decoding unit 42, and is used to add each received real-time transport protocol data packet to a queue to be decoded, and then sort each real-time transport protocol data packet in the queue to be decoded in a first-to-last order according to the timestamp and sequence number associated with each real-time transport protocol data packet;

[0100] The decoding unit 42 is connected to the sorting unit 41, and is used to decapsulate a real-time transport protocol data packet located at the front of the queue to be decoded to obtain corresponding encoded audio data and encoded video data, and then decode the encoded audio data and the encoded video data to obtain corresponding original audio data and original video data respectively;

[0101] The video playing unit 43 is connected to the decoding unit 42 and is used for time synchronization of the decoded original audio data and the original video data, and then rendering them on the screen of the command center for playing.

[0102] Specifically, in this embodiment, the RTP data packet is transmitted via the UDP (User Datagram Protocol) protocol. UDP is a connectionless transmission protocol suitable for real-time communication applications.

[0103] After receiving the RTP data packet, the command center first decapsulates it and extracts the audio and video data; then, the receiving end decodes the audio and video data and restores it to the original data. The decoding process is the opposite of the encoding process, involving steps such as decompression, inverse transformation, and inverse quantization; the decoded audio and video data needs to be synchronized to ensure that the audio and video remain synchronized during playback.

[0104] Due to network delay and jitter, received RTP packets may be out of order or lost. The jitter buffer establishes a queue to be decoded to cache received packets, and reorders the packets in the queue to be decoded according to the timestamp and sequence number (for example, reordering the three packets with sequence number CDB to obtain three packets with sequence numbers B, C, and D, where B, C, and D are used to indicate the order of sequence numbers) and completes (for example, inserting the packet with sequence number B between the packets with sequence numbers A and C, where ABC is used to indicate the order of sequence numbers). This helps reduce audio and video playback quality issues caused by network problems.

[0105] The decoded and synchronously processed audio data can be played in real time on the large screen platform of the command center.

[0106] The video data is rendered to a certain area on the screen for the user to watch. The rendering process may involve scaling, rotating, and superimposing the video frames.

[0107] The command center can remotely direct the evacuation of vehicles and personnel through real-time audio and video images, and promptly initiate response emergency plans to improve the efficiency of supervision and disposal.

[0108] The present invention also provides a mobile supervision record collection method based on real-time audio and video communication, which is applied to the above-mentioned mobile supervision record collection system, such as Figure 2 As shown, including:

[0109] Step S1, the command center generates a call request link and sends it to multiple selected mobile devices;

[0110] Step S2, the mobile device receives the call request link sent by the command center and establishes a network connection with the command center;

[0111] Step S3, after the mobile device establishes a connection with the command center, the mobile device encapsulates the original audio data and original video data obtained by real-time audio and video acquisition of the on-site environment into a real-time transport protocol data packet and uploads it to the command center;

[0112] Step S4, the command center receives the real-time transport protocol data packets uploaded by each connected mobile device and decodes them to obtain the corresponding original audio data and original video data and renders them on the screen of the command center for playback.

[0113] In a preferred embodiment of the present invention, step S2 includes collecting all candidate network paths between the mobile device and the command center after the mobile device receives and opens the call request link, and screening out the best candidate network path as the network path between the mobile device and the command center to establish a P2P connection.

[0114] In a preferred embodiment of the present invention, Figure 3 As shown, step S3 includes:

[0115] Step S31, the mobile device collects original audio data of the scene through a microphone, and simultaneously collects original video data of the scene through a camera;

[0116] Step S32, the mobile device performs analog-to-digital conversion on the original audio data to obtain digital audio data, and then performs noise reduction and echo cancellation processing on the digital audio data and then encodes the digital audio data to obtain encoded audio data;

[0117] Step S33, the mobile device performs image preprocessing on the original video data to obtain processed video data, and then encodes the processed video data to obtain encoded video data;

[0118] Step S34, each connected device of the mobile device encapsulates the encoded audio data and the encoded video data collected at the same time into a real-time transport protocol data packet;

[0119] Step S35, the mobile device uploads each real-time transport protocol data packet to the command center through the network channel of the P2P connection using the UDP protocol.

[0120] In a preferred embodiment of the present invention, step S4 includes:

[0121] Step S41, the command center decapsulates the real-time transport protocol data packet to obtain corresponding encoded audio data and encoded video data, and then decodes the encoded audio data and the encoded video data to obtain corresponding original audio data and original video data;

[0122] In step S42, the command center synchronizes the decoded original audio data and original video data, and then renders them on the screen of the command center for playback.

[0123] In a preferred embodiment of the present invention, in step S34, the mobile device further associates the acquisition time corresponding to the original audio data and the original video data with the real-time transport protocol data packet as a timestamp, and associates the corresponding sequence number during encapsulation with the real-time transport protocol data packet;

[0124] Then in step S4, if Figure 4 As shown, before executing step S41, the method further includes:

[0125] Step S40, the command center adds each received real-time transport protocol data packet to a queue to be decoded, and then sorts each real-time transport protocol data packet in the queue to be decoded in a first-to-last order according to the timestamp and sequence number associated with each real-time transport protocol data packet;

[0126] Then in step S41, the command center decapsulates a real-time transport protocol data packet located at the front of the queue to be decoded.

[0127] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of this specification and illustrations should be included in the protection scope of the present invention.

Claims

1. A mobile supervision record collection system based on real-time audio and video communication, characterized in that: Includes the command center and mobile devices carried by multiple supervisors; The mobile device comprises: A device access module, used to receive a call request link sent by the command center and establish a network connection with the command center; A data uploading module is used to encapsulate the original audio data and the original video data obtained by real-time audio and video acquisition of the on-site environment into a real-time transport protocol data packet and upload it to the command center after establishing a connection with the command center; The command center includes: A request sending module, used for generating the call request link and sending it to the selected multiple mobile devices; The audio and video presentation module is used to receive the real-time transport protocol data packets uploaded by each connected mobile device and decode them to obtain the corresponding original audio data and original video data and render them on the screen of the command center for playback.

2. The mobile supervision record collection system according to claim 1 is characterized in that: The device access module includes: The connection establishing unit is used to collect all candidate network paths between the mobile device and the command center after receiving and opening the call request link, and select the best candidate network path as the network path between the mobile device and the command center to establish a P2P connection.

3. The mobile supervision record collection system according to claim 1, characterized in that: The data upload module includes: A device initialization unit, used to collect the original audio data at the scene through a microphone, and synchronously collect the original video data at the scene through a camera; an audio processing unit, connected to the device initialization unit, configured to perform analog-to-digital conversion on the original audio data to obtain digital audio data, and then perform noise reduction and echo cancellation processing on the digital audio data and then encode the digital audio data to obtain encoded audio data; a video processing unit, connected to the device initialization unit, configured to perform image preprocessing on the original video data to obtain processed video data, and then encode the processed video data to obtain encoded video data; An encapsulation unit, connected to the audio processing unit and the video processing unit, for each of the connected devices to encapsulate the encoded audio data and the encoded video data collected at the same time into a real-time transport protocol data packet; The data packet uploading unit is connected to the encapsulation unit and is used to upload each of the real-time transport protocol data packets to the command center through a network channel connected by P2P using the UDP protocol.

4. The mobile supervision record collection system according to claim 3 is characterized in that: The audio and video presentation module comprises: A decoding unit, configured to decapsulate the real-time transport protocol data packet to obtain the corresponding encoded audio data and the encoded video data, and then respectively decode the encoded audio data and the encoded video data to obtain the corresponding original audio data and the original video data; The video playback unit is connected to the decoding unit and is used to synchronize the original audio data and the original video data obtained by decoding, and then render them on the screen of the command center for playback.

5. The mobile supervision record collection system according to claim 4 is characterized in that: The encapsulation unit is further used to associate the acquisition time corresponding to the original audio data and the original video data with the real-time transport protocol data packet as a timestamp, and to associate the corresponding sequence number during encapsulation with the real-time transport protocol data packet; The audio and video presentation module further includes a sorting unit connected to the decoding unit, configured to add each received real-time transport protocol data packet to a queue to be decoded, and then sort each real-time transport protocol data packet in the queue to be decoded in a first-to-last order according to the timestamp and the sequence number associated with each real-time transport protocol data packet; The decoding unit decapsulates the real-time transport protocol data packet located at the front of the queue to be decoded.

6. A mobile supervision record collection method based on real-time audio and video communication, characterized in that: The mobile supervision record collection system as claimed in any one of claims 1 to 5 comprises: Step S1, the command center generates the call request link and sends it to the selected multiple mobile devices; Step S2, the mobile device receives the call request link sent by the command center and establishes a network connection with the command center; Step S3, after the mobile device establishes a connection with the command center, the mobile device encapsulates the original audio data and the original video data obtained by real-time audio and video acquisition of the on-site environment into a real-time transport protocol data packet and uploads it to the command center; Step S4, the command center receives the real-time transport protocol data packets uploaded by each of the connected mobile devices, decodes them to obtain the corresponding original audio data and original video data, and renders them on the screen of the command center for playback.

7. The method for collecting mobile supervision records according to claim 6, characterized in that: Step S2 includes collecting all candidate network paths between the mobile device and the command center after the mobile device receives and opens the call request link, and selecting the best candidate network path as the network path between the mobile device and the command center to establish a P2P connection.

8. The method for collecting mobile supervision records according to claim 6, characterized in that: Step S3 includes: Step S31, the mobile device collects the original audio data at the scene through a microphone, and simultaneously collects the original video data at the scene through a camera; Step S32, the mobile device performs analog-to-digital conversion on the original audio data to obtain digital audio data, and then performs noise reduction and echo cancellation processing on the digital audio data and then encodes the digital audio data to obtain encoded audio data; Step S33, the mobile device performs image preprocessing on the original video data to obtain processed video data, and then encodes the processed video data to obtain encoded video data; Step S34, each of the connected devices of the mobile device encapsulates the encoded audio data and the encoded video data collected at the same time into a real-time transport protocol data packet; Step S35, the mobile device uploads each of the real-time transport protocol data packets to the command center via a network channel connected to P2P using the UDP protocol.

9. The method for collecting mobile supervision records according to claim 8, characterized in that: The step S4 comprises: Step S41, the command center decapsulates the real-time transport protocol data packet to obtain the corresponding encoded audio data and the encoded video data, and then decodes the encoded audio data and the encoded video data to obtain the corresponding original audio data and the original video data; In step S42, the command center synchronizes the decoded original audio data and the original video data, and then renders them on the screen of the command center for playback.

10. The mobile supervision record collection system according to claim 9, characterized in that: In step S34, the mobile device further associates the acquisition time corresponding to the original audio data and the original video data with the real-time transport protocol data packet as a timestamp, and associates the corresponding sequence number during encapsulation with the real-time transport protocol data packet; Then in step S4, before executing step S41, the following steps are also included: Step S40, the command center adds each received real-time transport protocol data packet to a queue to be decoded, and then sorts each real-time transport protocol data packet in the queue to be decoded in a first-to-last order according to the timestamp and the sequence number associated with each real-time transport protocol data packet; Then in step S41, the command center decapsulates the real-time transport protocol data packet located at the front of the queue to be decoded.

Images