Video recording device and audio and video synchronous output method

Abstract

The embodiment of the present application provides a recording device and a method for synchronous audio and video output, which relates to the field of audio and video technology. The recording device obtains the original audio and video data of the target image acquisition device indicated by the output instruction and parses it to obtain the target video data, obtains the target audio data of the device bound to the target image acquisition device, synchronously outputs the target audio data and the target video data, switches the device bound to the target image acquisition device to the device indicated by the switching instruction upon receiving the switching instruction, obtains the audio data of the device indicated by the switching instruction as the new target audio data, plays the target video data synchronously with the new target audio data, and upon receiving the confirmation instruction for the target video data and the new target audio data, outputs the target video data synchronously with the new target audio data to obtain the target audio and video data. The solution provided by the embodiment of the present application can simplify the evidence collection process.

Description

Technical Field

[0001] The present application relates to the field of audio and video technology, and in particular to a video recording device and an audio and video synchronous output method. Background Art

[0002] Traditional single video surveillance systems can no longer meet complex and changing detection needs. For example, in some areas of some hospitals, bank counters, service halls and other non-private public service places, or in some places and environments where information collection is authorized, it is necessary not only to monitor abnormal situations through video images, but also to capture the conversations between customers and staff in order to provide comprehensive and objective evidence materials in the event of disputes.

[0003] In the related art, traditional audio and video acquisition equipment such as cameras are usually used to collect audio and video data and save them, so that the staff can monitor or play them back in real time through a recorder. However, when collecting evidence in some scenes, the camera that shoots the scene where the conversation takes place cannot clearly collect the content of the conversation, resulting in incomplete evidence collection. For example, the prescription issued by the doctor to the patient includes medicine A and medicine B. The pharmacy happens to have no stock of medicine A. The pharmacy staff replaces medicine A with medicine C with the same ingredients and effects, and explains the reason for the replacement of medicines to the patient. When a dispute occurs between the patient and the hospital, the hospital needs to retrieve the audio and video data of the pharmacy for self-evidence. If the audio and video data collected by the pharmacy camera is directly retrieved, since the audio in the audio and video data collected by the pharmacy camera is not clear, it can only be known from the audio and video data that the pharmacy staff and the patient have a conversation, but the content of the conversation cannot be known. The audio data collected by the sound pickup device set at the pharmacy medicine window and the video data collected by the pharmacy camera can only be retrieved separately for evidence collection, and the evidence collection process is relatively cumbersome. Summary of the invention

[0004] The purpose of the embodiment of the present application is to provide a video recording device and a method for synchronously outputting audio and video to simplify the evidence collection process. The specific technical solution is as follows:

[0005] In a first aspect of an embodiment of the present application, a video recording device is provided, the video recording device being used to, in response to a display instruction, obtain original audio and video data from a target image acquisition device indicated by the display instruction, parse the original audio and video data to obtain target video data and first audio data; obtain second audio data from a device bound to the target image acquisition device as target audio data; and synchronously display the target video data and target audio data with the same timestamp; wherein the video recording device has established a communication connection with at least one sound pickup device in advance, and has established a communication connection with at least one image acquisition device in advance, the device bound to the target image acquisition device is the target image acquisition device or the first sound pickup device, and the first sound pickup device is one of the at least one sound pickup devices;

[0006] The video recording device is further used to, in response to a switching instruction, terminate the step of acquiring the second audio data from the device bound to the target image acquisition device, switch the device bound to the target image acquisition device to the device indicated by the switching instruction, acquire the third audio data from the device bound to the target image acquisition device as new target audio data; and synchronously display the target video data and the new target audio data with the same timestamp; wherein the device indicated by the switching instruction is the target image acquisition device or the second sound pickup device, and the second sound pickup device is a sound pickup device other than the first sound pickup device among the at least one sound pickup device;

[0007] The video recording device is also used to, in response to a confirmation instruction for the target video data and the new target audio data, synchronously output the target video data and the new target audio data to obtain the target audio and video data.

[0008] A second aspect of the embodiments of the present application provides a method for synchronously outputting audio and video, which is applied to a video recording device, and the method includes:

[0009] In response to a display instruction, original audio and video data from a target image acquisition device indicated by the display instruction is obtained; the original audio and video data is parsed to obtain target video data and first audio data, and second audio data from a device bound to the target image acquisition device is obtained as target audio data, and the target video data and target audio data having the same timestamp are synchronously displayed;

[0010] In response to the switching instruction, the step of acquiring the second audio data from the device bound to the target image acquisition device is terminated, the device bound to the target image acquisition device is switched to the device indicated by the switching instruction, and the third audio data from the second device bound to the target image acquisition device is acquired as the new target audio data;

[0011] Synchronously displaying the target video data and the target audio data having the same timestamp;

[0012] In response to a confirmation instruction for the target video data and the new target audio data, synchronously outputting the target video data and the new target audio data to obtain target audio and video data;

[0013] Among them, the video recording device has pre-established a communication connection with at least one sound pickup device, and has pre-established a communication connection with at least one image acquisition device, the device bound to the target image acquisition device is the target image acquisition device or the first sound pickup device, the device indicated by the switching instruction is the target image acquisition device or the second sound pickup device, the first sound pickup device is a sound pickup device in the at least one sound pickup device, and the second sound pickup device is a sound pickup device in the at least one sound pickup device except the first sound pickup device.

[0014] Beneficial effects of the embodiments of the present application:

[0015] The embodiment of the present application provides a recording device and an audio and video synchronous output method. After receiving a display instruction, the recording device obtains and parses the original audio and video data of the target image acquisition device indicated by the output instruction to obtain the target video data and the first audio data, obtains the second audio data of the device bound to the target image acquisition device as the target audio data, and synchronously outputs the target audio data and the target video data. When receiving a switching instruction, the second audio data of the device bound to the target image acquisition device is no longer output, but the device bound to the target image acquisition device is switched to the device indicated by the switching instruction, and the third audio data of the device indicated by the switching instruction is obtained as the new target audio data, and the target video data is synchronously played with the new target audio data, and when receiving a confirmation instruction for the target video data and the new target audio data, the target video data is synchronously output with the new target audio data to obtain the target audio and video data. Through the embodiment of the present application, when collecting evidence for audio and video data, since the source of the audio data can be switched according to the needs, when the audio is not clear enough, clear and accurate audio data can be obtained by switching the audio source, and then the target audio and video data with clear audio can be output, which simplifies the evidence collection process.

[0016] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0018] Figure 1a This is an example diagram of the connection relationship between common video recording equipment, sound pickup equipment, and image acquisition equipment;

[0019] Figure 1b An example diagram of the connection relationship between the video recording device, the sound pickup device, and the image acquisition device provided in the embodiment of the present application;

[0020] Figure 2a-1 , Figure 2a-2 , Figure 2a-3 An example diagram of a video recording device provided in an embodiment of the present application establishing a static binding relationship between a sound pickup device and an image acquisition device on a GUI side;

[0021] Figure 2b-1 , Figure 2b-2 , Figure 2b-3 An example diagram of a video recording device providing an embodiment of the present application establishing a static binding relationship between a sound pickup device and an image acquisition device on a Web terminal;

[0022] Figure 3 An example diagram of establishing a static binding relationship between a sound pickup device and an image acquisition device and performing data synchronization output provided in an embodiment of the present application;

[0023] Figure 4 A flowchart of data synchronization output of a video recording device under static binding provided in an embodiment of the present application;

[0024] Figure 5a and Figure 5b An example diagram of previewing a video recording device provided in an embodiment of the present application under dynamic binding on a GUI interface;

[0025] Figure 6 A schematic diagram of the first structure of a video recording device provided in an embodiment of the present application;

[0026] Figure 7 An internal interaction diagram of a video recording device provided in an embodiment of the present application when the video recording device is previewed under dynamic binding on a GUI side;

[0027] Figure 8aA flowchart of previewing a video recording device under dynamic binding on a GUI side provided in an embodiment of the present application;

[0028] Figure 8b A flowchart of switching the audio source to the camera itself when previewing the video recording device under dynamic binding on the GUI side provided by the embodiment of the present application;

[0029] Figure 8c A flowchart of switching the audio source to other sound pickup devices when previewing the video recording device under dynamic binding on the GUI side provided in an embodiment of the present application;

[0030] Fig. 9 An example diagram of previewing a video recording device provided in an embodiment of the present application under dynamic binding on a Web interface;

[0031] Fig.10 An interactive diagram of the video recording device provided in the embodiment of the present application when previewing under dynamic binding on the Web side;

[0032] Fig.11a A flowchart of previewing a video recording device under dynamic binding on a Web terminal provided in an embodiment of the present application;

[0033] Fig.11b A flowchart of switching the audio source to the camera itself when previewing the video recording device under dynamic binding on the Web side provided in an embodiment of the present application;

[0034] Fig.11c A flowchart of switching the audio source to other sound pickup devices when the video recording device provided in the embodiment of the present application is previewed under dynamic binding on the Web side;

[0035] Fig.12 An example diagram of a video recording device provided in an embodiment of the present application performing playback under dynamic binding of a GUI interface;

[0036] Fig.13 A second structural schematic diagram of a video recording device provided in an embodiment of the present application;

[0037] Fig.14 An internal interaction diagram of a video recording device provided in an embodiment of the present application when the video recording device is played back under dynamic binding on a GUI side;

[0038] Fig.15a A flowchart of a video recording device provided in an embodiment of the present application for playback under dynamic binding on a GUI side;

[0039] Fig.15b A flowchart of a recording device provided in an embodiment of the present application that switches the audio source to the camera itself during playback under dynamic binding on the GUI side;

[0040] Fig.15cA flowchart of a video recording device provided in an embodiment of the present application switching the audio source to other sound pickup devices during playback under dynamic binding on the GUI side;

[0041] Fig.16 An interactive diagram of the video recording device provided in the embodiment of the present application when playing back the video under dynamic binding on the Web side;

[0042] Fig.17a A flowchart of the video recording device provided in the embodiment of the present application for playback under dynamic binding on the Web side;

[0043] Fig.17b An example diagram of a recording device provided in an embodiment of the present application switching the audio source to the camera itself during playback under dynamic binding on the Web side;

[0044] Fig.17c An example diagram of a recording device provided in an embodiment of the present application switching the audio source to another sound pickup device during playback under dynamic binding on the Web side;

[0045] Fig.18a , Fig.18b , Fig.18c An example diagram of exporting a video recording device provided in an embodiment of the present application under dynamic binding on a GUI interface;

[0046] Fig.19 A third structural schematic diagram of the video recording device provided in the embodiment of the present application;

[0047] Fig. 20 An internal interaction diagram of a video recording device provided in an embodiment of the present application when the video recording device is exported under dynamic binding on a GUI side;

[0048] Fig.21 A flowchart of exporting a video recording device under dynamic binding on a GUI side provided in an embodiment of the present application;

[0049] Fig. 22 An example diagram of exporting a video recording device provided in an embodiment of the present application under dynamic binding on a Web interface;

[0050] Fig.23 An interactive diagram of the video recording device provided in the embodiment of the present application when exporting under dynamic binding on the Web side;

[0051] Fig.24 A flowchart of exporting a video recording device under dynamic binding on a Web side provided in an embodiment of the present application;

[0052] Fig.25 A schematic diagram of the audio and video synchronous output method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0054] First, the professional terms in the embodiments of the present application are explained:

[0055] GUI (Graphical User Interface) usually refers to the user interface of a desktop application, including elements such as windows, buttons, menus, icons, and text boxes, which provides users with an intuitive way to interact with the software.

[0056] Web (World Wide Web, global information network) refers to browser-based operations that do not require the installation of special application software on local computers. It mainly accesses and uses websites or applications through the Internet for interactive operations, and data storage and processing are mostly performed on remote servers.

[0057] The IOT (Internet of Things) module is a module that connects any device to the network through information sensing devices according to agreed protocols to achieve functions such as intelligent identification.

[0058] The PAL layer (Peripheral Abstraction Layer) is an intermediate layer between software and hardware. It provides a unified API interface for upper-layer applications, so that developers do not have to worry about the specific implementation details of the underlying hardware, thereby enabling efficient program development on different hardware platforms.

[0059] ISAPI (Internet Server Application Programming Interface) module is a dynamic link library used to extend the functions of a Web server.

[0060] API (Application Programming Interface) is a communication protocol used to achieve interaction and data exchange between two software systems.

[0061] The RTSP (Real Time Streaming Protocol) module is a module that controls the transmission of real-time streaming media based on the real-time streaming protocol.

[0062] DSP (Digital Signal Processor) module is a chip or processor that implements digital signal processing technology.

[0063] RTP (Real-time Transport Protocol) encapsulation refers to packaging digital audio and video data in the data packet format defined by the real-time transport protocol for easy network transmission.

[0064] In the related technology, an image acquisition device capable of collecting audio and video data is usually connected to a video recording device for communication, and the audio and video data collected by the image acquisition device is sent to the video recording device for user playback or export. However, the audio source and the video source in the audio and video data are both from the same image acquisition device. When the image acquisition device is far away from the person in the video screen, effective audio cannot be collected. If evidence is required, only the video data collected by the image acquisition device and the audio data collected by the sound pickup device set in the same space as the image acquisition device can be retrieved. For example, Figure 1a As shown, in a certain space, a camera is usually used to capture the audio and video data of the space, and then the captured audio and video are sent to a recording device. However, the camera itself can only clearly capture sounds within a limited range, such as Figure 1a The circle where the thin dotted line is located indicates the range in which the camera can clearly collect sounds. In this way, the camera can only collect the sounds of people who are close to the camera. The sounds of people who are far away from the camera may not be clearly collected by the camera and can only be collected by the sound collection device. The collection range of the sound collection device is as follows: Figure 1a The thick dotted line in the figure indicates that if it is necessary to collect evidence about the behavior and communication content of person A and person G within a certain period of time, the audio of the communication content between person A and person G in the audio and video data collected by the camera may not be clear, and the communication content between person A and person G cannot be accurately known. The only option is to extract the audio data collected by the sound pickup device during this period of time, and combine the video data collected by the camera with the audio data collected by the sound pickup device as evidence. The evidence collection process is relatively cumbersome.

[0065] In order to simplify the evidence collection process, the first aspect of the embodiment of the present application first provides a video recording device, which is pre-established with at least one sound pickup device and pre-established with at least one image acquisition device. For example, Figure 1b As shown, the video recording device 10 establishes communication connections with three sound pickup devices 20 (sound pickup device 20-1, sound pickup device 20-2, sound pickup device 20-3) and two image acquisition devices 30 (image acquisition device 30-1, image acquisition device 30-2);

[0066] The video recording device 10 is used to, in response to a display instruction, obtain original audio and video data from a target image acquisition device indicated by the display instruction, parse the original audio and video data to obtain target video data and first audio data; obtain second audio data from a device bound to the target image acquisition device as target audio data; and synchronously display the target video data and target audio data with the same timestamp;

[0067] The video recording device 10 is also used to, in response to a switching instruction, terminate the step of acquiring the second audio data from the device bound to the target image acquisition device, switch the device bound to the target image acquisition device to the device indicated by the switching instruction, acquire the third audio data from the device bound to the target image acquisition device as new target audio data; and synchronously display the target video data and the new target audio data with the same timestamp; wherein the device indicated by the switching instruction is the target image acquisition device or the second sound pickup device, and the second sound pickup device is a sound pickup device other than the first sound pickup device among the at least one sound pickup device;

[0068] The video recording device 10 is further configured to, in response to a confirmation instruction for the target video data and the new target audio data, synchronously output the target video data and the new target audio data to obtain target audio and video data;

[0069] The video recording device has established a communication connection with at least one sound pickup device in advance, and has established a communication connection with at least one image acquisition device in advance, the device bound to the target image acquisition device is the target image acquisition device or the first sound pickup device, and the first sound pickup device is one of the at least one sound pickup devices;

[0070] Understandably, Figure 1a and Figure 1b This is just an example. In other possible embodiments, the number of sound pickup devices that establish communication connection with the video recording device can be one or more, and the number of image acquisition devices that establish communication connection with the video recording device can be one or more. Exemplarily, the video recording device can establish connection with one image acquisition device and one sound pickup device respectively at the same time, or the video recording device can establish connection with two sound pickup devices and four image acquisition devices respectively at the same time, or the video recording device can establish communication connection with four sound pickup devices and five image acquisition devices respectively at the same time. All of these are possible, and this application does not limit the number of sound pickup devices and the number of image acquisition devices.

[0071] Take the doctor's office as an example. Figure 1aAs shown, with the authorization for information collection, a camera and a sound pickup device can be set up in the consulting room. The range in which the sound pickup device can effectively collect sound is represented by a thick dotted circle. It can be seen that the sound pickup device can effectively collect the sounds of people A~G around the sound pickup device in the consulting room. The sound pickup device is set on the consulting table (i.e., the table in the figure), and a communication connection is established with the video recording device. The collected audio is sent to the video recording device. The video recording device can synchronously output the video data in the audio and video data collected by the camera and the audio data collected by the sound pickup device. Compared with the audio and video data collected by the camera itself, the embodiment of the present application can obtain audio and video data with clear audio.

[0072] Through the video recording device provided in the embodiment of the present application, when collecting evidence from audio and video data, since the source of the audio data in the video recording device can be switched according to demand, when the audio is not clear enough, clear and accurate audio data can be obtained by simply switching the audio source, and then the target audio and video data with clear audio can be output, thereby simplifying the evidence collection process.

[0073] The device bound to the target image acquisition device may be the image acquisition device itself or a sound pickup device. When the device bound to the target image acquisition device is the target image acquisition device itself, the video recording device may directly display the original audio and video data from the target image acquisition device.

[0074] When the device bound to the target image acquisition device is a sound pickup device, the locations of the sound pickup device and the image acquisition device in the embodiment of the present application can be in the same environment or in different environments. For example, when collecting evidence on the communication content at a bank counter, the camera can be a camera installed in the bank lobby, and the sound pickup device can be a sound pickup device installed at the counter; when evidence is needed in a doctor's office, the camera and the sound pickup device can both be installed in the office. It can be understood that the hospitals, banks, government halls, etc. in the embodiments of the present application are merely possible application scenarios for the video recording equipment provided in the embodiments of the present application. In other possible embodiments, the video recording equipment provided in the embodiments of the present application can also be applied to other scenarios where there is a need for evidence collection. The embodiments of the present application do not impose any limitations on this.

[0075] The sound pickup device in the embodiments of the present application can be a sound pickup, a microphone, or other devices with the function of collecting sound signals; the image acquisition device refers to a device used to capture dynamic continuous images, such as a camera, an ordinary camera, a smartphone camera, etc.; the video recording device refers to a device used to record and store images and sounds, such as a digital video recorder, a network video recorder, etc.

[0076] It is understandable that in the embodiment of the present application, the number of image acquisition devices that establish a communication connection with the video recording device may be one or more, and the number of sound pickup devices that establish a communication connection with the video recording device may be one or more, and the present application does not limit this. For example, a video recording device is provided in a government hall, and sound pickup devices are provided at positions A, B, and C in the hall, and cameras are provided at positions D and E, respectively. The three sound pickup devices and the two cameras are respectively connected to the video recording device. The specific process of establishing a communication connection between the sound pickup device and the video recording device in the embodiment of the present application is described below and will not be repeated here.

[0077] Based on the above, a binding relationship can be established between the sound pickup device and the image acquisition device in the embodiment of the present application. The binding relationship can be pre-stored in the storage module of the recording device, or carried in the operation instruction input by the user. Since the binding relationship between the image acquisition device and the sound pickup device is pre-stored in the storage module of the recording device, when there is a demand for displaying the video screen of a certain image acquisition device, that is, when a display instruction is received, there is no need to parse the display instruction to determine the binding relationship. Therefore, this binding relationship is referred to as a static binding relationship below; and the binding relationship is carried in the display instruction, and the binding relationship can change with different display instructions. Therefore, this binding relationship is referred to as a dynamic binding relationship below.

[0078] Exemplarily, when the recording device establishes a one-to-one static binding relationship between the sound pickup device and the image acquisition device, the binding relationship between the sound pickup device and the image acquisition device can be stored in the storage module of the recording device, and the recording device can obtain the binding relationship between the sound pickup device and the image acquisition device from the storage module; in another example, the user does not statically bind the sound pickup device and the recording device, but needs to export the audio data of a certain sound pickup device and the video data of a certain image acquisition device when receiving a display instruction. In this case, the binding relationship between the sound pickup device and the image acquisition device can be included in the display instruction, and the display instruction can be sent to the recording device, and the recording device obtains the binding relationship between the sound pickup device and the image acquisition device from the instruction.

[0079] Based on the above, it can be known that in the embodiment of the present application, the sound pickup device and the image acquisition device can be statically bound one-to-one, or one-to-one static binding can be not established. Instead, when the user has a display requirement, the binding relationship between the sound pickup device and the image acquisition device is included in the corresponding instruction, where the display requirement can be any requirement such as preview, playback, export, etc. It can be understood that in the embodiment of the present application, the sound pickup device that establishes a communication connection with the video recording device and the image acquisition device that establishes a communication connection with the video recording device do not necessarily establish a one-to-one binding relationship. For example, Figure 1bIn the example shown, the sound pickup device 20-1 may establish a binding relationship with the image acquisition device 30-1, the sound pickup device 20-2 may establish a binding relationship with the image acquisition device 30-2, and the sound pickup device 20-3 may not establish a binding relationship with any image acquisition device.

[0080] The recording device can receive the original audio and video data from each image acquisition device that has established a communication connection with the recording device. However, since the image acquisition device that has established a communication connection with the recording device does not necessarily establish a binding relationship with the sound pickup device, the recording device is used to parse the original audio and video data from each target image acquisition device involved in the binding relationship, obtain the target video data and the first audio data, and then obtain the second audio data from the device bound to the target image acquisition device according to the binding relationship of the target image acquisition device as the target audio data, and synchronously display the target audio data and the target video data. For example, Figure 1b As shown, it is assumed that among the image acquisition devices and sound pickup devices that establish communication connections with the video recording device 10, only the sound pickup device 20-1 establishes a binding relationship with the image acquisition device 30-1. After the video recording device 10 receives the original audio and video data from the image acquisition device 30-1 and the image acquisition device 30-2, since the image acquisition device 30-1 establishes a binding relationship with the sound pickup device 20-1, the video recording device only parses the original audio and video data from the image acquisition device 30-1, obtains the video data corresponding to the image acquisition device 30-1, and then obtains the second audio data from the sound pickup device 20-1, and synchronously displays the video data corresponding to the image acquisition device 30-1 and the second audio data of the sound pickup device 20-1 with the same timestamp.

[0081] It is understandable that if the second audio data from the device bound to the target image acquisition device is not clear enough, the source of the audio data can be switched. Therefore, the video recording device is also used to respond to the switching instruction, no longer receive the second audio data from the device bound to the target image acquisition device, and at the same time switch the device bound to the image acquisition device to the device indicated by the switching instruction, and obtain the third audio data of the device as the new target audio data, and synchronously display the target audio data and target video data with the same timestamp. Exemplarily, the device bound to the target image acquisition device 1 is the sound pickup device 1, and the video recording device receives and synchronously outputs the video data of the target image acquisition device 1 and the audio data of the sound pickup device 1 according to the timestamp; if the switching instruction is received, the device bound to the target image acquisition device 1 needs to be switched to the sound pickup device 2, then the video recording device no longer receives the audio data collected by the sound pickup device 1, but obtains the audio data of the sound pickup device 2, and synchronously displays the target video data and the audio data from the sound pickup device 2 according to the timestamp.

[0082] Furthermore, if the target video data and the audio data from the sound pickup device 2 can be used as valid evidence, the target video data and the audio data from the sound pickup device 2 can be synchronously output to obtain the target audio and video data.

[0083] Based on this, the video recording device is also used to synchronously output the target video data and the new target audio data in response to the confirmation instructions of the target video data and the new target audio data.

[0084] In order to more clearly illustrate the video recording device provided in the embodiment of the present application, the video recording device provided in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0085] First, the one-to-one static binding between the sound pickup device and the image acquisition device is described in detail:

[0086] It is understandable that in actual application, before using a recording device to record audio and video images, the user can select the device bound to the target image acquisition device based on the clarity of the audio data, and establish a binding relationship between the target image acquisition device and the audio source device, so that the recording device can continuously display the video data in the original audio and video data captured by the image acquisition device and the first audio data of the device that has established a binding relationship with the image acquisition device in synchronization with the timestamp.

[0087] Therefore, in a possible implementation, the video recording device is further configured to, in response to a binding instruction input by a user, cache the binding relationship between the image acquisition device and the sound pickup device indicated by the binding instruction.

[0088] The binding relationship between the image acquisition device and the sound pickup device can be represented by the channel number of the image acquisition device and the channel number of the sound pickup device, or by the model and channel number of the image acquisition device and the model and channel number of the sound pickup device, or by pre-assigning a unique identifier to each image acquisition device and each sound pickup device, and representing it by the identifier of the image acquisition device and the identifier of the sound pickup device. All of these are possible, and the embodiments of the present application are not limited to this.

[0089] In this embodiment, the original audio and video data and the first audio data can be transmitted in the form of code streams or in other forms other than code streams. Taking the code stream as an example, the video recording device parses the original audio and video data collected by the target image acquisition device to obtain the video stream corresponding to the target image acquisition device, and then writes the video stream corresponding to the target image acquisition device and the second audio stream of the device bound to the target image acquisition device into the same data buffer, and encapsulates the video stream and audio stream in the data buffer to obtain the target audio and video data and output the target audio and video data.

[0090] It is understandable that the establishment of a static binding relationship between the sound pickup device and the image acquisition device can be performed in the GUI interface of the video recording device or on the Web end. The Web end can be the Web interface of the video recording device, or the platform corresponding to the video recording device of the embodiment of the present application, or the client of the application software used to control the video recording device of the embodiment of the present application. For the convenience of description, the GUI interface of the video recording device is referred to as the GUI end below. Different sound pickup devices or different image acquisition devices are connected to the video recording device through different channels, so different channel numbers can be used to describe the sound pickup device or the image acquisition device. For example, if a camera is connected to the video recording device through the IPC 02 channel, IPC02 can be used to describe the camera, and if a sound pickup device is connected to the video recording device through the MIC1 channel, MIC1 can be used to describe the sound pickup device.

[0091] For example, Figure 2a-1~Figure 2a-3 As shown, Figure 2a-1~Figure 2a-3 This is an example diagram of the GUI establishing a binding relationship between the sound pickup device and the image acquisition device. Figure 2a-1 In the audio device management interface, the channel number, name, address (i.e. IP address), protocol type, port number, security, channel of the associated image acquisition device (i.e. associated camera channel), model and status of the audio device connected to the recording device will be displayed in the audio device management interface. Users can delete the connected audio devices in this interface, or add new audio devices. Users can also edit the audio devices connected to the recording device. After clicking the "Add" button, the configuration interface pops up in the current interface, such as Figure 2a-2 As shown, the user can enter the protocol type, IP address, port number, user name, user name and password of the sound pickup device to be added in the interface, and select the transmission protocol between the sound pickup device and the video recording device to connect the sound pickup device to the video recording device. In addition, the user can select the channel of the image acquisition device to be bound in the interface, and whether to enable audio timing to calibrate the sound pickup device. For example, Figure 2a-2 Select the channel of the camera to be bound as "[D2] Camera 1". After the configuration is completed, click the "Add" button to complete the binding.

[0092] The interface after binding is completed is as follows Figure 2a-3 As shown, the interface can display the channel number, IP address, protocol type, port number, password security, channel of the associated image acquisition device, and model of the sound pickup device of all the sound pickup devices that have been connected to the video recording device. In the GUI interface, the user can also click "Export" to export the channel number of the sound pickup device that has established a communication connection with the video recording device, the port connected to the video recording device, the model of the sound pickup device, the channel number of the bound image acquisition device, and other information into a table.

[0093] It is understandable that when the user establishes a binding relationship between the sound pickup device and the image acquisition device on the Web, the video recording device is connected to the remote control device, which can be a computer, a smart phone, or other electronic devices with remote control capabilities.

[0094] Figure 2b-1~Figure 2b-3 This is an example diagram of how the Web client establishes a binding relationship between the sound pickup device and the image acquisition device. Figure 2b-1 In the Web interface, the channel number, IP address, protocol type, port number, password security, channel of the associated image acquisition device, and model of the pickup device connected to the recording device will be displayed. Users can delete the connected pickup device in the Web interface, or add or quickly add a new pickup device. After clicking the "Add" button, the configuration interface pops up in the current interface, such as Figure 2b-2 As shown, the user can enter the IP address, port number, user name, user name password, password confirmation of the sound pickup device to be added in the configuration interface, and select the transmission protocol between the sound pickup device and the video recording device and the protocol type of the sound pickup device to connect the sound pickup device to the video recording device. In addition, the user can select the channel of the image acquisition device to be bound in the configuration interface, and whether to turn on the audio time synchronization to calibrate the sound pickup device. For example, Figure 2b-2 Select the channel of the camera to be bound as "[D2] Camera 1". After the configuration is completed, click the "Add" button to complete the binding.

[0095] The interface after binding is completed is as follows Figure 2b-3 As shown, the Web interface can display information such as the channel number of the sound pickup device that establishes a communication connection with the video recording device, the device type connected to the video recording device, the status of the sound pickup device, the port connected to the video recording device, and the channel number of the bound camera. The user can also synchronously preview the audio data of the sound pickup device and the video data of the image acquisition device that have established a binding relationship in the Web interface.

[0096] The above is an explanation of the establishment of a static binding relationship between the sound pickup device and the image acquisition device and the data synchronization output in the embodiment of the present application from the perspective of the user operation interface. The following is an explanation of the establishment of a static binding relationship between the sound pickup device and the image acquisition device and the data synchronization output in the embodiment of the present application from the perspective of the module interaction of the video recording device. Figure 3As shown, the recording device includes a DSP module (i.e., a digital signal processing module) and an application layer. The application layer includes an ISAPI module (i.e., an application programming interface module), a GUI module (i.e., a user graphical interface module), an IOT module (i.e., an Internet of Things module) and an RTSP module (i.e., a real-time streaming protocol module). When the user establishes a binding relationship (i.e., an association relationship) between the sound pickup device and the image acquisition device on the GUI side (or the Web side), the user first enters the channel number of the image acquisition device to be bound to the sound pickup device on the GUI side (or the Web side). The GUI module (or ISAPI module) of the recording device sends the associated channel to the IOT module. The IOT module saves the association relationship and sends the association relationship to the PAL layer (i.e., the peripheral abstraction layer) to configure the association relationship and perform channel association management. It can be seen that the binding relationship is actually configured by the application layer of the recording device. Combined with Figure 3 and Figure 4 The sound pickup device sends the audio data to the download buffer area (i.e. Figure 4 The image acquisition device (network camera) sends the audio and video data to the download buffer area (i.e., the download buffer area) in the recording device for caching audio and video data via the RTSP module of the recording device. Figure 4 The network camera download buffer area), the DSP module of the recording device (i.e. Figure 4 The track combining processing unit in the download buffer is first parsed in the parsing library, and in the process of decoding the original audio and video data to encode, the binding relationship of the image acquisition device involved in generating the audio and video data is identified in the recording device (i.e. Figure 4 ), discard the audio data in the original audio and video data, retain the video data, then obtain the audio data of the sound pickup device bound to the image acquisition device from the buffer of the sound pickup device, and perform RTP encapsulation on the video data of the original audio and video data in the transcapsulation library, and perform RTP encapsulation on the audio data of the sound pickup device, merge the RTP-encapsulated video data and audio data in the associated channel management to obtain the combined audio and video data, and encapsulate the combined audio and video data in the transcapsulation library to obtain the combined audio and video data (i.e. Figure 4 The encoded audio and video data in the

[0097] The DSP module can store the encoded and packaged bitstream (i.e., the target audio and video data) in the video cache for storage in the hard disk or network disk, in the network transmission cache for remote preview, and in the local GUI decoding buffer (i.e., the local user graphical interface decoding cache) for GUI decoding display, so that subsequent users can play back, export / download, and perform other operations on the target audio and video.

[0098] Through the recording device provided in the embodiment of the present application, by establishing a one-to-one static binding between the sound pickup device and the image acquisition device, since the audio data in the target audio and video in the recording device is acquired by the target image acquisition device and the device bound to the target image acquisition device, and since the device bound to the target image acquisition device is selected by the user based on the clarity of the audio data, the recording device can obtain and output the target audio and video data with clear audio, without the need to collect evidence from the image acquisition device and the sound pickup device separately, thereby simplifying the evidence collection process.

[0099] It is understandable that the user can also collect evidence without establishing a static binding between the sound pickup device and the image acquisition device. For example, evidence can be collected when previewing the target audio and video data, or when replaying the audio and video data of a certain period of time. For ease of description, the process of synchronously outputting the video data from the target image acquisition device and the audio data from the device bound to the target image acquisition device by the recording device provided in the embodiment of the present application without establishing a static binding between the sound pickup device and the image acquisition device is referred to as dynamic binding, and the preview, playback, and download in this case are referred to as preview under dynamic binding, playback under dynamic binding, and export under dynamic binding.

[0100] It is understandable that, similar to the process of establishing the static binding between the sound pickup device and the image acquisition device in the previous embodiment, the dynamic binding in the embodiment of the present application can also be performed on the GUI side of the video recording device or on the Web side. When the dynamic binding is established on the Web side, the video recording device also establishes a communication connection with the remote control device.

[0101] The following will explain in detail the preview, playback and export under dynamic binding established on the GUI and Web sides respectively:

[0102] 1. Preview under dynamic binding

[0103] 1. Preview under dynamic binding established on the GUI side

[0104] When the user clicks the preview function button in the GUI interface, the following is displayed: Figure 5a In the interface shown, the user can select the image acquisition device for preview in the channel, for example, select the video data collected by camera 1 or the video data collected by camera 1 for preview, and click the microphone-style icon to pop up a selection box such as Figure 5b As described above, the user can select in the audio option in the selection box whether the audio in the previewed audio or video is the audio of the image acquisition device itself (ie, the camera 1 ) or the audio of the sound pickup device 1 .

[0105] In this case, for the recording device, it actually receives the preview instruction including the binding relationship sent by the user, and obtains the first audio data from the buffer corresponding to the sound pickup device according to the binding relationship in the preview instruction, obtains the original audio and video data from the buffer corresponding to the image acquisition device, and then parses the original audio and video data to obtain the video data corresponding to the target image acquisition device indicated by the preview instruction, and then synchronously plays the first audio data corresponding to the sound pickup device indicated by the preview instruction and the video data corresponding to the target image acquisition device indicated by the preview instruction for the user to preview.

[0106] It can be understood that the target image acquisition device and sound pickup device indicated by the preview instruction are pre-established in communication with the video recording device. When the sound pickup device and the image acquisition device are connected to the video recording device, the video recording device is further configured to use the buffer for storing the audio and video data from the image acquisition device as the buffer corresponding to the image acquisition device and record it in the data buffer information library; and use the buffer for storing the first audio data from the sound pickup device as the buffer corresponding to the sound pickup device and record it in the data buffer information library.

[0107] Among them, the audio and video data from different image acquisition devices are stored in different buffers, and the first audio data from different sound pickup devices are stored in different buffers. Exemplarily, assuming that the video recording device establishes communication connections with image acquisition device 1, image acquisition device 2, sound pickup device 1, and sound pickup device 2 respectively, the original audio and video data from image acquisition device 1 is cached in buffer 1, the original audio and video data from image acquisition device 2 is cached in buffer 2, the first audio data from sound pickup device 1 is cached in buffer 3, and the first audio data from sound pickup device 2 is cached in buffer 4.

[0108] When receiving the preview instruction, the video recording device is further used to, in response to the preview instruction, search in the data buffer information library of the video recording device for a buffer corresponding to the image acquisition device indicated by the preview instruction as a first data buffer, and search in the data buffer information library of the video recording device for a buffer corresponding to the sound pickup device indicated by the preview instruction as a second data buffer; read the original audio and video data corresponding to the image acquisition device indicated by the preview instruction from the first data buffer, and read the first audio data corresponding to the sound pickup device indicated by the preview instruction from the second data buffer;

[0109] The recording device then parses the original audio and video data corresponding to the image acquisition device indicated by the preview instruction to obtain the video data corresponding to the image acquisition device indicated by the preview instruction; the video data corresponding to the image acquisition device indicated by the preview instruction and the first audio data corresponding to the sound pickup device indicated by the preview instruction are played synchronously to complete the preview.

[0110] Through the recording device of the embodiment of the present application, the user can preview the first audio data corresponding to any sound pickup device and the video data corresponding to any image acquisition device on the GUI end, so as to determine the binding relationship between the audio pickup device and the image acquisition device according to the preview effect, and then obtain an audio and video picture with clear audio.

[0111] For a more detailed description of the preview under dynamic binding on the GUI side, see Figure 6 , Figure 6 The first structural example diagram of the video recording device provided in the embodiment of the present application, the video recording device includes a codec module 61, a digital signal processing module 62, a storage module 63, a decoding library 64, and a data buffer module 65; the storage module 63 is used to temporarily store the instruction parameters of the preview instruction; the data buffer module 65 is used to store the audio and video data from the image acquisition device and the first audio data from the sound pickup device;

[0112] The preview process under dynamic binding on the GUI side is as follows Figure 7 and Figure 8a As shown, the user selects a binding relationship through a GUI interface, and sends a preview instruction including the binding relationship to the video recording device. The codec module of the video recording device responds to the preview instruction, starts the preview, and sends the preview instruction to the digital signal processing module (i.e., the codec module starts the preview, forwards the binding relationship, and sends the instruction parameters of the preview instruction); the digital signal processing module responds to the preview instruction, temporarily stores the preview instruction in the storage module (i.e., the digital signal processing module receives the preview instruction, and stores the binding relationship in the binding information), and the processing integration relationship unit of the digital signal processing module (i.e., the digital signal integration processing unit) searches the data buffer module (i.e., the code stream buffer information library) for the buffer corresponding to the image acquisition device indicated by the preview instruction (i.e., searches for the code stream buffer), as the first data buffer (i.e., The method comprises the following steps: first, reading the original audio and video data corresponding to the image acquisition device indicated by the preview instruction from the first data buffer (that is, performing stream processing, reading the video stream data of the network camera from the network camera stream decoding buffer), and parsing the original audio and video data to obtain the parsed video data and video frame information list; reading the first audio data corresponding to the sound pickup device indicated by the preview instruction from the second data buffer, and parsing it to obtain the parsed first audio data and audio frame information list; wherein the frame information includes the frame type, the frame data address, the frame data length and the global time of the frame.

[0113] Then the digital signal processing module reads the video data that matches the frame type and frame data length in the video frame information, and reads the parsed first audio data that matches the frame type and frame data length in the audio frame information (i.e., reads the bitstream data); sends the video data and the parsed first audio data to the decoding library (i.e., sends the bitstream to the decoding library); the decoding library decodes and plays the video data and the parsed first audio data.

[0114] It can be understood that the decoding library and the DSP module in the embodiment of the present application can be two independent components, or the decoding library can be integrated into the DSP module. Similarly, the decoding library and the DSP module in the playback and export schemes under dynamic binding established on the GUI end below can be two independent components, or the decoding library can be integrated into the DSP module. This application does not limit this.

[0115] In a possible embodiment, the decoding library may be integrated into the DSP module, thereby reducing the delay caused by data transmission and improving the processing speed of the code stream by the DSP module.

[0116] When the user has a need to switch the audio source, that is, when the video recording device receives a switching instruction, if it needs to switch to the image acquisition device itself, the video recording device no longer reads the audio data corresponding to the device bound to the target image acquisition device from the second data buffer and no longer parses the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction, but plays the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction;

[0117] If it is necessary to switch to the second sound pickup device, the recording device will no longer read the audio data corresponding to the device bound to the target image acquisition device from the second data buffer, nor will it synchronously play the video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the device bound to the target image acquisition device in the order of timestamps. Instead, the recording device searches the data buffer information library of the recording device for the third data buffer corresponding to the second sound pickup device, and reads the third audio data corresponding to the second sound pickup device from the third data buffer; the video data corresponding to the target image acquisition device indicated by the preview instruction and the third audio data are synchronously played in the order of timestamps.

[0118] like Figure 8bThe flowchart shown is a flowchart of switching the audio source to the camera itself during preview under dynamic binding on the GUI side. Assuming that the current recording device reads video data from camera A and audio data from sound pickup device B, when switching the audio source to camera A itself, the user switches the binding relationship through the GUI interface, and sends a switching instruction including a new binding relationship to the recording device. The codec module of the recording device sends the switching instruction to the digital signal processing module (that is, the codec module forwards the binding relationship and sends the instruction parameters of the preview instruction); the digital signal processing module receives the modification, stores the binding relationship, and sends a notification of the binding relationship change to the processing and combining relationship unit of the digital signal processing module (that is, the digital signal combining processing unit). The digital signal combining processing unit re-reads the binding relationship and searches for the code stream buffer in the code stream buffer information library to obtain the code stream buffer. In addition, the digital signal combining processing unit stops reading the code stream of the sound pickup device B, but reads the audio and video code stream of camera A from the code stream parsing buffer of camera A, and sends the code stream to the decoding library, and the decoding library decodes and plays the audio and video code stream data.

[0119] like Figure 8c The flowchart shown is a flowchart of switching the audio source to other pickup devices during preview under dynamic binding on the GUI side. Assuming that the current recording device reads video data from camera A and audio data from pickup device B, when the audio source is switched to pickup device C, the user switches the binding relationship through the GUI interface, and sends a switching instruction including a new binding relationship to the recording device. The codec module of the recording device sends the switching instruction to the DSP module (i.e., the codec module forwards the binding relationship and sends the instruction parameters of the preview instruction); the DSP module receives the modification, stores the binding relationship, and sends a notification of the binding relationship change to the processing and merging relationship unit of the DSP module (i.e., the DSP processing and merging relationship unit). The DSP processing and merging relationship unit re-reads the binding relationship, and searches for the code stream buffer area in the code stream buffer area information library to obtain the code stream buffer area. In addition, the DSP processing unit stops reading the code stream of the sound pickup device B, but reads the video code stream of camera A from the code stream parsing buffer of camera A, and sends the code stream to the decoding library, reads the audio code stream of the sound pickup device C from the code stream parsing buffer of the sound pickup device C, and sends the code stream to the decoding library, and the decoding library decodes and plays the audio code stream data and the video code stream data.

[0120] Through the embodiments of the present application, the source of audio data can be replaced according to actual needs, so that the video recording device can directly output the target audio and video data with clear audio, thereby simplifying the evidence collection process.

[0121] 2. Preview under dynamic binding established on the Web side

[0122] The user previews the dynamic binding on the Web end as follows Fig. 9 As shown, after the user clicks the preview function button in the Web interface, he can click "Video Source" to choose to preview the video data captured by camera 1 or the video data captured by camera 1, and click "Audio Source" to select the audio channel in the previewed audio and video as the image acquisition device itself (i.e., camera 1) or the sound pickup device 1.

[0123] In this case, if Fig.10 As shown, the remote control device actually receives the preview instruction issued by the user and sends the preview instruction to the recording device. In response to the preview instruction, the recording device obtains the original audio and video data of the image acquisition device indicated by the preview instruction and the first audio data corresponding to the device bound to the target image acquisition device, and sends the original audio and video data and the first audio data to the remote control device.

[0124] After the remote control device receives the original audio and video data corresponding to the image acquisition device indicated by the preview instruction and the audio data corresponding to the device bound to the target image acquisition device, it first parses the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction to obtain the video data corresponding to the target image acquisition device indicated by the preview instruction, and then synchronously plays the video data corresponding to the target image acquisition device indicated by the preview instruction and the first audio data corresponding to the device bound to the target image acquisition device in the order of timestamps.

[0125] In a specific embodiment, Fig.11a As shown, the remote control device includes a playback library and a playback library track processing unit. The user selects the binding relationship through the Web interface, that is, sends the instruction parameters of the preview instruction to the playback library (that is, the parameters are sent), and the playback library stores the binding relationship. Then, the playback library track processing unit first reads the binding relationship from the binding information, and then performs stream processing on the recording device through the protocol, and reads the code stream data from the recording device, that is, pulls the audio and video code stream corresponding to the image acquisition device indicated by the preview instruction and the audio code stream corresponding to the sound pickup device indicated by the preview instruction from the recording device. Then, the audio and video code stream is processed, the video data is read according to the frame type, the audio data is removed, and the video code stream data and the audio code stream data are sent to the playback library, which is decoded and played by the playback library.

[0126] It can be seen that the preview under dynamic binding on the Web side is similar to the preview under dynamic binding on the GUI side. The difference is that during the preview under dynamic binding on the Web side, the playback library in the preset terminal or the playback library on the server temporarily stores the binding relationship and decodes and plays the video data and audio data, rather than the DSP module. In addition, the played audio data and video data are obtained from the recording device through the protocol, rather than being read from its own decoding area.

[0127] Through the video recording device of the embodiment of the present application, the user can remotely preview the first audio data corresponding to any sound pickup device and the video data corresponding to any image acquisition device, so as to determine the binding relationship between the audio pickup device and the image acquisition device according to the preview effect, and then remotely obtain audio and video images with clear audio.

[0128] When the user has a need to switch the audio source, that is, when the remote control device receives the switching instruction, if it needs to switch to the image acquisition device itself, then the video recording device, under the control of the remote control device, no longer parses the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction, and no longer sends the audio data corresponding to the device bound to the target image acquisition device to the remote control device, but sends the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction to the remote control device, so that the remote control device plays the original audio and video data synchronously in the order of timestamps;

[0129] If it is necessary to switch to the second sound pickup device, the video recording device, under the control of the remote control device, will no longer send the audio data corresponding to the device bound to the target image acquisition device to the remote control device, but will send the audio data corresponding to the second sound pickup device to the remote control device, so that the remote control device will synchronously play the video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the second sound pickup device in the order of timestamps.

[0130] like Fig.11b The figure shows a flowchart of switching the audio source to the camera itself during preview under dynamic binding on the Web side. Assuming that the current recording device reads video data from camera A and audio data from sound pickup device B, when the audio source is switched to camera A itself, the user switches the binding relationship through the Web interface, sends the parameters of the new binding relationship to the playback library, the playback library modifies and stores the binding relationship, and sends a notification of the binding relationship change. Then the playback library track processing unit first re-reads the binding relationship from the binding information, and closes the stream processing of the audio stream of the sound pickup device B from the recording device, and then continues to pull the original audio and video data corresponding to camera A from the recording device, and finally processes the bitstream of the original audio and video data, and sends the audio and video data to the playback library, which is decoded and played by the playback library.

[0131] like Fig.11cThe figure shows a flowchart of switching the audio source to other pickup devices during preview under dynamic binding on the Web side. Assuming that the current recording device reads video data from camera A and audio data from pickup device B, when the audio source is switched to pickup device C, the user switches the binding relationship through the Web interface, sends the parameters of the new binding relationship to the playback library, the playback library modifies and stores the binding relationship, and sends a notification of the binding relationship change. Then, the playback library track processing unit first reads the binding relationship from the binding information, pulls the audio and video code stream data of camera A and the audio code stream of pickup device C from the recording device respectively, and turns off the stream processing of the audio stream of pickup device B from the recording device. Then, the video data in the audio and video data is read according to the frame type, the audio data in the audio and video data is removed, and the video code stream data and the audio code stream data of pickup device C are sent to the playback library, which is decoded and played by the playback library.

[0132] Through the embodiments of the present application, the user can change the source of audio data according to actual needs during remote preview, so that the video recording device can directly output the target audio and video data with clear audio, thereby simplifying the evidence collection process.

[0133] 2. Playback under dynamic binding

[0134] 1. Playback under dynamic binding established on the GUI side

[0135] like Fig.12 As shown, after the user clicks the playback function button in the GUI interface, he can select the image acquisition device to be played back in the channel, such as camera 1, camera 1 or camera 3, etc., and can also select the time when the video data to be played back was generated, such as playing back the video data generated on April 24, 2024. He can also select the audio in the target audio and video to be played back in the audio options in the interface, whether it is the audio of camera 1, the audio of camera 1, or the audio of sound pickup device 1, etc.

[0136] In this case, for the video recording device, it is actually the video recording device that receives the playback instruction including the binding relationship and the time range issued by the user, and reads the original audio and video data of the image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction from the first data buffer according to the binding relationship in the playback instruction, reads the first audio data of the sound pickup device indicated by the playback instruction within the time range from the second data buffer, and stores the read original audio and video data in the first decoding buffer, and stores the read first audio data in the second decoding buffer. Among them, the first data buffer is the buffer corresponding to the image acquisition device indicated by the playback instruction, and the second data buffer is the buffer corresponding to the sound pickup device indicated by the playback instruction.

[0137] Then, the original audio and video data is read from the first decoding buffer, and the original audio and video data is parsed, and the parsed audio and video data is stored in the first code stream parsing buffer, and the first audio data in the previous playback is read from the second decoding buffer, and the first audio and video data is parsed, and the parsed first audio data is stored in the second code stream parsing buffer; finally, the video data of the image acquisition device indicated by the playback instruction within the time range is read from the first code stream parsing buffer, and the first audio data of the sound pickup device indicated by the playback instruction within the time range is read from the second code stream parsing buffer; the video data corresponding to the image acquisition device indicated by the playback instruction within the time range and the first audio data corresponding to the sound pickup device indicated by the playback instruction are played synchronously for the user to watch and playback.

[0138] It can be understood that, similar to the solution of the GUI preview above, the image acquisition device and the sound pickup device indicated by the playback instruction also establish a communication connection with the video recording device in advance. When the sound pickup device and the image acquisition device are connected to the video recording device, the video recording device uses the buffer for storing the audio and video data from the image acquisition device as the buffer corresponding to the image acquisition device, and records it in the data buffer information library; uses the buffer for storing the first audio data from the sound pickup device as the buffer corresponding to the sound pickup device, and records it in the data buffer information library. Among them, the audio and video data from different image acquisition devices are stored in different buffers, and the first audio data from different sound pickup devices are stored in different buffers.

[0139] To ensure smoothness and stability of playback, when the video recording device receives a playback instruction, it responds to the playback instruction and allocates a first decoding buffer for storing the code stream to the image acquisition device indicated by the playback instruction, and allocates a second decoding buffer for storing the code stream to the sound pickup device.

[0140] Through the recording device of the embodiment of the present application, the user can play back the first audio data corresponding to any sound pickup device and the video data corresponding to any image acquisition device on the GUI end to obtain an audio and video picture with clear audio, reducing the possibility of unclear audio in the played back video.

[0141] For a more detailed description of playback under dynamic binding on the GUI side, see Fig.13 , Fig.13The second structural example diagram of the video recording device provided in the embodiment of the present application, the video recording device includes a playback module 131, a digital signal processing module 132, a data buffer module 133, a decoding library 134, a parsing buffer module 135, and a data buffer information library 136; the parsing buffer module 135 is used to temporarily store the audio and video data of the image acquisition device indicated by the playback instruction and the first audio data of the sound pickup device indicated by the playback instruction; the data buffer information library 136 is used to record the buffer for storing the audio and video data of each image acquisition device and the buffer for storing the first audio data of each sound pickup device.

[0142] The playback process under dynamic binding on the GUI side is as follows Fig.14 and Fig.15a As shown, the user selects the binding relationship and the playback time through the GUI interface, and sends a playback instruction including the binding relationship and the time range to the video recording device. The playback module of the video recording device responds to the playback instruction and sends the playback instruction to the DSP module (i.e., the digital signal processing module); the DSP module responds to the playback instruction and applies for a first decoding buffer for storing the audio and video data of the image acquisition device indicated by the playback instruction and a second decoding buffer for storing the first audio data of the sound pickup device indicated by the playback instruction in the data buffer module (i.e., Fig.15a Then, the buffer corresponding to the image acquisition device and the buffer corresponding to the sound pickup device indicated by the playback instruction are searched in the data buffer information library of the recording device (i.e. Fig.15a The video information of the network camera and the video information of the sound pickup device are retrieved from the database, and the audio and video data and the first audio data (i.e. Fig.15a The code stream data in the hard disk is read from the decoder), and stored in the first decoding buffer and the second decoding buffer respectively.

[0143] After the playback module sends the first track-merging instruction to the DSP module, the DSP module responds to the first track-merging instruction and reads the original audio and video data (i.e. Fig.15a The digital signal processing module reads the audio and video code stream data of the network camera, and reads the first audio data (i.e. Fig.15a The digital signal processing module reads the audio code stream data of the sound pickup device), and sends the original audio and video data and the first audio data to the decoding library (ie, the parsing library).

[0144] The decoding library parses the original audio and video data (i.e., the parsing library parses the bitstream) to obtain the parsed video data and video frame information list, and sends and stores the parsed video data and video frame information list in the first parsing buffer of the parsing buffer module (i.e., the bitstream parsing buffer); parses the first audio data to obtain the parsed first audio data and audio frame information list, and sends and stores the parsed first audio data and audio frame information list in the second parsing buffer of the parsing buffer module; wherein the frame information includes the frame type, frame data address, frame data length, and global time of the frame.

[0145] Finally, the DSP module reads the video data that matches the frame data address, frame data length, frame type and global time of the frame in the video frame information list from the first parsing buffer, and reads the first audio data that is the same as the global time of the frame of the video data and sends it to the decoding library (that is, the digital signal processing module reads the bit stream data according to the strategy), and then sends it to the decoding library for decoding and playback.

[0146] During the playback process, if the current audio is not clear enough and there is a need to switch the audio source, that is, when the recording device receives a switching instruction, if it needs to switch to the image acquisition device itself, the recording device no longer reads the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction from the second data buffer, nor does it store the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction into the first bitstream parsing buffer. Instead, the second bitstream buffer is released, and the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction is stored in the decoding buffer for analysis, the original audio and video data is read from the decoding buffer, and the original audio and video data is decoded and played.

[0147] If it is necessary to switch to the second sound pickup device, the video recording device releases the second code stream buffer and allocates a third code stream buffer for storing the code stream to the second sound pickup device; searches for the third data buffer in the data buffer information library of the video recording device; reads the audio data of the second sound pickup device within the time range indicated by the playback instruction from the third data buffer, and stores the audio data of the second sound pickup device within the time range indicated by the playback instruction in the third code stream buffer; reads the audio data of the second sound pickup device within the time range indicated by the playback instruction from the third code stream buffer, and stores it in the third code stream parsing buffer; decodes and synchronously plays the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the second sound pickup device within the time range indicated by the playback instruction.

[0148] like Fig.15bThe flowchart shown is a flowchart of switching the audio source to the camera itself during playback under dynamic binding on the GUI side. Assuming that the current recording device reads video data from camera A and audio data from sound pickup device B, when the audio source is switched to camera A itself, the user switches the binding relationship through the GUI interface, the playback module turns off the playback of sound pickup device B and releases resources, reads the audio and video code stream data of camera A from the hard disk, and stores the audio and video code stream data in the code stream buffer. The digital signal processing module reads the audio and video code stream data of camera A, sends the audio and video data to the parsing library, and the parsing library parses the audio and video data to obtain the parsed video data and video frame information list, and stores the parsed video data and video frame information list in the code stream parsing buffer area; finally, the digital signal processing module reads the video data and audio data that match the frame data address, frame data length, frame type and global time of the frame in the video frame information list from the code stream parsing buffer area and sends them to the decoding library for decoding and playback.

[0149] like Fig.15c The figure shows a flow chart of switching the audio source to other sound pickup devices during playback under dynamic binding on the GUI side. Assuming that the current recording device reads video data from camera A and audio data from sound pickup device B, when switching the audio source to sound pickup device C, the user switches the binding relationship through the GUI interface, and the playback module turns off the playback of sound pickup device B and releases resources. According to the current recording playback time point, the recording information of sound pickup device C is retrieved from the database, and the code stream data of sound pickup device C is read from the hard disk, and the audio and video code stream data of camera A is read from the hard disk, and the audio and video code stream data of camera A and the audio code stream data of sound pickup device C are respectively stored in the code stream buffer. The digital signal processing module reads the audio and video stream data of camera A and the audio stream data of pickup device C from the stream buffer respectively, and sends the audio and video data to the parsing library, which parses the audio and video data to obtain the parsed video data and video frame information list, and stores the parsed video data and video frame information list in the stream parsing buffer; the parsing library parses the audio data to obtain the parsed audio data and audio frame information list, and stores the parsed audio data and audio frame information list in the stream parsing buffer; finally, the digital signal processing module reads the video data and audio data that match the frame data address, frame data length, frame type and global time of the frame in the video frame information list from the stream parsing buffer (that is, the digital signal processing module reads the stream data according to the strategy) and sends them to the decoding library for decoding and playback.

[0150] Through the embodiments of the present application, the source of audio data can be changed according to actual needs during the playback process, so that the video recording device can directly output the target audio and video data with clear audio, thereby simplifying the evidence collection process.

[0151] 2. Playback under dynamic binding established on the Web side

[0152] The user's playback under dynamic binding on the Web is similar to the user's preview under dynamic binding on the Web in the previous article. The user can click the playback function button in the Web interface, choose to play back the video data collected by camera 1 or the video data collected by IPC, and select in the audio options in the interface whether the audio in the target audio and video is the audio of the image acquisition device itself or the audio of the pickup device MIC1.

[0153] In this case, if Fig.16 As shown, the remote control device actually receives the playback command issued by the user and sends the playback command to the video recording device. In response to the playback command, the video recording device obtains the first audio data of the sound pickup device indicated by the playback command within the time range indicated by the playback command and the original audio and video data of the image acquisition device indicated by the playback command within the time range indicated by the playback command, and sends the obtained first audio data and original audio and video data to the remote control device.

[0154] After the remote control device receives the first audio data of the sound pickup device indicated by the playback instruction within the time range indicated by the playback instruction and the original audio and video data of the image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction, it first parses the original audio and video data of the image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction to obtain the video data of the image acquisition device indicated by the playback instruction within the time range, and then plays the video data synchronously with the first audio data of the sound pickup device indicated by the playback instruction within the time range.

[0155] In a specific embodiment, Fig.17a As shown, the remote control device includes a playback library. The user selects a binding relationship through a Web interface, that is, sends instruction parameters (i.e., a binding relationship) of a playback instruction to the playback library. The playback library retrieves the camera recording information from the recording device through a protocol, and pulls the audio and video code stream data from the recording device, and then stores the audio and video code stream data in a code stream buffer; retrieves the recording information of the pickup device in the recording device, and pulls the audio code stream data from the recording device, and stores the audio code stream data in the code stream buffer.

[0156] The playback library reads the original audio and video data of the camera indicated by the playback instruction within the time range indicated by the playback instruction, and the first audio data of the sound pickup device indicated by the playback instruction within the time range indicated by the playback instruction from the code stream buffer (the playback library reads the audio and video code stream data of the camera, and the playback library reads the audio code stream data of the sound pickup device), and sends them to the parsing library to parse the code stream data, and stores the parsed audio and video code stream data in the first parsing buffer (i.e., code stream parsing buffer area) of the recording device, and stores the parsed first audio code stream data in the second parsing buffer (i.e., code stream parsing buffer area). Then the playback library reads the video data that matches the frame data address, frame data length, frame type and global time of the frame in the video frame information list from the code stream parsing buffer area, and reads the first audio data with the same global time as the frame of the video data and decodes and plays it (i.e., the playback library reads the code stream data according to the strategy for decoding and playback).

[0157] It can be seen that the playback under dynamic binding on the Web side is similar to the playback under dynamic binding on the GUI side. The difference is that during the playback process under dynamic binding on the Web side, the video data and audio data are decoded and played by the playback library in the preset terminal or the playback library on the server, rather than by the DSP module or the playback library. In addition, the played audio data and video data are obtained from the recording device through the protocol, rather than being read from its own decoding area.

[0158] Through the recording device of the embodiment of the present application, the user can remotely play back the first audio data corresponding to any sound pickup device and the video data corresponding to any image acquisition device to obtain an audio and video picture with clear audio, thereby reducing the possibility of unclear audio in the played back video.

[0159] When the user has a need to switch the audio source, that is, when the remote control device receives a switching instruction, if it needs to switch to the image acquisition device itself, then the video recording device, under the control of the remote control device, no longer parses the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction, and no longer sends the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction to the remote control device, but sends the original audio and video data corresponding to the target image acquisition device indicated by the playback instruction to the remote control device, so that the remote control device plays the original audio and video data synchronously in the order of timestamps;

[0160] If it is necessary to switch to the second sound pickup device, the video recording device, under the control of the remote control device, no longer sends the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction to the remote control device, but sends the audio data of the second sound pickup device within the time range indicated by the playback instruction to the remote control device, so that the remote control device decodes and plays synchronously the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the second sound pickup device within the time range indicated by the playback instruction.

[0161] Through the embodiments of the present application, the user can change the source of audio data according to actual needs during remote playback, so that the video recording device can directly output the target audio and video data with clear audio, thereby simplifying the evidence collection process.

[0162] like Fig.17b The figure shows a flowchart of switching the audio source to the camera itself during playback under dynamic binding on the GUI side. Assuming that the current recording device reads video data from camera A and audio data from sound pickup device B, when the audio source is switched to camera A itself, the user switches the binding relationship through the GUI interface, the playback library closes the playback of sound pickup device B and releases resources (i.e., the playback library stops pulling the code stream of sound pickup device B), pulls the audio and video code stream data of camera A from the recording device, and stores the audio and video code stream data in the code stream buffer area. The playback library reads the audio and video code stream data of camera A, sends the audio and video data to the parsing library, and the parsing library parses the audio and video data to obtain the parsed video data and video frame information list, and stores the parsed video data and video frame information list in the code stream parsing buffer area; finally, the playback library reads the audio and video code stream data of camera A from the code stream parsing buffer area and sends it to the decoding library for decoding and playback.

[0163] like Fig.17cThe figure shows a flow chart of switching the audio source to other sound pickup devices during playback under dynamic binding on the GUI side. Assuming that the current recording device reads video data from camera A and audio data from sound pickup device B, when switching the audio source to sound pickup device C, the user switches the binding relationship through the GUI interface, the playback library closes the playback of sound pickup device B and releases resources (that is, the playback library stops pulling the code stream of sound pickup device B), and according to the current recording playback time point, retrieves the recording information of sound pickup device C from the recording device, reads the code stream data of sound pickup device C from the recording device, reads the audio and video code stream data of camera A from the recording device, and stores the audio and video code stream data of camera A and the audio code stream data of sound pickup device C in the code stream buffer area respectively. The playback library reads the audio and video stream data of camera A and the audio stream data of pickup device C from the stream buffer respectively, and sends the audio and video data to the parsing library, which parses the audio and video data to obtain the parsed video data and video frame information list, and stores the parsed video data and video frame information list in the stream parsing buffer; the parsing library parses the audio data to obtain the parsed audio data and audio frame information list, and stores the parsed audio data and audio frame information list in the stream parsing buffer; finally, the playback library reads the video data and audio data that match the frame data address, frame data length, frame type and global time of the frame in the video frame information list from the stream parsing buffer (that is, the playback library reads the stream data according to the strategy for decoding and playback) and sends them to the decoding library for decoding and playback.

[0164] It is understandable that after receiving the switching instruction, when the user plays the video data and audio data within the time range indicated by the playback instruction, if the user performs a fast forward or rewind operation, the target audio data and target video data of the device indicated by the switching instruction are always displayed. Exemplarily, the video data from the image acquisition device 1 and the audio data from the sound pickup device 1 from 9:00 to 10:00 on September 29 are played synchronously. After ten minutes of playing, the switching instruction is received. The device indicated by the switching instruction is the sound pickup device 2. At this time, the video recording device plays the video data from the image acquisition device 1 and the audio data from the sound pickup device 2 from 9:10 to 10:00 on September 29 synchronously. During the playback process, if the user wants to rewind to the beginning of the video data and play it again, the video recording device will play the video data from the image acquisition device 1 and the audio data from the sound pickup device 2 from 9:00 to 10:00 on September 29 synchronously.

[0165] 3. Export under dynamic binding

[0166] 1. Export under dynamic binding established on the GUI side

[0167] like Fig.18aAs shown, the user clicks "File Management" in the GUI interface to enter the video export interface. By clicking "Video", the source of the video data to be exported can be selected in the channel, for example Fig.18a In the channel, select the image acquisition device of channel D2 as the source of the exported video data. You can also select the start and end time of the video data to be exported, for example, export the video data within the time period of 12:22:10-12:23:26. The user can also check "Associate Audio Backup" in this interface to determine whether the exported video data should be associated with audio.

[0168] After selecting the audio and video data to be exported and checking the associated audio backup, click the "Backup" button to display the following Fig.18b The interface shows the start and end time of the video data to be exported. The user can select the source of the audio in the target audio and video to be exported in this interface, for example Fig.18b The audio data associated with the image acquisition device is from channel D2, and then click "OK" to bind the corresponding image acquisition device to the sound pickup device. Fig.18c interface.

[0169] exist Fig.18c In the interface shown, the user can choose the export path when exporting the audio and video data of the bound image acquisition device and the audio data of the sound pickup device. The interface can also display the memory space size of the importing device, the name, size, type, modification date and other information of the exported audio and video data. The user can also create a new folder in the importing device when exporting. After the user clicks the "OK" button, the audio and video data selected by the user will be exported to the importing device selected by the user in the format edited by the user, thereby completing the data export.

[0170] In this case, for the video recording device, the video recording device actually receives the export instruction including the binding relationship issued by the user, and reads the original audio and video data of the image acquisition device indicated by the export instruction within the time range indicated by the export instruction from the first data buffer according to the binding relationship in the export instruction, and reads the first audio data of the sound pickup device indicated by the export instruction within the time range indicated by the export instruction from the second data buffer. The first data buffer is the buffer corresponding to the image acquisition device indicated by the export instruction, and the second data buffer is the buffer corresponding to the sound pickup device indicated by the export instruction.

[0171] In order to ensure the efficiency and stability of data export, when the video recording device receives an export instruction, it responds to the export instruction and allocates a first decoding buffer for storing the code stream to the image acquisition device indicated by the export instruction, allocates a second decoding buffer for storing the code stream to the sound pickup device, and allocates a video recording export buffer for storing the audio and video data after the merge.

[0172] Then the recording device stores the original audio and video data of the image acquisition device indicated by the export instruction within the time range indicated by the export instruction into the first decoding buffer, and stores the first audio data of the sound pickup device indicated by the export instruction within the time range indicated by the export instruction into the second decoding buffer. Then, the original audio and video data of the image acquisition device indicated by the export instruction within the time range indicated by the export instruction is read from the first decoding buffer, parsed, and the parsed audio and video data is stored in the first code stream parsing buffer, and the first audio data of the sound pickup device indicated by the export instruction within the time range indicated by the export instruction is read from the second decoding buffer, parsed, and the parsed first audio data is stored in the second code stream parsing buffer; finally, the video data of the image acquisition device indicated by the export instruction within the time range indicated by the export instruction is read from the first code stream parsing buffer, and the first audio data of the sound pickup device indicated by the export instruction within the time range indicated by the export instruction is read from the second code stream parsing buffer; the read video data and the first audio data are merged by sending them to the export buffer to obtain merged audio and video data, and finally the merged audio and video data is sent to the target storage device indicated by the export instruction through the export buffer to realize the export of the target audio and video.

[0173] It can be understood that, similar to the solution of the GUI preview above, the image acquisition device and the sound pickup device indicated by the export instruction also establish a communication connection with the video recording device in advance. When the sound pickup device and the image acquisition device are connected to the video recording device, the video recording device uses the buffer for storing the audio and video data from the image acquisition device as the buffer corresponding to the image acquisition device, and records it in the data buffer information library; uses the buffer for storing the first audio data from the sound pickup device as the buffer corresponding to the sound pickup device, and records it in the data buffer information library. Among them, the audio and video data from different image acquisition devices are stored in different buffers, and the first audio data from different sound pickup devices are stored in different buffers.

[0174] The target storage device can be a device with storage function such as a USB flash drive, a hard disk, etc., and this application does not limit this.

[0175] Through the recording device of the embodiment of the present application, the user can export the first audio data corresponding to any sound pickup device and the video data corresponding to any image acquisition device on the GUI end to obtain an audio and video picture with clear audio, reducing the possibility of unclear audio in the exported video.

[0176] For a more detailed description of the export under dynamic binding on the GUI side, see Fig.19 , Fig.19The third structural example diagram of the video recording device provided in the embodiment of the present application, the video recording device includes a backup module 191, a digital signal processing module 192, a data buffer module 193, a decoding library 194, a parsing buffer module 195, a data buffer information library 196, and a video recording export buffer module 197; the parsing buffer module 195 is used to temporarily store the audio and video data of the image acquisition device indicated by the export instruction and the first audio data of the sound pickup device indicated by the export instruction; the data buffer information library 196 is used to record the buffer for storing the audio and video data of each image acquisition device and the buffer for storing the first audio data of each sound pickup device; the video recording export buffer module is used to merge the video data of the image acquisition device indicated by the export instruction within the time range indicated by the export instruction and the first audio data of the sound pickup device indicated by the export instruction within the time range indicated by the export instruction, and obtain and store the merged audio and video data;

[0177] The export process under dynamic binding on the GUI side is as follows Fig. 20 and Fig.21 As shown,

[0178] The user selects the binding relationship and the export time through the GUI interface, and sends the export instruction including the binding relationship to the video recording device. The backup module of the video recording device responds to the export instruction and sends the export instruction to the DSP module (i.e., the digital signal processing module). The DSP module responds to the export instruction and applies for a first decoding buffer for storing the audio and video data of the image acquisition device indicated by the export instruction and a second decoding buffer for storing the first audio data of the sound pickup device indicated by the export instruction in the data buffer module (i.e., the second decoding buffer for storing the first audio data of the sound pickup device indicated by the export instruction). Fig.21 Then, the buffer corresponding to the image acquisition device and the buffer corresponding to the sound pickup device indicated by the export instruction are searched in the data buffer information library of the recording device (i.e. Fig.21 The video information of the network camera and the video information of the sound pickup device are retrieved from the database, and the audio and video data and the first audio data (i.e. Fig.21 The code stream data in the hard disk is read from the decoder), and stored in the first decoding buffer and the second decoding buffer respectively.

[0179] After the backup module sends the first track-merging instruction to the DSP module, the DSP module responds to the first track-merging instruction and reads the original audio and video data (i.e. Fig.21 The digital signal processing module reads the audio and video code stream data of the network camera, and reads the first audio data (i.e. Fig.21 The digital signal processing module reads the audio code stream data of the sound pickup device), and sends the original audio and video data and the first audio data to the decoding library (ie, the parsing library).

[0180] The decoding library parses the original audio and video data (i.e., the parsing library parses the bitstream) to obtain the parsed video data and video frame information list, and sends and stores the parsed video data and video frame information list in the first parsing buffer of the parsing buffer module (i.e., the bitstream parsing buffer); parses the first audio data to obtain the parsed first audio data and audio frame information list, and sends and stores the parsed first audio data and audio frame information list in the second parsing buffer of the parsing buffer module; wherein the frame information includes the frame type, frame data address, frame data length, and global time of the frame.

[0181] Finally, the DSP module reads the video data that matches the frame data address, frame data length, frame type and global time of the frame in the video frame information list from the first parsing buffer, and reads the first audio data that is the same as the global time of the frame of the video data (that is, the DSP module reads the code stream data according to the strategy), and sends the read video data and the first audio data to the video export buffer for merging, and obtains the merged audio and video data. Finally, the backup module reads the merged audio and video data from the video export buffer and transpacks it to obtain the video file, and sends the video file to the target storage device.

[0182] 2. Export under dynamic binding established on the Web side

[0183] like Fig. 22 As shown, the user can select the image acquisition device to which the video data needs to be exported (remote export is referred to as downloading in the following text) in the channel of the download interface of the Web terminal, such as selecting camera 1, and can also select the start and end time of the downloaded video data, and select the source of the audio data associated with the downloaded video data, or choose not to associate and directly use the audio data of the image acquisition device itself. You can also select the type of file to be downloaded in "File Type", for example, "All" means downloading audio and video data, "Video" means downloading only video data, and "Audio" means downloading only audio data.

[0184] In this case, if Fig.23 As shown, the remote control device actually receives the export instruction issued by the user and sends the export instruction to the video recording device. In response to the export instruction, the video recording device obtains the original audio and video data of the image acquisition device indicated by the export instruction within the time range indicated by the export instruction and the first audio data of the sound pickup device indicated by the export instruction (i.e., the sound pickup device bound to the image acquisition device indicated by the export instruction) within the time range indicated by the export instruction, and sends the read first audio data and the original audio and video data to the remote control device.

[0185] After the remote control device receives the first audio data of the sound pickup device indicated by the export instruction within the time range indicated by the export instruction and the original audio and video data of the image acquisition device indicated by the export instruction within the time range indicated by the export instruction, it first parses the original audio and video data of the image acquisition device indicated by the export instruction within the time range indicated by the export instruction to obtain the video data of the image acquisition device indicated by the export instruction within the time range indicated by the export instruction, and then outputs the parsed video data and the first audio data of the sound pickup device indicated by the export instruction within the time range indicated by the export instruction as a video file, and finally the backup module sends the video file to the target storage device.

[0186] It is understandable that the playback library of the remote control device can first output the video data corresponding to the image acquisition device indicated by the parsed export instruction and the first audio data corresponding to the sound pickup device indicated by the export instruction as a temporary video file, and then delete the temporary video file in the playback library after generating the final video file in the playback library. The video file format can be MP4, AVI, or other formats.

[0187] In a specific embodiment, Fig.24 As shown, the remote control device includes a playback library. The user selects a binding relationship through a Web interface, that is, sends an instruction parameter (i.e., a binding relationship) of an export instruction to the playback library. The playback library retrieves the recording information of the network camera A from the recording device, and pulls the audio and video stream data of the network camera A, and stores it in a first decoding buffer (stream buffer); retrieves the recording information of the sound pickup device from the recording device, and pulls the audio stream data of the sound pickup device B, and stores it in a second decoding buffer (stream buffer).

[0188] The playback library reads the audio and video code stream data of the network camera and the audio code stream data of the sound pickup device from the code stream buffer, and sends the read original audio and video data and the first audio data to the parsing library to parse the code stream data, and outputs the parsed audio and video code stream data as a temporary video file, and outputs the parsed first audio code stream data as a temporary video file. Then the playback library reads the code stream data according to the strategy and sends it to the video export buffer for merging to obtain the merged audio and video data. The playback library reads the merged audio and video data from the video export buffer through the protocol and exports it to the file after transpackaging to obtain the video file.

[0189] It can be seen that the export under dynamic binding on the Web side is similar to the export under dynamic binding on the GUI side, with the following differences: 1. During the export process under dynamic binding on the Web side, the video data and audio data are decoded by the playback library of the preset terminal or the playback library on the server, rather than by the DSP module or the playback library; 2. During the export process under dynamic binding on the Web side, the playback library of the preset terminal obtains the audio data and video data from the recording device through the protocol, rather than reading from its own decoding area; 3. During the export process under dynamic binding on the Web side, the playback library of the preset terminal directly outputs the parsed data as a recording file without caching it in the parsing cache area.

[0190] It can be understood that in the embodiments of the present application, the export under the dynamic binding of the Web end and the export under the dynamic binding of the GUI end can be to directly export the data to be exported as a video file, or to first cache the data to be exported as a temporary file and then export the temporary file. This is all possible and the present application does not limit this.

[0191] Through the video recording device of the embodiment of the present application, the user can remotely download or export the first audio data corresponding to any sound pickup device and the video data corresponding to any image acquisition device to obtain audio and video images with clear audio, reducing the possibility of unclear audio.

[0192] In the second aspect of the embodiment of the present application, Fig.25 As shown, a method for synchronously outputting audio and video is provided, which is applied to a video recording device, and the method includes:

[0193] Step S2501, in response to a display instruction, obtaining original audio and video data from a target image acquisition device indicated by the display instruction; parsing the original audio and video data to obtain target video data and first audio data, and obtaining second audio data from a device bound to the target image acquisition device as target audio data, and synchronously displaying the target video data and the target audio data with the same timestamp;

[0194] Step S2502, in response to the switching instruction, terminate the step of acquiring the second audio data from the device bound to the target image acquisition device, switch the device bound to the target image acquisition device to the device indicated by the switching instruction, acquire the third audio data from the second device bound to the target image acquisition device as the new target audio data; synchronously display the target video data and the target audio data with the same timestamp;

[0195] Step S2503, in response to the confirmation instruction for the target video data and the new target audio data, synchronously output the target video data and the new target audio data to obtain the target audio and video data;

[0196] Among them, the video recording device has pre-established a communication connection with at least one sound pickup device, and has pre-established a communication connection with at least one image acquisition device, the device bound to the target image acquisition device is the target image acquisition device or the first sound pickup device, the device indicated by the switching instruction is the target image acquisition device or the second sound pickup device, the first sound pickup device is a sound pickup device in the at least one sound pickup device, and the second sound pickup device is a sound pickup device in the at least one sound pickup device other than the first sound pickup device.

[0197] Through the audio and video synchronous output method provided in the embodiment of the present application, when collecting evidence on audio and video data, since the source of the audio data can be switched according to demand, when the audio is not clear enough, clear and accurate audio data can be obtained by simply switching the audio source, and then the target audio and video data with clear audio can be output, which simplifies the evidence collection process.

[0198] In a possible implementation, the display instruction is a preview instruction, and the method further includes:

[0199] The buffer used to store the original audio and video data from the image acquisition device is used as the buffer corresponding to the image acquisition device and recorded in the data buffer information library; the buffer used to store the audio data from the sound pickup device is used as the buffer corresponding to the sound pickup device and recorded in the data buffer information library; wherein the audio and video data from different image acquisition devices are stored in different buffers, and the audio data from different sound pickup devices are stored in different buffers;

[0200] Acquiring original audio and video data from a target image acquisition device indicated by a display instruction and second audio data from a device indicated by the display instruction bound to the target image acquisition device, including:

[0201] In response to the preview instruction, searching the first data buffer and the second data buffer in the data buffer information library of the video recording device, reading the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction from the first data buffer, and reading the audio data corresponding to the device bound to the target image acquisition device from the second data buffer; wherein the first data buffer is the buffer corresponding to the target image acquisition device indicated by the preview instruction, and the second data buffer is the buffer corresponding to the device bound to the target image acquisition device;

[0202] Synchronously display the target video data and target audio data with the same timestamp, including:

[0203] The video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the device bound to the target image acquisition device are played synchronously in the order of timestamps.

[0204] In a possible implementation manner, the device indicated by the switching instruction is an image acquisition device; and the method further includes:

[0205] In response to the switching instruction, the step of reading the audio data corresponding to the device bound to the target image acquisition device from the second data buffer is terminated, and the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction is parsed, and the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction is played.

[0206] In a possible implementation manner, the device indicated by the switching instruction is a second sound pickup device; and the method further includes:

[0207] In response to the switching instruction, the step of reading the audio data corresponding to the device bound to the target image acquisition device from the second data buffer is terminated, the video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the device bound to the target image acquisition device are synchronously played in the order of the timestamps, the third data buffer corresponding to the second sound pickup device is searched in the data buffer information library of the video recording device, and the third audio data corresponding to the second sound pickup device is read from the third data buffer;

[0208] Synchronously display the target video data and target audio data with the same timestamp, including:

[0209] The video data and the third audio data corresponding to the target image acquisition device indicated by the preview instruction are synchronously played in the order of the timestamps.

[0210] In a possible implementation, the display instruction is a playback instruction, and the method further includes:

[0211] The buffer used to store the original audio and video data from the image acquisition device is used as the buffer corresponding to the image acquisition device and recorded in the data buffer information library; the buffer used to store the audio data from the sound pickup device is used as the buffer corresponding to the sound pickup device and recorded in the data buffer information library; wherein the audio and video data from different image acquisition devices are stored in different buffers, and the audio data from different sound pickup devices are stored in different buffers;

[0212] In response to a playback instruction, a first code stream buffer for storing code streams is allocated to the target image acquisition device indicated by the playback instruction, and a second code stream buffer for storing code streams is allocated to the sound pickup device; the first data buffer and the second data buffer are searched in the data buffer information library of the video recording device, and the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction are read from the first data buffer, and the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction are stored in the first code stream buffer; the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction are read from the second data buffer, and the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction are stored in the second code stream buffer; wherein the first data buffer is the buffer corresponding to the target image acquisition device indicated by the playback instruction, and the second data buffer is the buffer corresponding to the device bound to the target image acquisition device;

[0213] Acquiring original audio and video data from a target image acquisition device indicated by a display instruction and second audio data from a device indicated by the display instruction bound to the target image acquisition device, including:

[0214] Read the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction from the first code stream buffer, and store it in the first code stream parsing buffer; read the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction from the second code stream buffer, and store it in the second code stream parsing buffer;

[0215] Parsing the original audio and video data to obtain target video data and first audio data includes:

[0216] Parsing the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction to obtain the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction;

[0217] Acquiring original audio and video data from a target image acquisition device indicated by a display instruction and second audio data from a device indicated by the display instruction bound to the target image acquisition device, including:

[0218] Reading the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction from the first code stream parsing buffer, and reading the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction from the second code stream parsing buffer;

[0219] Synchronously display the target video data and target audio data with the same timestamp, including:

[0220] The video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction are decoded and played synchronously.

[0221] In a possible implementation manner, the device indicated by the switching instruction is an image acquisition device; and the method further includes:

[0222] In response to the switching instruction, the steps of reading the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction from the second data buffer and storing the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction in the first code stream parsing buffer are terminated, and the second code stream buffer is released; the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction are stored in the decoding buffer;

[0223] The method also includes:

[0224] Read the original audio and video data from the decoding buffer, and decode and play the original audio and video data.

[0225] In a possible implementation manner, the device indicated by the switching instruction is a second sound pickup device; and the method further includes:

[0226] In response to the switching instruction, the second code stream buffer is released, and a third code stream buffer for storing the code stream is allocated to the second sound pickup device; the third data buffer is searched in the data buffer information library of the video recording device; the audio data of the second sound pickup device within the time range indicated by the playback instruction is read from the third data buffer, and the audio data of the second sound pickup device within the time range indicated by the playback instruction is stored in the third code stream buffer; wherein the third data buffer is the buffer corresponding to the second sound pickup device;

[0227] Reading the audio data of the second sound pickup device within the time range indicated by the playback instruction from the third code stream buffer, and storing it in the third code stream parsing buffer;

[0228] Synchronously display the target video data and target audio data with the same timestamp, including:

[0229] The video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the second sound pickup device within the time range indicated by the playback instruction are decoded and played synchronously.

[0230] The present application also provides an electronic device, including:

[0231] The memory is used to store computer programs; the processor is used to implement the steps of any of the above-mentioned audio and video synchronous output methods when executing the programs stored in the memory.

[0232] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0233] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0234] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned audio and video synchronous output methods are implemented.

[0235] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the audio and video synchronous output methods in the above embodiments.

[0236] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0237] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A video recording device, characterized in that: The video recording device is used to, in response to the display instruction, obtain original audio and video data from the target image acquisition device indicated by the display instruction, and parse the original audio and video data to obtain target video data and first audio data; Acquire second audio data from a device bound to the target image acquisition device as target audio data; Synchronously display the target video data and the target audio data with the same timestamp; wherein the video recording device has established a communication connection with at least one sound pickup device in advance, and has established a communication connection with at least one image acquisition device in advance, the device bound to the target image acquisition device is the target image acquisition device or the first sound pickup device, and the first sound pickup device is one of the at least one sound pickup devices; The video recording device is further used to, in response to a switching instruction, terminate the step of acquiring the second audio data from the device bound to the target image acquisition device, switch the device bound to the target image acquisition device to the device indicated by the switching instruction, acquire the third audio data from the device bound to the target image acquisition device as new target audio data; and synchronously display the target video data and the new target audio data with the same timestamp; wherein the device indicated by the switching instruction is the target image acquisition device or the second sound pickup device, and the second sound pickup device is a sound pickup device other than the first sound pickup device among the at least one sound pickup device; In response to a confirmation instruction for the target video data and the new target audio data, the target video data and the new target audio data are synchronously output to obtain target audio and video data.

2. The video recording device according to claim 1, characterized in that: The display instruction is a preview instruction; The video recording device is further used to use the buffer for storing the original audio and video data from the image acquisition device as the buffer corresponding to the image acquisition device, and record it in the data buffer information library; use the buffer for storing the audio data from the sound pickup device as the buffer corresponding to the sound pickup device, and record it in the data buffer information library; wherein the audio and video data from different image acquisition devices are stored in different buffers, and the audio data from different sound pickup devices are stored in different buffers; The video recording device is specifically configured to, in response to a preview instruction, search a first data buffer and a second data buffer in a data buffer information library of the video recording device, read original audio and video data corresponding to a target image acquisition device indicated by the preview instruction from the first data buffer, and read audio data corresponding to a device bound to the target image acquisition device from the second data buffer; wherein the first data buffer is a buffer corresponding to the target image acquisition device indicated by the preview instruction, and the second data buffer is a buffer corresponding to a device bound to the target image acquisition device; The recording device is specifically used to parse the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction to obtain the video data corresponding to the target image acquisition device indicated by the preview instruction; and synchronously play the video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the device bound to the target image acquisition device in the order of timestamps.

3. The video recording device according to claim 2, characterized in that: The device indicated by the switching instruction is the image acquisition device; The recording device is also used to, in response to a switching instruction, terminate the execution of the step of reading the audio data corresponding to the device bound to the target image acquisition device from the second data buffer, parsing the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction, and playing the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction.

4. The video recording device according to claim 2, characterized in that: The device indicated by the switching instruction is the second sound pickup device; The video recording device is specifically used to, in response to the switching instruction, terminate the execution of the step of reading the audio data corresponding to the device bound to the target image acquisition device from the second data buffer, synchronously play the video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the device bound to the target image acquisition device in the order of timestamps, search the data buffer information library of the video recording device for the third data buffer corresponding to the second sound pickup device, read the third audio data corresponding to the second sound pickup device from the third data buffer; and synchronously play the video data corresponding to the target image acquisition device indicated by the preview instruction and the third audio data in the order of timestamps.

5. The video recording device according to claim 1, characterized in that: The display instruction is a preview instruction, and the video recording device also establishes a communication connection with the remote control device; The video recording device is also used to send the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the device bound to the target image acquisition device to the remote control device, so that the remote control device parses the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction, obtains the video data corresponding to the target image acquisition device indicated by the preview instruction, and synchronously plays the video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the device bound to the target image acquisition device in the order of timestamps.

6. The video recording device according to claim 5, characterized in that: The device indicated by the switching instruction is the image acquisition device; The video recording device is also used to terminate the step of parsing the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction and sending the audio data corresponding to the device bound to the target image acquisition device to the remote control device under the control of the remote control device; The remote control device controls the video recording device in response to a switching instruction, and the control device also plays the original audio and video data synchronously in the order of timestamps in response to the switching instruction.

7. The video recording device according to claim 5, characterized in that: The device indicated by the switching instruction is the second sound pickup device; The video recording device is further used to, under the control of the remote control device, terminate the step of sending the audio data corresponding to the device bound to the target image acquisition device to the remote control device, and send the audio data corresponding to the second sound pickup device to the remote control device; Among them, the remote control device controls the recording device in response to the switching instruction, and the remote control device also responds to the switching instruction to synchronously play the video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the second sound pickup device in the order of timestamps.

8. The video recording device according to claim 1, characterized in that: The display instruction is a playback instruction; The video recording device is further used to use the buffer for storing the original audio and video data from the image acquisition device as the buffer corresponding to the image acquisition device, and record it in the data buffer information library; use the buffer for storing the audio data from the sound pickup device as the buffer corresponding to the sound pickup device, and record it in the data buffer information library; wherein the audio and video data from different image acquisition devices are stored in different buffers, and the audio data from different sound pickup devices are stored in different buffers; The video recording device is also used to, in response to a playback instruction, allocate a first code stream buffer for storing code streams to the target image acquisition device indicated by the playback instruction, and allocate a second code stream buffer for storing code streams to the sound pickup device; search for a first data buffer and a second data buffer in a data buffer information library of the video recording device, read from the first data buffer the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction, and store the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction in the first code stream buffer; read from the second data buffer the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction, and store the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction in the second code stream buffer; wherein the first data buffer is a buffer corresponding to the target image acquisition device indicated by the playback instruction, and the second data buffer is a buffer corresponding to the device bound to the target image acquisition device; The video recording device is specifically used to read the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction from the first code stream buffer, and store it in the first code stream parsing buffer; read the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction from the second code stream buffer, and store it in the second code stream parsing buffer; parse the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction to obtain the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction; The video recording device is also used to read from the first code stream parsing buffer the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction, and read from the second code stream parsing buffer the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction; and decode and play synchronously the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction.

9. The video recording device according to claim 8, characterized in that: The device indicated by the switching instruction is the image acquisition device; The video recording device is also used to, in response to the switching instruction, terminate the execution of the steps of reading the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction from the second data buffer, storing the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction in the first code stream parsing buffer, and release the second code stream buffer; storing the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction in the decoding buffer; The video recording device is specifically used to read the original audio and video data from the decoding buffer, and decode and play the original audio and video data.

10. The video recording device according to claim 8, characterized in that: The device indicated by the switching instruction is the second sound pickup device; The video recording device is further configured to, in response to the switching instruction, release the second code stream buffer and allocate a third code stream buffer for storing the code stream to the second sound pickup device; search for a third data buffer in a data buffer information library of the video recording device; read the audio data of the second sound pickup device within the time range indicated by the playback instruction from the third data buffer, and store the audio data of the second sound pickup device within the time range indicated by the playback instruction in the third code stream buffer; wherein the third data buffer is a buffer corresponding to the second sound pickup device; The video recording device is further used to read the audio data of the second sound pickup device within the time range indicated by the playback instruction from the third code stream buffer, and store it in the third code stream parsing buffer; The video recording device is specifically used to decode and synchronously play the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the second sound pickup device within the time range indicated by the playback instruction.

11. The video recording device according to claim 1, characterized in that: The display instruction is a playback instruction, and the video recording device also establishes a communication connection with the remote control device; The video recording device is also used to send the original video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction to the remote control device, so that the remote control device parses the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction, obtains the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction, and synchronously plays the video data of the image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction.

12. The video recording device according to claim 11, characterized in that: The device indicated by the switching instruction is the image acquisition device; The video recording device is also used to terminate the step of parsing the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction, and sending the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction to the remote control device under the control of the remote control device; The remote control device controls the video recording device in response to a switching instruction, and the remote control device also decodes and plays the original audio and video data in response to the switching instruction.

13. The video recording device according to claim 11, characterized in that: The device indicated by the switching instruction is the second sound pickup device; The video recording device is further used to, under the control of the remote control device, terminate the step of sending the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction to the remote control device, and send the audio data of the second sound pickup device within the time range indicated by the playback instruction to the remote control device; Among them, the remote control device controls the recording device in response to the switching instruction, and the remote control device also decodes and synchronously plays the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the second sound pickup device within the time range indicated by the playback instruction.

14. The video recording device according to claim 1, characterized in that: The display instruction is a playback instruction, and the recording device is also used to, in response to a fallback instruction, terminate the step of synchronously displaying the target video data and the new target audio data with the same timestamp, and synchronously display the target audio data and the target video data from the device indicated by the switching instruction from the time indicated by the fallback instruction.

15. A method for synchronously outputting audio and video, characterized in that: Applied to a video recording device, the method comprises: In response to a display instruction, original audio and video data from a target image acquisition device indicated by the display instruction is obtained; the original audio and video data is parsed to obtain target video data and first audio data, and second audio data from a device bound to the target image acquisition device is obtained as target audio data, and the target video data and target audio data having the same timestamp are synchronously displayed; In response to the switching instruction, the step of acquiring the second audio data from the device bound to the target image acquisition device is terminated, the device bound to the target image acquisition device is switched to the device indicated by the switching instruction, and the third audio data from the second device bound to the target image acquisition device is acquired as the new target audio data; Synchronously displaying the target video data and the target audio data having the same timestamp; In response to a confirmation instruction for the target video data and the new target audio data, synchronously outputting the target video data and the new target audio data to obtain target audio and video data; Among them, the video recording device has pre-established a communication connection with at least one sound pickup device, and has pre-established a communication connection with at least one image acquisition device, the device bound to the target image acquisition device is the target image acquisition device or the first sound pickup device, the device indicated by the switching instruction is the target image acquisition device or the second sound pickup device, the first sound pickup device is a sound pickup device in the at least one sound pickup device, and the second sound pickup device is a sound pickup device in the at least one sound pickup device except the first sound pickup device.

16. The method according to claim 15, characterized in that The display instruction is a preview instruction, and the method further includes: The buffer used to store the original audio and video data from the image acquisition device is used as the buffer corresponding to the image acquisition device and recorded in the data buffer information library; the buffer used to store the audio data from the sound pickup device is used as the buffer corresponding to the sound pickup device and recorded in the data buffer information library; wherein the audio and video data from different image acquisition devices are stored in different buffers, and the audio data from different sound pickup devices are stored in different buffers; The acquiring of original audio and video data from the target image acquisition device indicated by the display instruction and second audio data from the device indicated by the display instruction bound to the target image acquisition device includes: In response to the preview instruction, searching the data buffer information base of the video recording device for a first data buffer and a second data buffer, reading the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction from the first data buffer, and reading the audio data corresponding to the device bound to the target image acquisition device from the second data buffer; wherein the first data buffer is the buffer corresponding to the target image acquisition device indicated by the preview instruction, and the second data buffer is the buffer corresponding to the device bound to the target image acquisition device; The synchronously displaying the target video data and the target audio data having the same timestamp includes: The video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the device bound to the target image acquisition device are synchronously played in the order of timestamps.

17. The method according to claim 16, characterized in that The device indicated by the switching instruction is the image acquisition device; the method further includes: In response to the switching instruction, the step of reading the audio data corresponding to the device bound to the target image acquisition device from the second data buffer is terminated, and the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction is parsed, and the original audio and video data corresponding to the target image acquisition device indicated by the preview instruction is played.

18. The method according to claim 16, characterized in that The device indicated by the switching instruction is the second sound pickup device; and the method further includes: In response to the switching instruction, the step of reading the audio data corresponding to the device bound to the target image acquisition device from the second data buffer is terminated, the video data corresponding to the target image acquisition device indicated by the preview instruction and the audio data corresponding to the device bound to the target image acquisition device are synchronously played in the order of timestamps, the third data buffer corresponding to the second sound pickup device is searched in the data buffer information library of the video recording device, and the third audio data corresponding to the second sound pickup device is read from the third data buffer; The synchronously displaying the target video data and the target audio data having the same timestamp includes: The video data corresponding to the target image acquisition device indicated by the preview instruction and the third audio data are synchronously played in the order of the timestamps.

19. The method according to claim 15, characterized in that The display instruction is a playback instruction, and the method further includes: The buffer used to store the original audio and video data from the image acquisition device is used as the buffer corresponding to the image acquisition device and recorded in the data buffer information library; the buffer used to store the audio data from the sound pickup device is used as the buffer corresponding to the sound pickup device and recorded in the data buffer information library; wherein the audio and video data from different image acquisition devices are stored in different buffers, and the audio data from different sound pickup devices are stored in different buffers; In response to a playback instruction, a first code stream buffer for storing code streams is allocated to the target image acquisition device indicated by the playback instruction, and a second code stream buffer for storing code streams is allocated to the sound pickup device; a first data buffer and a second data buffer are searched in a data buffer information library of the video recording device, original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction are read from the first data buffer, and the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction are stored in the first code stream buffer; audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction are read from the second data buffer, and the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction are stored in the second code stream buffer; wherein the first data buffer is a buffer corresponding to the target image acquisition device indicated by the playback instruction, and the second data buffer is a buffer corresponding to the device bound to the target image acquisition device; Acquiring original audio and video data from a target image acquisition device indicated by the display instruction and second audio data from a device indicated by the display instruction bound to the target image acquisition device, including: Reading the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction from the first code stream buffer, and storing it in the first code stream parsing buffer; reading the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction from the second code stream buffer, and storing it in the second code stream parsing buffer; The parsing of the original audio and video data to obtain target video data and first audio data includes: Parsing the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction to obtain the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction; The acquiring of original audio and video data from the target image acquisition device indicated by the display instruction and second audio data from the device indicated by the display instruction bound to the target image acquisition device includes: Reading the video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction from the first code stream parsing buffer, and reading the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction from the second code stream parsing buffer; The synchronously displaying the target video data and the target audio data having the same timestamp includes: The video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction are decoded and played synchronously.

20. The method according to claim 19, characterized in that The device indicated by the switching instruction is the image acquisition device; the method further includes: In response to the switching instruction, the steps of reading the audio data of the device bound to the target image acquisition device within the time range indicated by the playback instruction from the second data buffer and storing the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction in the first code stream parsing buffer are terminated, and the second code stream buffer is released; the original audio and video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction are stored in the decoding buffer; The method further comprises: The original audio and video data is read from the decoding buffer, and the original audio and video data is decoded and played.

21. The method according to claim 19, characterized in that The device indicated by the switching instruction is the second sound pickup device; and the method further includes: In response to the switching instruction, the second code stream buffer is released, and a third code stream buffer for storing the code stream is allocated to the second sound pickup device; the third data buffer is searched in the data buffer information library of the video recording device; the audio data of the second sound pickup device within the time range indicated by the playback instruction is read from the third data buffer, and the audio data of the second sound pickup device within the time range indicated by the playback instruction is stored in the third code stream buffer; wherein the third data buffer is the buffer corresponding to the second sound pickup device; Read the audio data of the second sound pickup device within the time range indicated by the playback instruction from the third code stream buffer, and store it in the third code stream parsing buffer; The synchronously displaying the target video data and the target audio data having the same timestamp includes: The video data of the target image acquisition device indicated by the playback instruction within the time range indicated by the playback instruction and the audio data of the second sound pickup device within the time range indicated by the playback instruction are decoded and played synchronously.

Images