PjProject-based video recording method, apparatus and device, and storage medium

Abstract

The invention provides a pjProject-based video recording method, device and equipment and a storage medium, and the pjProject-based video recording method comprises the steps: obtaining a video stream and an audio stream in a call process through a sip client at the beginning of a call, storing the video stream and the audio stream in a storage queue, calling a video recording function through a pjProject framework, and carrying out the video recording through the video recording function. The method comprises the steps of determining a playing time difference between a video stream and an audio stream, determining a compensation judgment result according to the playing time difference and a time threshold value, generating a video file according to a video recording function, the video stream and the audio stream when a call is ended according to the compensation judgment result, realizing a video recording function by a sip client developed based on pjProject, and generating the video file based on the compensation judgment result. And the video recording effect is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of video recording, and in particular to a video recording method and device based on pjproject, an equipment and a storage medium. BACKGROUND

[0002] Currently, in a video call, when it is necessary to record the call, generally only the server can record, and the client only provides an audio recording function, but does not provide a video recording function. In the server recording, on the one hand, the server performance is required to be high, and on the other hand, in some special scenarios, for example, in some network environment with high security requirements, the server of the network environment other than the server of the network environment is prohibited to store the video, so as to prevent data from flowing out, at this time, the server can only record the video call content of one party, which cannot meet the needs of users. SUMMARY

[0003] The embodiments of the present application provide a video recording method and device based on pjproject, an equipment and a storage medium, to solve at least one problem in the related art, and the technical solutions are as follows:

[0004] In a first aspect, the embodiments of the present application provide a video recording method based on pjproject, comprising:

[0005] When the call starts, the video stream and the audio stream in the call process are acquired through a sip client;

[0006] The video stream and the audio stream are stored in a storage queue, and a video recording function is called through a pjproject framework;

[0007] The video recording function is used to determine a playing time difference between the video stream and the audio stream, and a compensation determination result is determined according to the playing time difference and a time threshold;

[0008] When the call ends, a video recording file is generated by using the video recording function, the video stream and the audio stream according to the compensation determination result.

[0009] In an implementation manner, the storage queue comprises a video queue and an audio queue; and the storage of the video stream and the audio stream in the storage queue comprises:

[0010] The video stream is stored in the video queue through ffmpeg_vid_codecs.c;

[0011] When the video queue contains the video stream, the audio stream is stored in the audio queue through conference.c.

[0012] In an embodiment, the calling the recording function through the pjproject framework comprises:

[0013] When the video queue of the storage queue has a video stream and the audio queue of the storage queue has an audio stream, calling the recording function through the pjproject framework.

[0014] In an embodiment, the determining the playback time difference between the video stream and the audio stream through the recording function comprises:

[0015] Through the recording function, taking out the video stream and the audio stream from the storage queue respectively, and storing the taken-out video stream and the taken-out audio stream into an AVPacket object;

[0016] Determining a first display timestamp of a video frame in the video stream and a second display timestamp of an audio frame in the audio stream, and storing the first display timestamp and the second display timestamp into the AVPacket object;

[0017] According to the difference between the first display timestamp and the second display timestamp in the AVPacket object, determining the playback time difference between the video stream and the audio stream.

[0018] In an embodiment, the determining the compensation judgment result according to the playback time difference and a time threshold comprises:

[0019] Comparing the playback time difference with the time threshold;

[0020] When the playback time difference is greater than the time threshold, determining a compensation judgment result representing that compensation is needed;

[0021] When the playback time difference is less than or equal to the time threshold, determining a compensation judgment result representing that compensation is not needed.

[0022] In an embodiment, the generating a recording file through the recording function, the video stream and the audio stream according to the compensation judgment result comprises:

[0023] Through the recording function, performing write operation on the video frame of the video stream to generate an initial recording;

[0024] When the compensation judgment result represents that compensation is needed, performing compensation processing on the audio stream to obtain a compensated audio stream, and through the recording function, processing the initial recording and the compensated audio stream to generate a recording file;

[0025] When the compensation judgment result represents that compensation is not needed, the initial video and the audio stream are processed by the video recording function to generate a video recording file.

[0026] In an implementation, the compensating processing of the audio stream to obtain a compensated audio stream when the compensation judgment result represents that compensation is needed includes:

[0027] When the compensation judgment result represents that compensation is needed, empty packets are supplemented in the audio stream to obtain a new audio stream;

[0028] According to the new audio stream, the step of determining the playing time difference between the video stream and the audio stream by the video recording function is returned until the compensation judgment result representing that compensation is not needed is obtained.

[0029] In a second aspect, an embodiment of the present application provides a video recording device based on pjproject, including:

[0030] The obtaining module is configured to obtain a video stream and an audio stream in a call process by a sip client when the call starts;

[0031] The storage module is configured to store the video stream and the audio stream into a storage queue and call a video recording function by a pjproject framework;

[0032] The determining module is configured to determine a playing time difference between the video stream and the audio stream by the video recording function and determine a compensation judgment result according to the playing time difference and a time threshold;

[0033] The generating module is configured to generate a video recording file by the video recording function, the video stream and the audio stream according to the compensation judgment result when the call ends.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory stores instructions, the instructions are loaded and executed by the processor to implement the method in any of the implementation modes of the above aspects.

[0035] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed to implement the method in any of the implementation modes of the above aspects.

[0036] The above technical solution has at least the following beneficial effects:

[0037] At the beginning of the call, the video stream and the audio stream in the call are acquired by the sip client, the video stream and the audio stream are stored into a storage queue, and a video recording function is called through the pjproject framework, through the video recording function, a playing time difference of the video stream and the audio stream is determined, and a compensation judgment result is determined according to the playing time difference and a time threshold, at the end of the call, according to the compensation judgment result, a video recording file is generated by using the video recording function, the video stream and the audio stream, the sip client developed based on the pjproject realizes the video recording function, and the video recording file is generated based on the compensation judgment result, which is beneficial to guarantee the video recording effect.

[0038] The foregoing summary is provided only for purposes of summary only and is not intended to limit the application in any manner. Further aspects of the application, as well as aspects and features of the application, will become apparent to those skilled in the art from the following detailed description, which is to be taken in conjunction with the drawings and the appended claims. BRIEF DESCRIPTION OF DRAWINGS

[0039] In the drawings, like numerals refer to like elements throughout the various drawings. The drawings are not necessarily to scale, the drawings are merely schematic representations, and are not intended to portray specific parameters of the application. The drawings are intended to depict only typical embodiments of the application, and therefore should not be considered as limiting the scope of the application.

[0040] Figure 1 A step flowchart of a video recording method based on pjproject for an embodiment of the application;

[0041] Figure 2 A flowchart of a video recording method based on pjproject for an embodiment of the application;

[0042] Figure 3 A structure block diagram of a video recording device based on pjproject for an embodiment of the application;

[0043] Figure 4 A structure block diagram of an electronic device for an embodiment of the application. DETAILED DESCRIPTION

[0044] In the following, only certain example embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the application. Therefore, the drawings and the description are considered to be essentially exemplary rather than limiting.

[0045] pjproject: a free and open source multimedia communication library, which can be used to develop a sip client.

[0046] sink port: structure used to collect and process data in pjproject.

[0047] ffmpeg: a multimedia framework with important status, with audio and video codec functions.

[0048] sip client: a soft phone based on pjproject, which can be used to replace a hard phone.

[0049] Jitterbuffer: jitter buffer is a shared data area in voice over IP (VoIP).

[0050] Referring to Figure 1 , a flowchart of a video recording method based on pjproject according to an embodiment of the present application is shown, which can at least include steps S100-S400:

[0051] S100, when the call starts, the video stream and the audio stream in the call process are obtained through the sip client.

[0052] S200, the video stream and the audio stream are stored in the storage queue, and a video recording function is called through the pjproject framework.

[0053] S300, the playback time difference between the video stream and the audio stream is determined through the video recording function, and the compensation judgment result is determined according to the playback time difference and the time threshold.

[0054] S400, when the call ends, the video recording file is generated according to the compensation judgment result, using the video stream and the audio stream in the video recording function and the storage queue.

[0055] The technical scheme of the embodiment of the present application can be executed by using the sip client (referred to as the client). When the call starts, the video stream and the audio stream in the call process are obtained through the sip client, the video stream and the audio stream are stored in the storage queue, and a video recording function is called through the pjproject framework. The playback time difference between the video stream and the audio stream is determined through the video recording function, and the compensation judgment result is determined according to the playback time difference and the time threshold. When the call ends, the video recording file is generated according to the compensation judgment result, using the video stream and the audio stream in the video recording function. The sip client based on pjproject realizes the video recording function, and the video recording file is generated based on the compensation judgment result, which is beneficial to guarantee the video recording effect.

[0056] In this embodiment of the application, a SIP client is developed based on the PJProject framework. The recording function code implemented in PJProject enables users to record audio and video through the SIP client.

[0057] It's important to note that when developing a SIP client, the data structures for audio and video frames should be pre-defined. This includes frame size, timestamps (denoted as time_stamp.u64) of RTP packets (the standard data packet format for transmitting real-time audio and video), the type of storage queue (e.g., video queue and audio queue), the presence of marker flags (for audio), and the local clock of the RTP packets (used to assist in calculating the corresponding display timestamps for audio or video). Additionally, the data structure of the storage queue can be configured, including memory pool allocation, a flag indicating whether a complete audio or video frame has been received (has_pps), locks for queue read / write operations, and the necessary content for recording. For example, a new file named record_conf.c can be created for initializing the ffmpeg encoder (audio / video codec), recording processing, and associating recording functions.

[0058] like Figure 2 As shown, at the start of a call, the FFmpeg encoders for video and audio are initialized via a SIP client. A connection is established with the `src_port` (the object in PJProject that generates video and audio streams; all `dst_port` objects connected to `src_port` are polled, with `src_port` sending received data to `dst_port`, which is used by the PJProject framework to call functions for writing recordings upon receiving audio or video frames). This connection is also linked to functions for writing recording files, such as the recording function. Furthermore, at the start of the call, the SIP client acquires the video and audio streams during the call. These streams can be acquired and transmitted in RTP packet format. It should be noted that during the acquisition process, `has_pps = true` to ensure that the video and audio are initially synchronized when the recording is played back.

[0059] In one implementation, step S200, storing the video stream and audio stream into a storage queue, includes steps S210-S220:

[0060] S210. Store the video stream to the video queue using ffmpeg_vid_codecs.c.

[0061] In this embodiment of the application, after unpacking the RTP packet using h264, the video stream can be extracted and stored.

[0062] S220, when the video queue exists a video stream, store the audio stream into the audio queue through conference.c.

[0063] In the embodiment of the present application, the client can extract the video stream (one or more frames of video data) through ffmpeg_vid_codecs.c and store it into the video queue, and when the video queue exists a video stream, store the audio stream (one or more frames of audio data) into the audio queue through conference.c, so as to ensure that the audio stream is stored when the video stream is collected, facilitate the synchronization of the video stream and the audio stream in the future, and ensure the final recording effect. It can be understood that the storage queue includes the video queue (also referred to as the video frame queue) and the audio queue (also referred to as the audio frame queue).

[0064] It should be noted that the RTP timestamp of the first frame of the video frame is recorded as vid_random_offset, which is recorded when the first frame of video is read, and the first display timestamp (pts) of the first frame of the video frame is 0, which is used to tell the player when to display the data of this frame; the cumulative timestamp that has been written is recorded as vid_pkt_pts_offset. At the same time, when the next frame of video frame is received, the first display timestamp pts of the video frame in the recording is time_stamp.u64-vid_random_offset+vid_pkt_pts_offset, when the RTP timestamp changes greatly, vid_random_offset will be reset to 0, vid_pkt_pts_offset is used to record the time that has been written into the recording, and the has_pps of the audio queue will be reset to false, which will prohibit the audio queue to continue to save the audio frame.

[0065] In one embodiment, the complete RTP packet read from the Jitterbuffer is stored into the video queue when it is sent to play, and when the first I frame is received, the recording file for recording the video is started to be written, so as to ensure that the first frame can be completely displayed when the video is played and displayed; audio processing: on the basis of the original, when the received audio frame is stored into the Jitterbuffer, the value of the RTP packet mark bit is added, and when the audio frame is read from the Jitterbuffer and sent to play, it is stored into the audio frame queue at the same time.

[0066] In one embodiment, the recording function is called through the pjproject framework in step S200, including step S230:

[0067] S230. When the video queue of the storage queue contains a video stream and the audio queue of the storage queue contains an audio stream, the recording function is called through the pjproject framework.

[0068] Optionally, if the video queue of the storage queue contains a video stream and the audio queue of the storage queue contains an audio stream, it means that the video stream and audio are ready for generating the recording file. At this time, the client developed based on the pjproject framework can use its pjproject framework to call the recording function to start the recording function.

[0069] like Figure 2 As shown, in one embodiment, step S300 determines the playback time difference between the video stream and the audio stream using a recording function, including steps S310-S330:

[0070] S310. Using the recording function, retrieve the video stream and audio stream from the storage queue respectively, and store the retrieved video stream and audio stream into the AVPacket object.

[0071] Optionally, when the recording function is called in the pjproject framework, the recording function is used to retrieve the video stream from the video queue and the audio stream from the audio queue of the storage queue, respectively. Then, the retrieved video and audio streams are stored in an AVPacket object to represent a single compressed video frame or multiple compressed audio frames, with a reference counting mechanism used to manage memory.

[0072] S320. Determine the first display timestamp of the video frame in the video stream and the second display timestamp of the audio frame in the audio stream, and store the first display timestamp and the second display timestamp into the AVPacket object.

[0073] Optionally, when the SIP client obtains the video and audio streams during a call, it uses its local clock to help determine the first display timestamp of the video frames in the video stream and the second display timestamp of the audio frames in the audio stream. Then, it stores the first and second display timestamps in an AVPacket object.

[0074] S330. Determine the playback time difference between the video stream and the audio stream based on the difference between the first display timestamp and the second display timestamp in the AVPacket object.

[0075] Optionally, after the AVPacket object receives the first display timestamp and the second display timestamp, it can determine the difference between the first display timestamp and the second display timestamp, that is, determine the playback time difference delta between the video stream and the audio stream.

[0076] In an embodiment, the step S300 of determining the compensation judgment result according to the play time difference and the time threshold comprises steps S340-S350:

[0077] S340, comparing the play time difference with the time threshold.

[0078] S350, when the play time difference is greater than the time threshold, determining a compensation judgment result representing that compensation is needed; when the play time difference is less than or equal to the time threshold, determining a compensation judgment result representing that compensation is not needed.

[0079] Optionally, the time threshold can be set according to actual conditions, for example, 200 ms. If the play time difference is greater than the time threshold, it is determined that the compensation judgment result represents that compensation is needed, that is, the video play time and the audio play time are too different, and the audio stream and the video stream are obviously out of synchronization, and subsequent compensation processing (equivalent to audio-visual synchronization compensation) is needed. When the play time difference is less than or equal to the time threshold, it is determined that the compensation judgment result represents that compensation is not needed, indicating that the audio stream and the video stream are relatively synchronized, and the recording effect will not be affected, and compensation processing is not needed.

[0080] In an embodiment, when the call ends, the recording is ended, and a final complete recording file is generated; the step S400 comprises steps S410-S430:

[0081] S410, using a recording function to perform write operation on the video frames of the video stream to generate an initial recording.

[0082] In the embodiment of the application, the client can use the recording function to perform write operation on the video frames of the video stream to generate an initial recording. It should be noted that the video stream in the video queue or the video stream in the AVPacket object can be used.

[0083] S420, when the compensation judgment result represents that compensation is needed, performing compensation processing on the audio stream to obtain a compensated audio stream, and processing the initial recording and the compensated audio stream by using the recording function to generate a recording file.

[0084] Optionally, when the compensation judgment result represents that compensation is needed, empty packets are supplemented in the audio stream to obtain a new audio stream, and then the step of determining the play time difference between the video stream and the audio stream by using the recording function is returned, that is, the step S300 is returned. If the compensation judgment result still represents that compensation is needed, empty packets are continuously supplemented in the audio stream until the compensation judgment result representing that compensation is not needed is obtained, indicating that the audio stream and the video stream are synchronized or very close to synchronization at this time.

[0085] Finally, the initial video recording and the compensated audio stream (i.e., the final audio stream determined after one or more compensations of empty packets) are processed by a video recording function, such as a video stream and an audio stream are synthesized, to generate a video recording file.

[0086] S430, when the compensation determination result indicates that no compensation is needed, the initial video recording and the audio stream are processed by a video recording function to generate a video recording file.

[0087] Optionally, when the compensation determination result indicates that no compensation is needed, the initial video recording and the audio stream are processed by a video recording function, such as a video stream and an audio stream are synthesized, to generate a video recording file. It should be noted that the audio stream in the audio queue or the audio stream in the AVPacket object can be used.

[0088] Referring to Figure 3 , a structural block diagram of a pjproject-based video recording device according to an embodiment of the present application is shown, which can include:

[0089] The acquisition module is configured to acquire the video stream and the audio stream in the call process through the sip client when the call starts.

[0090] The storage module is configured to store the video stream and the audio stream into a storage queue and call a video recording function through a pjproject framework.

[0091] The determination module is configured to determine a playback time difference between the video stream and the audio stream by the video recording function, and determine a compensation determination result according to the playback time difference and a time threshold.

[0092] The generation module is configured to generate a video recording file by the video recording function, the video stream and the audio stream according to the compensation determination result when the call ends.

[0093] The functions of each module in each device according to the embodiments of the present application can be referred to the corresponding description in the above method, which will not be repeated here.

[0094] Referring to Figure 4 , a structural block diagram of an electronic device according to an embodiment of the present application is shown, which includes a memory 310 and a processor 320, the memory 310 stores instructions executable on the processor 320, and the processor 320 loads and executes the instructions to implement the pjproject-based video recording method according to the above embodiments. The number of the memory 310 and the processor 320 can be one or more.

[0095] In an implementation, the electronic device further includes a communication interface 330 for communicating with external devices to transmit and receive data. If the memory 310, the processor 320 and the communication interface 330 are implemented independently, the memory 310, the processor 320 and the communication interface 330 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 4 Only one thick line is used to represent the bus in the figure, but it does not mean that there is only one bus or only one type of bus.

[0096] Optionally, in a specific implementation, if the memory 310, the processor 320 and the communication interface 330 are integrated on a chip, the memory 310, the processor 320 and the communication interface 330 can communicate with each other through an internal interface.

[0097] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the video recording method based on pjproject provided in the above embodiment.

[0098] The embodiment of the present application further provides a chip, which includes a processor, and the processor is used to call and run instructions stored in a memory, so that a communication device installed with the chip executes the method provided in the embodiment of the present application.

[0099] The embodiment of the present application further provides a chip, which includes an input interface, an output interface, a processor and a memory, the input interface, the output interface, the processor and the memory are connected through an internal connection path, and the processor is used to execute code in the memory, and when the code is executed, the processor is used to execute the method provided in the embodiment of the present application.

[0100] It is to be understood that the above-described processor can be a central processing unit (CPU), but can also be other general purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general purpose processor can be a microprocessor or any conventional processor, etc. It is to be noted that the processor can be an advanced RISC machine (ARM) architecture processor.

[0101] Further, the memory described above can include a read-only memory and a random access memory, and can also include a non-volatile random access memory. The memory can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can include a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM) and direct rambus RAM (DR RAM) can be used.

[0102] In the above-described embodiments, all or part can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded on a computer, all or part generates a flow or function according to the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium.

[0103] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Also, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0104] In addition, the terms "first", "second", etc. are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.

[0105] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions or other processes. Also, the preferred embodiments of the present application can include additional implementation with respect to the order in which steps are performed, including a substantially simultaneous performance of the functions according to the functionality involved, or a performance of the functions in reverse order, depending upon the functionality involved.

[0106] The logic and / or steps represented in flow charts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, which can be specifically embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus or device, such as a computer-based system, a system including a processor, or other system that can fetch the instructions from the instruction execution system, apparatus or device and execute the instructions, or in conjunction with these instructions execution system, apparatus or device.

[0107] It should be understood that each part of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above-mentioned embodiment method can be completed by a program instructing the relevant hardware, which can be stored in a computer readable storage medium, and the program includes one of the steps of the method embodiment or a combination thereof when executed.

[0108] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of a software functional module. The above-mentioned integrated module, if realized in the form of a software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium. The storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0109] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for recording a video based on pjproject, characterized in that, The method comprises the following steps: When the call starts, the video stream and the audio stream in the call are acquired by a sip client; The video stream and the audio stream are stored in a storage queue, and a video recording function is called through a pjproject framework; The playing time difference between the video stream and the audio stream is determined through the video recording function, and a compensation judgment result is determined according to the playing time difference and a time threshold value; When the call ends, a video recording file is generated according to the compensation judgment result, by using the video recording function, the video stream and the audio stream.

2. The method of claim 1, wherein the method is based on pjproject. The storage queue comprises a video queue and an audio queue; the video stream and the audio stream are stored in the storage queue by the following steps: The video stream is stored in the video queue through ffmpeg_vid_codecs.c; When the video queue contains the video stream, the audio stream is stored in the audio queue through conference.c.

3. The method of claim 1 or 2, wherein the method is based on pjproject. The video recording function is called through the pjproject framework by the following steps: When the video queue of the storage queue contains the video stream and the audio queue of the storage queue contains the audio stream, the video recording function is called through the pjproject framework.

4. The method of claim 3, wherein the pjproject-based recording method is characterized by: The playing time difference between the video stream and the audio stream is determined through the video recording function by the following steps: The video stream and the audio stream are taken out from the storage queue respectively through the video recording function, and the taken-out video stream and audio stream are stored in an AVPacket object; The first display timestamp of a video frame in the video stream and the second display timestamp of an audio frame in the audio stream are determined, and the first display timestamp and the second display timestamp are stored in the AVPacket object; The playing time difference between the video stream and the audio stream is determined according to the difference between the first display timestamp and the second display timestamp in the AVPacket object.

5. The method of claim 3, wherein the pjproject-based video recording method is characterized by: The compensation judgment result is determined according to the playing time difference and the time threshold value by the following steps: The playing time difference is compared with the time threshold value; When the playing time difference is greater than the time threshold value, a compensation judgment result indicating that compensation is needed is determined; When the playing time difference is less than or equal to the time threshold value, a compensation judgment result indicating that compensation is not needed is determined.

6. The method of claim 5, wherein the pjproject-based recording method is characterized by: The video recording file is generated according to the compensation judgment result by using the video recording function, the video stream and the audio stream by the following steps: The video frame of the video stream is written through the video recording function to generate an initial video recording; When the compensation judgment result indicates that compensation is needed, the audio stream is compensated to obtain a compensated audio stream, and the initial video recording and the compensated audio stream are processed through the video recording function to generate a video recording file; When the compensation judgment result indicates that compensation is not needed, the initial video recording and the audio stream are processed through the video recording function to generate a video recording file.

7. The method of claim 6, wherein the pjproject-based video recording method is characterized by: The audio stream is compensated to obtain a compensated audio stream when the compensation judgment result indicates that compensation is needed. When the compensation judgment result represents that compensation is needed, a null packet is supplemented in the audio stream to obtain a new audio stream; According to the new audio stream, the step of determining the playing time difference between the video stream and the audio stream through the video recording function is returned until the compensation judgment result representing that compensation is not needed is obtained.

8. A video recording apparatus based on pjproject, characterized by, The method comprises the steps of: When the call starts, a video stream and an audio stream in the call are acquired through a sip client by an acquisition module; The video stream and the audio stream are stored into a storage queue by a storage module through a pjproject framework calling a video recording function; A playing time difference between the video stream and the audio stream is determined through the video recording function by a determination module, and a compensation judgment result is determined according to the playing time difference and a time threshold value; When the call ends, a video recording file is generated according to the compensation judgment result, the video recording function, the video stream and the audio stream by a generation module.

9. An electronic device, comprising: The method comprises the steps of: A processor and a memory, the memory stores instructions, the instructions are loaded and executed by the processor to implement the method as claimed in any one of claims 1-7. 10.A computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed to implement the method as claimed in any one of claims 1-7.

Images