PTZ control method and device, storage medium and electronic device

Abstract

The present application discloses a pan-tilt control method and device, a storage medium and an electronic device. The method includes: in response to a pan-tilt control start instruction, controlling the pan-tilt to execute a target action corresponding to the pan-tilt control start instruction, and obtaining the delay and frame rate corresponding to the pan-tilt, wherein the pan-tilt control start instruction is sent by the target terminal, the delay indicates the transmission delay between the target terminal and the pan-tilt, and the frame rate indicates the frame rate used by the pan-tilt to transmit data; based on the number of frames sent by the pan-tilt to the target terminal and the number of frames received by the target terminal, the delay and the frame rate, the target frame number is determined, the non-key frame data corresponding to the target frame number in the first cache data is cleared, and in response to a pan-tilt control end instruction, the pan-tilt is controlled to stop executing the target action. The present application solves the technical problem in the related art that the pan-tilt is not synchronized enough with the video screen and the video screen is delayed.

Description

Technical Field

[0001] The present application relates to the field of computers, and more specifically, to a pan / tilt control method and device, a storage medium, and an electronic device. Background Art

[0002] In the field of video surveillance, the PTZ is an auxiliary device in the video surveillance system, usually used to control the direction and field of view of the monitoring equipment. It can realize the remote rotation, zooming and zooming of the monitoring equipment to monitor a specific area. The PTZ has a wide range of application scenarios in the field of video surveillance, including indoor and outdoor security monitoring, traffic monitoring, urban monitoring, public place monitoring, etc. Through the remote control and flexibility of the PTZ, monitoring personnel can more efficiently manage and monitor activities in different areas, enhancing security and monitoring effects.

[0003] In practical applications, the real-time synchronization of PTZ rotation and video images is crucial. The delay of video images is affected by multiple links such as encoding, transmission, decoding and display. In particular, network factors in the transmission process will cause a backlog of transmission data, which will increase the delay. At the same time, when the PTZ rotates, it will cause changes in the video image, which will increase the complexity of the image. Therefore, more data is needed to accurately express the details of the image, which will also increase the delay between the PTZ and the video image.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a pan-tilt control method and device, a storage medium and an electronic device to at least solve the technical problems in the related art that the pan-tilt and the video screen are not synchronized enough and the video screen has a delay.

[0006] According to one aspect of an embodiment of the present application, a gimbal control method is provided, comprising: in response to a gimbal control start instruction, controlling the gimbal to execute a target action corresponding to the gimbal control start instruction, and obtaining a delay and frame rate corresponding to the gimbal, wherein the gimbal control start instruction is sent by a target terminal, the delay represents a transmission delay between the target terminal and the gimbal, and the frame rate represents a frame rate used by the gimbal to transmit data; determining a target frame number based on a flight frame number, the delay and the frame rate, wherein the flight frame number represents the number of frames that the gimbal has sent to the target terminal but has not yet determined the number of frames that the target terminal has received; clearing non-key frame data corresponding to the target frame number in a first cache data, wherein the first cache data represents data to be sent by the gimbal to the target terminal, and the first cache data includes the non-key frame data; in response to a gimbal control end instruction, controlling the gimbal to stop executing the target action.

[0007] According to another aspect of an embodiment of the present application, a gimbal control device is also provided, including: a first control module, used to respond to a gimbal control start instruction, control the gimbal to perform a target action corresponding to the gimbal control start instruction, and obtain the delay and frame rate corresponding to the gimbal, wherein the gimbal control start instruction is sent by a target terminal, the delay represents the transmission delay between the target terminal and the gimbal, and the frame rate represents the frame rate used by the gimbal to transmit data; a determination module, used to determine the target frame number based on the number of flight frames, the delay and the frame rate, wherein the number of flight frames represents the number of frames that the gimbal has sent to the target terminal but has not yet determined the number of frames received by the target terminal; a clearing module, used to clear non-key frame data corresponding to the target frame number in the first cache data, wherein the first cache data represents the data to be sent by the gimbal to the target terminal, and the first cache data includes the non-key frame data; a second control module, used to respond to a gimbal control end instruction, control the gimbal to stop performing the target action.

[0008] In an exemplary embodiment, the device is used to determine the target number of frames based on the number of flight frames, the delay and the frame rate in the following manner: obtaining the number of flight frames; determining the number of delayed frames based on the ratio between the delay and the frame rate; determining the target number of frames based on the value relationship between the number of flight frames and the number of delayed frames.

[0009] In an exemplary embodiment, the device is used to determine the target number of frames according to the value relationship between the number of flight frames and the number of delayed frames in the following manner: when the number of flight frames is greater than or equal to the number of delayed frames, the target number of frames is determined as a first number; when the number of flight frames is less than the number of delayed frames, the target number of frames is determined as a second number, wherein the second number is less than the first number.

[0010] In an exemplary embodiment, the device is also used to: after determining the target number of frames based on the number of flight frames, the delay and the frame rate, generate an auxiliary control frame based on the target number of frames, wherein the auxiliary control frame is used to mark the non-critical frame data; send the auxiliary control frame to the target terminal so that the target terminal clears the non-critical frame data in the second cache data, wherein the second cache data represents the data that the target terminal has received from the gimbal, and the second cache data includes the non-critical frame data.

[0011] In an exemplary embodiment, the device is also used to: in response to the gimbal control start instruction, control the gimbal to perform a target action corresponding to the gimbal control start instruction, and obtain the delay and frame rate corresponding to the gimbal; determine the target time interval based on the frame rate and a predetermined key frame interval; loop the following steps in the target time interval: determine the target number of frames based on the number of flight frames, the delay and the frame rate; clear the non-key frame data in the first cache data; in response to the gimbal control end instruction, control the gimbal to stop performing the target action.

[0012] In an exemplary embodiment, the device is used to determine the target time interval based on the frame rate and a predetermined key frame interval in the following manner: obtaining a predetermined sample time interval; determining the key frame time interval based on the key frame interval and the frame rate; and determining any time interval in the sample time interval that is less than or equal to the key frame time interval as the target time interval.

[0013] In an exemplary embodiment, the device is also used to: in response to the gimbal control start instruction, obtain the delay and frame rate corresponding to the gimbal; reduce the frame rate corresponding to the gimbal to the target frame rate; in response to the gimbal control end instruction, control the gimbal to stop executing the target action, and restore the target frame rate corresponding to the gimbal to the frame rate.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned pan-tilt control method when running.

[0015] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above pan / tilt control method.

[0016] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the gimbal control method through the computer program.

[0017] In the embodiment of the present application, the pan-tilt control start command is sent through the target terminal, the pan-tilt responds and executes the command, and the delay and frame rate are measured at the same time. Then, the target number of frames to be processed is determined based on the number of flight frames that have been sent but not confirmed to be received, the delay and frame rate. The target number of frames is based on the network status and the real-time feedback of the pan-tilt operation to ensure that the video picture can reflect the physical action of the pan-tilt in a timely manner. Then, the non-key frame data corresponding to the target number of frames in the first cache data is cleared to reduce the amount of data transmission, reduce the network burden, and avoid video picture delays caused by data backlogs. Finally, when the pan-tilt receives the control end command, it stops executing the target action to ensure the accuracy of the pan-tilt operation and the synchronization of the video picture, thereby achieving the purpose of reducing the video picture delay and synchronizing the pan-tilt operation, thereby achieving the technical effect of synchronizing the pan-tilt with the video picture, and then solving the technical problem that the pan-tilt and the video picture are not synchronized enough and the video picture is delayed in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a schematic diagram of an application environment of an optional PTZ control method according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of an optional pan / tilt control method according to an embodiment of the present application;

[0021] Figure 3 is a network topology block diagram of an optional PTZ control method according to an embodiment of the present application;

[0022] Figure 4 is a module block diagram of an optional pan / tilt control method according to an embodiment of the present application;

[0023] Figure 5 is a flow chart of an optional pan / tilt control method according to an embodiment of the present application;

[0024] Figure 6 is a schematic structural diagram of an optional pan / tilt control device according to an embodiment of the present application;

[0025] Figure 7 is a schematic structural diagram of an optional pan / tilt control product according to an embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] According to one aspect of an embodiment of the present application, a pan / tilt control method is provided. Optionally, in this embodiment, the pan / tilt control method can be applied to Figure 1 In the hardware environment composed of the server 101 and the terminal device 103 shown in FIG. Figure 1As shown, the server 101 is connected to the terminal device 103 via a network, and can be used to provide services for the terminal device or an application 107 installed on the terminal device. The application can be a video application, an instant messaging application, a browser application, an educational application, a game application, etc. A database 105 may be set up on the server or independently of the server to provide data storage services for the server 101, for example, a game data storage server. The above-mentioned network may include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that realize wireless communication. The terminal device 103 may be a terminal configured with an application, and may include but is not limited to at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop computer, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, a virtual reality (Virtual Reality, VR for short) terminal, an augmented reality (Augmented Reality, AR for short) terminal, a mixed reality (Mixed Reality, MR for short) terminal and other computer devices. The above-mentioned server may be a single server, or a server cluster consisting of multiple servers, or a cloud server.

[0030] Combination Figure 1 As shown, the above-mentioned pan-tilt control method can be executed by an electronic device, which can be a terminal device or a server. The above-mentioned pan-tilt control method can be implemented by the terminal device or the server respectively, or by the terminal device and the server together.

[0031] The above is only an example and is not specifically limited in this embodiment.

[0032] Optionally, as an optional implementation, as Figure 2 As shown, the above-mentioned PTZ control method includes:

[0033] S202, in response to the pan / tilt control start instruction, controlling the pan / tilt to execute a target action corresponding to the pan / tilt control start instruction, and obtaining a delay and a frame rate corresponding to the pan / tilt, wherein the pan / tilt control start instruction is sent by the target terminal, the delay indicates a transmission delay between the target terminal and the pan / tilt, and the frame rate indicates a frame rate used by the pan / tilt to transmit data;

[0034] S204, determining a target frame number based on the number of flight frames, latency, and frame rate, wherein the number of flight frames represents the number of frames that the PTZ has sent to the target terminal, and the number of frames that the target terminal has received is not determined;

[0035] S206, clearing non-key frame data corresponding to the target frame number in the first cache data, wherein the first cache data represents data to be sent by the PTZ to the target terminal, and the first cache data includes non-key frame data;

[0036] S208, in response to the PTZ control end instruction, controlling the PTZ to stop executing the target action.

[0037] Optionally, in an embodiment of the present application, the above-mentioned pan-tilt control start instruction refers to an instruction sent by the target terminal to the pan-tilt, which is used to instruct the pan-tilt to start performing a specific action, including but not limited to rotation, zoom or other operations. The above-mentioned delay refers to the time required for data transmission between the target terminal and the pan-tilt, including but not limited to network delay, processing delay, etc., which can be determined by measuring the round-trip time of the data packet. The above-mentioned frame rate refers to the number of frames transmitted per second when the pan-tilt transmits video data, including but not limited to 24 frames / second, 30 frames / second, etc. This parameter affects the smoothness and real-time performance of the video.

[0038] It should be noted that the specific content and form of the pan-tilt control start instruction may vary depending on the application scenario and the target terminal, and this application does not limit this. For example, in a security monitoring scenario, the instruction may include the precise rotation angle and speed of the pan-tilt; in traffic monitoring, the instruction may involve the pan-tilt tracking the action of a specific vehicle. The measurement method of delay can also be diversified. It can be based on the analysis of the network protocol or determined by actually sending a test data packet. This application does not limit this. The frame rate setting can also be adjusted according to the performance of the monitoring equipment and the network bandwidth. Some devices may support transmission of up to 60 frames per second, while others may only support lower frame rates. This application does not limit this.

[0039] Optionally, in an embodiment of the present application, the above-mentioned number of flight frames refers to the number of frames that the gimbal has sent out but has not yet received confirmation of receipt from the target terminal. This number reflects the status and efficiency of the current network transmission.

[0040] It should be noted that the determination of the number of flight frames does not only depend on the amount of data sent by the gimbal, but may also be affected by network conditions and the processing capabilities of the target terminal, which is not limited in this application. For example, when the network bandwidth is limited, even if the gimbal sends a large amount of data, the number of flight frames may remain at a low level due to slow data transmission. When determining the target number of frames, in addition to considering the number of flight frames, latency and frame rate, factors such as the current position of the gimbal and the display delay of the target terminal may also need to be considered, which is not limited in this application.

[0041] Optionally, in the embodiment of the present application, the first cache data refers to the video data cache that the PTZ is ready to send to the target terminal, including but not limited to video frames to be encoded, encoded but not sent video frames, etc. The non-key frame data refers to other frames (such as P frames and B frames) in the video stream except key frames (such as I frames). These frames rely on key frames to reconstruct images and do not contain complete image information.

[0042] It should be noted that the strategy for clearing non-key frame data in the first cache data can be adjusted according to the actual video encoding format and network conditions, and this application does not limit this. For example, in H.264 encoding, it may be necessary to retain I frames and some P frames to ensure the continuity of the video; while in H.265 encoding, due to its higher compression efficiency, it may allow more non-key frame data to be cleared. The operation of clearing non-key frame data can also be progressive, and dynamically adjusted according to real-time changes in network conditions, and this application does not limit this.

[0043] Optionally, in an embodiment of the present application, the sending of the above-mentioned pan-tilt control end instruction can be manually triggered by an operator, or automatically triggered by the system according to preset conditions, and the present application does not limit this. For example, after detecting that a specific security event has ended, the system may automatically send a control end instruction; or after the network condition returns to normal levels, the system may also automatically stop the action of the pan-tilt. The behavior of the pan-tilt after stopping executing the target action can also be diverse. It can return to the initial position, or remain in the last position waiting for the next instruction, and the present application does not limit this.

[0044] Exemplarily, the specific implementation process of the solution and method for synchronizing the pan / tilt with the video screen is as follows:

[0045] S1, when the target terminal user needs to control the gimbal to perform a specific action, the target terminal sends a gimbal control start command to the gimbal. This command may include parameters such as the direction and speed of the gimbal's rotation.

[0046] S2, the gimbal responds to the gimbal control start instruction and executes the target action corresponding to the instruction. For example, if the instruction requires the gimbal to rotate 90 degrees clockwise, the gimbal executes this rotation action.

[0047] S3, while the PTZ is executing the action, the system obtains the latency and frame rate corresponding to the PTZ. The latency may be determined by measuring the time interval from the command being issued to the PTZ's response, and the frame rate is the number of frames when the PTZ transmits video data, such as 30 frames per second.

[0048] S4, the system determines the target frame number based on the number of flight frames, latency and frame rate. The number of flight frames refers to the number of frames that the gimbal has sent but the target terminal has not yet confirmed receipt. This number may be determined by monitoring network traffic and confirmation responses.

[0049] S5, according to the determined target frame number, the system clears the non-key frame data corresponding to the target frame number in the first cache data. The first cache data may include video frames waiting to be sent, and the non-key frame data may refer to video frames (such as P frames and B frames) that are not key frames (such as I frames).

[0050] S6, if the network condition is good, the number of flight frames may be small. At this time, the system may only need to clear a small amount of non-key frame data to maintain the smoothness and synchronization of the video.

[0051] S7, if the network condition is not good, the number of flight frames may be large. At this time, the system may need to clear more non-key frame data to reduce network congestion and improve the synchronization between the video image and the gimbal operation.

[0052] S8, when the system clears non-key frame data, it needs to ensure that the remaining key frame data can support the correct decoding and playback of the video to avoid jumping or freezing of the video.

[0053] S9, when the target terminal user completes the control operation of the pan / tilt, the target terminal sends a pan / tilt control end instruction to the pan / tilt. This instruction may indicate that the user no longer needs the pan / tilt to perform a specific action.

[0054] S10, the PTZ stops executing the target action in response to the PTZ control end instruction, and returns to the original state or waits for the next instruction. For example, if the PTZ is executing a rotation action, the control end instruction will stop the PTZ from rotating.

[0055] S11, after the gimbal stops performing actions, the system may need to re-evaluate the network status and cached data to determine whether it is necessary to further adjust the frame rate or clear the cached data to maintain the stability and real-time nature of the video stream.

[0056] S12, the system may also need to monitor the status of the PTZ and the network status during the whole process, so as to adjust the strategy in time when abnormal situations occur. For example, when the PTZ fails or the network is interrupted, the system may need to send an error message to the target terminal and take corresponding recovery measures.

[0057] Through the above steps, the solution and method for synchronizing the pan / tilt with the video image can effectively reduce the video image delay, improve the real-time responsiveness of the pan / tilt operation, and thus enhance the user experience.

[0058] In an exemplary embodiment, taking the application scenario of urban traffic monitoring as an example, the urban traffic management department needs to monitor the traffic conditions at busy intersections in real time. The PTZ is installed on the monitoring pole at the intersection, and the target terminal is the monitoring screen and console used by the operator of the traffic management center, including but not limited to the following:

[0059] S1, the operator in the monitoring center observes traffic congestion at a certain intersection and decides to adjust the PTZ angle to better observe the situation. The operator sends a PTZ control start command to the PTZ through the console, which includes the requirement to rotate to a specific angle and speed.

[0060] S2, after receiving the command, the PTZ starts to perform the corresponding rotation action, and the system starts to measure the latency and frame rate. The latency is the time it takes for the command to be sent from the console to the PTZ and get a response, and the frame rate is the number of frames of video data transmitted by the PTZ, such as 25 frames per second.

[0061] S3, during the pan / tilt rotation, the system monitors the number of flight frames, that is, the number of video frames that the pan / tilt has sent but the monitoring center has not yet confirmed receipt. This number reflects the efficiency and stability of network transmission.

[0062] S4, based on the number of flight frames, latency and frame rate, the system determines the target number of frames to be cleared. If the number of flight frames is large, it indicates that there is a delay in network transmission. The system will clear the non-key frame data in the first cache data to reduce the network burden and ensure the real-time nature of the video.

[0063] S5, when clearing non-key frame data, the system ensures that key frames (such as I frames) are retained to ensure the continuity and decodability of the video.

[0064] S6, after observing that the traffic conditions have improved, the operator sends a PTZ control end command, and the PTZ stops rotating and returns to the preset position.

[0065] S7, the system continuously monitors the PTZ status and network conditions throughout the process to ensure that the video image is synchronized with the PTZ operation and adjusts the strategy when necessary.

[0066] In another exemplary embodiment, taking the application scenario of internal monitoring of a bank as an example, the bank needs to ensure the safety of its business hall and vault, including but not limited to the following:

[0067] S1, bank security personnel monitor the internal situation of the bank through the target terminal in the monitoring room. When detecting abnormal activities in the vault area, the security personnel decide to adjust the PTZ for close observation. The security personnel send the PTZ control start command to the PTZ through the target terminal, which includes the requirements of rotating to a specific position and adjusting the focal length.

[0068] S2, after receiving the command, the PTZ starts to perform the corresponding rotation and zooming actions, and the system starts to measure the latency and frame rate. The latency is the time it takes for the command to be sent from the monitoring room to the PTZ and get a response, and the frame rate is the number of frames of video data transmitted by the PTZ, such as 30 frames per second.

[0069] S3, during the pan / tilt adjustment and rotation process, the system monitors the number of flight frames, that is, the number of video frames sent by the pan / tilt but not yet confirmed by the monitoring room. This number reflects the efficiency and stability of network transmission.

[0070] S4, based on the number of flight frames, latency and frame rate, the system determines the target number of frames to be cleared. If the number of flight frames is large, it indicates that there is a delay in network transmission. The system will clear the non-key frame data in the first cache data to reduce the network burden and ensure the real-time nature of the video.

[0071] S5, when clearing non-key frame data, the system ensures that key frames (such as I frames) are retained to ensure the continuity and decodability of the video.

[0072] S6, after confirming that the abnormal activity has been handled, the security personnel sends a PTZ control end command, and the PTZ stops rotating and zooming and returns to the preset position.

[0073] S7, the system continuously monitors the PTZ status and network conditions during the entire process to ensure that the video image is synchronized with the PTZ operation, and adjusts the strategy when necessary, such as reducing the frame rate to ensure real-time performance when the network conditions are poor.

[0074] Through this embodiment, the pan-tilt control start instruction is sent through the target terminal, the pan-tilt responds and executes the instruction, and the delay and frame rate are measured at the same time. Then, the target number of frames to be processed is determined based on the number of flight frames that have been sent but not confirmed to be received, the delay and frame rate. The target number of frames is based on the network status and the real-time feedback of the pan-tilt operation to ensure that the video picture can reflect the physical action of the pan-tilt in time. Then, the non-key frame data corresponding to the target number of frames in the first cache data is cleared to reduce the amount of data transmission, reduce the network burden, and avoid video picture delays caused by data backlogs. Finally, when the pan-tilt receives the control end instruction, it stops executing the target action to ensure the accuracy of the pan-tilt operation and the synchronization of the video picture, thereby achieving the purpose of reducing the video picture delay and synchronizing the pan-tilt operation, thereby achieving the technical effect of synchronizing the pan-tilt with the video picture, and then solving the technical problem that the pan-tilt and the video picture are not synchronized enough and the video picture is delayed in the related technology.

[0075] As an optional solution, the target number of frames is determined based on the number of flight frames, latency, and frame rate, including:

[0076] Get the number of flight frames;

[0077] Determine the number of delayed frames according to the ratio between the delay and the frame rate;

[0078] The target frame number is determined based on the relationship between the number of flying frames and the number of delayed frames.

[0079] Optionally, in an embodiment of the present application, the above-mentioned number of delayed frames refers to the number of frames calculated based on the ratio between the delay and the frame rate, including but not limited to if the delay is 200 milliseconds and the frame rate is 25 frames / second, then the number of delayed frames is 5 frames.

[0080] It should be noted that the calculation method for determining the number of delayed frames may vary depending on the encoding method and video format of the PTZ, and this application does not limit this. For example, the calculation of the number of delayed frames can be based on a video stream with a fixed frame rate, or it can be dynamically adjusted based on a video stream with a variable frame rate, or the calculation method of the number of delayed frames can be adjusted based on the complexity and movement speed of the video content, including but not limited to fixed frame rate calculation based on H.264 encoding, variable frame rate calculation based on H.265 encoding, dynamic frame rate adjustment based on video content analysis, etc.

[0081] Optionally, in an embodiment of the present application, the above-mentioned target number of frames refers to the number of frames that need to be processed based on the value relationship between the number of flight frames and the number of delayed frames, including but not limited to the number of non-key frames that need to be cleared in order to keep the video screen synchronized with the gimbal operation.

[0082] It should be noted that the logic for determining the target number of frames can be adjusted according to actual application requirements and system performance, and this application does not limit this. For example, the determination of the target number of frames can be based on a simple comparison of the number of flight frames and the number of delayed frames, or it can be intelligently adjusted in combination with the current working status of the PTZ and the network quality, or optimized according to the user's operating habits and the special requirements of the monitoring scene, including but not limited to a simple threshold comparison based on the number of flight frames being greater than the number of delayed frames, a dynamic adjustment strategy based on network bandwidth fluctuations, and a personalized target number of frames based on user operating habits.

[0083] In addition, the method of obtaining the number of flight frames may vary according to different monitoring systems and network environments, and this application does not limit this. For example, the number of flight frames can be obtained by monitoring network traffic, or can be determined by the communication protocol between the PTZ and the target terminal, or can be tracked and calculated using special monitoring software, including but not limited to traffic analysis based on TCP / IP protocol, data packet counting based on UDP, query based on PTZ API, etc.

[0084] For example, in a video surveillance system, the implementation process can be described in detail as follows:

[0085] The system first needs to obtain the number of flying frames. This can be achieved by monitoring the video frames that the PTZ has sent to the target terminal but has not yet received a confirmation of receipt. For example, the system can use a network traffic analysis tool to monitor and count the number of frames in the video stream being transmitted, or use the built-in sending log of the PTZ to track the frames that have been sent but not confirmed.

[0086] Next, the system determines the number of delayed frames based on the ratio between latency and frame rate. Latency can be obtained by measuring the time interval from the PTZ sending data to the target terminal receiving the data. The frame rate is the number of frames when the PTZ transmits video data. For example, if the PTZ transmits video at a rate of 30 frames per second and the latency is 100 milliseconds, the number of delayed frames is 3 frames.

[0087] After determining the number of delayed frames, the system needs to determine the target number of frames based on the relationship between the number of flight frames and the number of delayed frames. For example, if the number of flight frames is greater than the number of delayed frames, the system may need to clear more non-key frames to reduce network congestion and improve the synchronization between the video screen and the PTZ operation. If the number of flight frames is less than the number of delayed frames, the system may need to reduce the number of non-key frames to be cleared to maintain video quality.

[0088] The system may need to consider multiple situations when determining the target number of frames. For example, if the network bandwidth suddenly increases, the system may not need to clear too many non-critical frames because the network can handle more data transmission. On the contrary, if the network bandwidth is limited, the system may need to clear more non-critical frames to avoid video delay.

[0089] The system also needs to consider the current working state of the gimbal. For example, if the gimbal is performing a fast-moving operation, more frames may be needed to maintain the smoothness of the picture. At this time, the system will adjust the target frame number to adapt to this change.

[0090] In addition, the system may also need to adjust the target frame number based on the complexity of the video content. For example, in a surveillance scene, if there are fast-moving objects in the picture, the system may need to retain more frames to ensure the continuity of the picture. In static or slow-moving scenes, the system can clear more non-critical frames because the human eye is less sensitive to the loss of frames.

[0091] When clearing non-key frames, the system also needs to ensure that key frames (such as I frames) are retained to ensure the decodability of the video. For example, in a GOP (Group of Pictures), the system may need to retain I frames and some P frames to ensure that the video stream can be correctly decoded at any time.

[0092] Finally, after determining the target frame number and clearing non-critical frames, the system needs to monitor the real-time performance of the video stream to ensure that the video image is synchronized with the PTZ operation. If the system detects video image delay, it may need to further adjust the target frame number or take other measures, such as adjusting the PTZ frame rate or optimizing the network transmission strategy.

[0093] Through the above steps, the system can effectively manage the video stream, ensure the real-time synchronization of the PTZ operation and the video screen, and improve the efficiency of the monitoring system and the user experience.

[0094] As an optional solution, the target frame number is determined according to the value relationship between the number of flight frames and the number of delayed frames, including:

[0095] When the number of flying frames is greater than or equal to the number of delayed frames, determining the target number of frames as the first number;

[0096] When the number of flying frames is less than the number of delayed frames, the target number of frames is determined to be a second number, wherein the second number is less than the first number.

[0097] Optionally, in an embodiment of the present application, the above-mentioned first number refers to the target number of frames determined when the number of flight frames is greater than or equal to the number of delayed frames. This number may include all flight frames or a value slightly smaller than the number of flight frames calculated based on a specific algorithm, including but not limited to retaining all flight frames to maintain video continuity when the network conditions are good.

[0098] Optionally, in an embodiment of the present application, the above-mentioned second number refers to the target number of frames determined when the number of flight frames is less than the number of delayed frames, and this number is reduced to reduce network congestion and improve the synchronization of video images and gimbal operations, including but not limited to reducing the number of frames when the network bandwidth is limited, for example, reducing it to half of the number of flight frames when the network is congested to quickly synchronize the video stream.

[0099] It should be noted that the specific method for determining the first number and the second number may vary according to different application scenarios and system requirements, and this application does not limit this. For example, in a security monitoring system, when the network bandwidth is limited, the first number may include all non-key frames to ensure the integrity of the picture. When the network bandwidth is relatively abundant, the first number may only include some non-key frames to ensure a smooth video call experience.

[0100] The determination of the first number and the second number may also be affected by the characteristics of the video content, which is not limited in this application. For example, in a scene of monitoring a high-speed moving object, more frames may be required to maintain the continuity of the picture, so the first number may be relatively large; while when monitoring a static scene, the first number may be relatively small because the human eye is not very sensitive to the loss of frames. Similarly, the determination of the second number may also be adjusted according to the motion complexity of the video content to maintain picture quality and synchronization.

[0101] Furthermore, the determination of the first number and the second number may also involve the specific requirements of the PTZ operation, which is not limited in this application. For example, in monitoring operations that require rapid response, such as tracking emergencies, the first number may be set to a smaller value to quickly update the image; while in conventional monitoring, the first number may be larger to maintain the stability of the image. The determination of the second number may also be adjusted according to the urgency of the PTZ operation to balance the image quality and response speed.

[0102] For example, in a video surveillance system, the specific implementation process may be as follows:

[0103] The system first monitors and records the number of flying frames in the video stream sent by the PTZ to the target terminal. This can be achieved by analyzing network traffic or the internal counter of the PTZ, and at the same time measuring the transmission delay between the PTZ and the target terminal, which may be done by sending a test data packet and calculating its round-trip time.

[0104] The system determines the number of delayed frames based on the gimbal's frame rate, which can be calculated by dividing the delay time by the time interval between each frame (the inverse of the frame rate). The number of flight frames and the number of delayed frames are compared to determine the target number of frames.

[0105] When the number of flying frames is greater than or equal to the number of delayed frames, the system determines the target number of frames as the first number. For example, if the number of flying frames is 100 and the number of delayed frames is 80, then the first number is all non-key frames, such as 120. In this case, the system may need to consider the network bandwidth, the encoding capability of the PTZ, and the decoding capability of the target terminal to decide whether the video stream needs to be optimized.

[0106] After determining the first number, if the network bandwidth is sufficient, the system may keep the video stream unchanged because there is enough bandwidth to process these frames. If the network bandwidth is tight, the system may choose to clear some non-critical frames to reduce network congestion and maintain the smoothness of the video stream. If the processing power of the target terminal is limited, the system may reduce the resolution or frame rate of the video to adapt to the decoding capability of the terminal.

[0107] When the number of flying frames is less than the number of delayed frames, the system determines the target number of frames as the second number. For example, if the number of flying frames is 60 and the number of delayed frames is 80, then the second number is part of the non-critical frames, for example, 50. In this case, the system may need to take measures to reduce the delay or increase the number of flying frames.

[0108] After determining the second number, if the network conditions are not good, the system may increase the encoding efficiency of the PTZ, such as by adjusting the encoding parameters to reduce the amount of data per frame. If the target terminal has a strong cache capacity, the system may temporarily store more frames until the network conditions improve before sending them. If the PTZ moves quickly, the system may reduce the rotation speed of the PTZ to reduce frame loss caused by fast movement.

[0109] After determining the first and second quantities, the system needs to monitor the network status and PTZ status in real time and dynamically adjust the target frame quantity based on this information. For example, if the network condition suddenly improves, the system may increase the number of frames sent; if the PTZ detects abnormal activity, the system may increase the frame rate to capture more details. The system needs to ensure the continuity and real-time nature of the video stream throughout the process, while taking into account the accuracy of the PTZ operation and the display effect of the target terminal. This may involve real-time analysis of the video stream and rapid response to PTZ control instructions.

[0110] Through the above steps, the system can flexibly adjust the video stream processing strategy under different network conditions and monitoring requirements to ensure the efficient operation and picture quality of the video surveillance system.

[0111] As an optional solution, after determining the target number of frames based on the number of flight frames, latency, and frame rate, the method further includes:

[0112] Generate an auxiliary control frame based on the target frame number, wherein the auxiliary control frame is used to mark non-key frame data;

[0113] An auxiliary control frame is sent to the target terminal to enable the target terminal to clear non-key frame data in the second cache data, wherein the second cache data represents data that the target terminal has received from the PTZ, and the second cache data includes the non-key frame data.

[0114] Optionally, in an embodiment of the present application, the auxiliary control frame refers to a special data frame used to mark non-key frame data in a video stream, including but not limited to P frames, B frames, etc. in the video encoding process. These frames rely on key frames (such as I frames) to reconstruct images, but do not contain complete image information themselves.

[0115] It should be noted that the generation of auxiliary control frames can be adjusted according to different coding standards and video stream characteristics, and this application does not limit this. For example, the auxiliary control frame may be designed according to different video coding formats (such as H.264, H.265) to adapt to different frame types and coding efficiencies; it may also adjust the marking strategy according to the complexity of the video content (such as motion scenes, static scenes) to optimize video quality and transmission efficiency; in addition, the generation of auxiliary control frames may also consider network conditions (such as bandwidth fluctuations, delays) to dynamically adjust to adapt to different transmission environments. The timing and frequency of sending auxiliary control frames can also be optimized according to the actual network conditions and system performance, and this application does not limit this. For example, the auxiliary control frame may be sent more frequently when network congestion is detected to quickly respond and reduce non-critical frames in the second cache data; it may also reduce the sending frequency when the network condition is good to reduce control overhead; in addition, the sending of auxiliary control frames may also be adjusted in combination with the feedback information of the target terminal (such as cache status, decoding capability) to achieve more refined flow control.

[0116] Optionally, in an embodiment of the present application, the above-mentioned second cache data refers to data that the target terminal has received from the pan-tilt head but has not yet been decoded, including but not limited to video frames stored in the target terminal memory or hard disk, which data may include key frames and non-key frames, where non-key frame data is video frames that rely on key frames to be correctly decoded.

[0117] It should be noted that the clearing strategy for non-key frame data in the second cache data can be customized according to different application requirements and user experience requirements, and this application does not limit this. For example, the clearing strategy may prioritize clearing non-key frames that have less visual impact while ensuring video smoothness; it may also dynamically adjust the clearing strategy during user interaction (such as fast forward and rewind) to quickly respond to user operations; in addition, the clearing strategy may also combine the importance of the video content (such as key events in surveillance videos) to determine which frames need to be retained to ensure that key information is not lost.

[0118] For example, in a video surveillance system, the specific implementation process is as follows: When the surveillance system starts running, the PTZ device starts sending a video stream to the target terminal, and the video stream contains key frame and non-key frame data. The PTZ device generates auxiliary control frames based on the target frame number. These auxiliary control frames are used to mark non-key frame data in the video stream so that the target terminal can identify and process it accordingly. The generation of auxiliary control frames may be diversified according to different video coding standards. For example, in H.264 encoding, the auxiliary control frame may contain markings for P frames and B frames; while in H.265 encoding, it may contain markings for more complex frame dependencies. The PTZ device sends auxiliary control frames to the target terminal. The sending timing may be adjusted according to the network conditions and the working status of the PTZ device. For example, when the network bandwidth is sufficient, the auxiliary control frame may be sent in real time together with the video frame; when the bandwidth is tight, the auxiliary control frame may be sent first to quickly adjust the cache strategy of the target terminal.

[0119] After receiving the auxiliary control frame, the target terminal clears the non-key frame data in the second cache data according to the mark in the auxiliary control frame. This step may involve fast retrieval and deletion of data in the cache to ensure that only key frames and necessary non-key frames are retained in the cache.

[0120] When clearing non-key frame data in the second cache data, the target terminal may adopt different strategies according to different application scenarios. For example, in a security monitoring scenario, the latest key frame may be kept first to ensure the continuity of the picture; while in a video conference, more emphasis may be placed on real-time performance, and more non-key frames may be cleared to reduce delay.

[0121] After clearing non-key frame data, the target terminal may need to resynchronize the video stream. This may involve requesting the PTZ device to send the latest key frame, or adjusting the decoder state to match the new video stream state. After sending the auxiliary control frame, the PTZ device may need to monitor the feedback information from the target terminal to evaluate the effect of the clearing operation. This may include receiving a report on the cache status from the target terminal, or monitoring the transmission efficiency of the video stream through a network traffic analysis tool.

[0122] In special cases, such as when the processing power of the target terminal is limited or the network conditions are extremely poor, the PTZ device may need to further adjust the encoding parameters of the video stream, such as reducing the frame rate or resolution, to adapt to the capabilities of the target terminal. The communication between the PTZ device and the target terminal may use a variety of protocols and interfaces. For example, RTSP, RTMP or other streaming protocols can be used to transmit video streams and auxiliary control frames while ensuring the compatibility and scalability of the protocol.

[0123] Through the above steps, the video surveillance system can effectively manage the frame data in the video stream, ensure the transmission and display of key information, and optimize the use of network resources and improve the user experience.

[0124] As an optional solution, the above method also includes:

[0125] In response to the gimbal control start instruction, control the gimbal to perform a target action corresponding to the gimbal control start instruction, and obtain a delay and a frame rate corresponding to the gimbal;

[0126] determining a target time interval based on the frame rate and a predetermined key frame interval;

[0127] The following steps are cyclically performed in the target time interval: determining the target frame number based on the number of flying frames, the delay and the frame rate; clearing the non-key frame data in the first buffer data;

[0128] In response to the gimbal control end instruction, the gimbal is controlled to stop executing the target action.

[0129] Optionally, in an embodiment of the present application, the above-mentioned key frame interval refers to the interval between key frames in video encoding, which is an important parameter for video encoding efficiency and compression rate, including but not limited to the interval between I frames in H.264 encoding. For example, the key frame interval may be one I frame every 30 frames.

[0130] It should be noted that the method for determining the target time interval based on the frame rate and key frame interval can be adjusted according to the dynamic changes of the video content and the different encoding strategies, and this application does not limit this. For example, for video scenes containing fast motion, a shorter key frame interval may be required to maintain the smoothness of the video; while for static scenes, the interval can be increased to improve the encoding efficiency. The determination of the target time interval may also be affected by the network bandwidth and terminal processing capabilities to ensure the stable transmission and playback of the video stream.

[0131] Optionally, in an embodiment of the present application, the above-mentioned target time interval refers to a time period determined based on the frame rate and the key frame interval, during which specific processing steps are performed, including but not limited to a period of 1 second or 2 seconds. For example, at a frame rate of 25 frames per second, if the key frame interval is 25 frames, the target time interval is 1 second.

[0132] It should be noted that the steps cyclically executed in the target time interval can be dynamically adjusted according to the network conditions and system performance, and this application does not limit this. For example, when the network is congested, it may be necessary to clear the non-key frame data in the first cache data more frequently to reduce latency; when the network conditions are good, the clearing frequency can be reduced to maintain video quality. The strategy for clearing non-key frame data may also be adjusted according to the importance of the video content and the specific needs of the user. For example, more frame data may need to be retained when monitoring key events.

[0133] For example, in an intelligent monitoring system, the specific implementation process is as follows:

[0134] S1, the operator of the monitoring center sends the PTZ control start command to the PTZ through the console. The command contains the details of the action that the PTZ needs to perform, such as rotation angle, movement speed, etc. After receiving the control start command, the PTZ starts to perform the corresponding target action, such as rotating to a specific position or changing the focal length.

[0135] S2, the system starts to obtain the latency and frame rate corresponding to the PTZ. Latency refers to the time delay from the PTZ sending data to the monitoring center receiving data, and frame rate refers to the number of frames of the video transmitted by the PTZ, such as 25 frames per second. The system determines the target time interval based on the frame rate and the predetermined key frame interval. For example, if the key frame interval is set to an I frame every 5 seconds, the target time interval is 5 seconds.

[0136] S3, the system enters a loop and performs the following steps at each target time interval:

[0137] S3-1, determine the target frame number based on the number of flight frames, latency and frame rate. The number of flight frames refers to the number of frames that the PTZ has sent but the monitoring center has not yet confirmed receipt.

[0138] S3-2, if the number of flight frames is greater than the number of delayed frames, the system will clear the non-key frame data in the first cache data to reduce network congestion and improve video synchronization.

[0139] S3-3, if the number of flight frames is less than the number of delayed frames, the system may need to adjust the gimbal's transmission strategy, such as increasing the frame rate or optimizing the encoding settings.

[0140] S3-4, the system monitors the network status and PTZ status to dynamically adjust the target frame number and clearing strategy to ensure the stability and real-time performance of the video stream.

[0141] S4, when the operator completes the control operation of the PTZ, a PTZ control end instruction is sent to the PTZ. The PTZ responds to the PTZ control end instruction, stops executing the target action, and returns to the standby state or executes the next instruction.

[0142] In special circumstances, such as network failure or mechanical failure of the PTZ, the system may need to execute error handling procedures, including retrying transmission, switching to an alternate network, or prompting the operator for manual intervention. The system may need to record log information throughout the process, including PTZ control instructions, latency and frame rate data, target frame number, and cleared non-critical frame data, etc., to facilitate troubleshooting and performance optimization. The system may also need to be integrated with other monitoring systems, such as alarm systems, access control systems, etc., to achieve more complex linkage control. For example, when abnormal behavior is detected, the PTZ automatically turns to the scene of the incident, triggering an alarm and notifying security personnel.

[0143] Through the above steps, the intelligent monitoring system can achieve precise control of the PTZ, optimize the transmission and processing of video streams, and improve monitoring efficiency and response speed.

[0144] As an optional solution, a target time interval is determined based on the frame rate and a predetermined key frame interval, including:

[0145] Obtaining a preset sample time interval;

[0146] Determine a key frame time interval based on the key frame interval and the frame rate;

[0147] Any time interval among the sample time intervals that is less than or equal to the key frame time interval is determined as the target time interval.

[0148] Optionally, in an embodiment of the present application, the above-mentioned sample time interval refers to a fixed time unit preset in the system, which is used as a benchmark for operations in video stream processing, including but not limited to every second, every two seconds or every five seconds, etc. For example, the sample time interval can be every 3 seconds as a sample time interval.

[0149] It should be noted that the setting of the sample time interval can be adjusted according to different monitoring requirements and system performance, and this application does not limit this. For example, the sample time interval may be set according to the scene sensitivity of the video monitoring. For financial transaction monitoring that requires high real-time performance, the sample time interval may be set shorter, such as every second; while for general urban traffic monitoring, the sample time interval may be set every few minutes. In addition, the sample time interval may also be set according to storage capacity and data processing capabilities to balance system load and response speed.

[0150] Optionally, in an embodiment of the present application, the above-mentioned key frame interval refers to the difference in the number of frames between two key frames in the video encoding process. This interval determines the frequency of occurrence of key frames, including but not limited to an I frame appearing every 30 frames in H.264 encoding, or an I frame appearing every 45 frames in H.265 encoding, etc. For example, the key frame interval can be a key frame appearing every 60 frames in the video stream.

[0151] It should be noted that the determination of the key frame interval can be set in a variety of ways according to the characteristics of the video content and the requirements of coding efficiency, and this application does not limit this. For example, the key frame interval may be adjusted according to the motion changes of the video content. For sports events with intense movements, the key frame interval may be smaller, such as every 10 frames; while for conference room monitoring with little change, the key frame interval may be larger, such as every 30 frames. The key frame interval may also be set according to the network bandwidth and the decoding capability of the terminal device to ensure efficient transmission and playback of the video stream.

[0152] Optionally, in an embodiment of the present application, the above-mentioned key frame time interval refers to the length of time calculated based on the key frame interval and the frame rate of the video stream. This time interval determines the frequency of key frames appearing in actual time, including but not limited to a key frame appearing every 60 frames at a frame rate of 25 frames / second, and the key frame time interval is 2.4 seconds. For example, the key frame time interval may be a key frame appearing every 50 frames at a frame rate of 30 frames / second, and the calculated key frame time interval is 1.67 seconds.

[0153] It should be noted that the determination of the target time interval can be flexibly selected based on the matching of the sample time interval and the key frame time interval, and this application does not limit this. For example, the target time interval may be selected as one that is equal to or closest to the key frame time interval in the sample time interval to achieve the best synchronization effect. In some cases, if the sample time interval is generally larger than the key frame time interval, the smallest sample time interval may be selected as the target time interval to reduce latency. In other cases, if there are multiple sample time intervals that match the key frame time interval, the most appropriate one may be selected as the target time interval based on the current workload and performance indicators of the system.

[0154] For example, in a video processing system, the specific implementation process is as follows:

[0155] S1, when the system starts, the sample time interval is pre-set. This interval can be determined based on the system design requirements and application scenarios. For example, it can be set to every 1 second, every 2 seconds, or every 5 seconds as a sample time interval. The system identifies key frames in the video stream and records the intervals between key frames. The key frame interval can be fixed or dynamically changed according to the complexity of the video content. For example, in a fast-moving scene, the key frame interval may be smaller.

[0156] S2, the system determines the key frame time interval based on the key frame interval and the frame rate of the video stream. For example, if the key frame interval is one key frame every 30 frames and the frame rate is 25 frames per second, then the key frame time interval is 1.2 seconds. The system compares the sample time interval with the key frame time interval to determine the target time interval. The target time interval is any time interval in the sample time interval that is less than or equal to the key frame time interval. In different application scenarios, the determination of the sample time interval and the key frame time interval may be different. For example, in security monitoring, a shorter sample time interval may be required to respond to events quickly, while in video conferencing, the key frame interval may need to be adjusted according to the network conditions of the participants. The system performs data processing tasks according to the target time interval. This may include operations such as encoding, cache management, and data synchronization. For example, the system may clear non-key frame data in the cache within each target time interval to optimize network bandwidth usage.

[0157] S3, if the frame rate of the video stream changes, the system may need to recalculate the key frame time interval and adjust the sample time interval and the target time interval. For example, if the frame rate increases from 25 frames / second to 30 frames / second, the key frame time interval may be reduced accordingly. In the case of limited network bandwidth, the system may need to select a shorter sample time interval as the target time interval to reduce the amount of data transmission. For example, if the network bandwidth suddenly decreases, the system may adjust the target time interval from 5 seconds to 2 seconds. The system may also need to consider the decoding capabilities of the terminal device. If the decoding capabilities of the terminal device are limited, the system may need to select a shorter key frame interval to reduce the burden on the terminal device. When performing the above operations, the system may also need to interact with other system components, for example, working with the pan-tilt control system to ensure the synchronization of the video stream and the pan-tilt action.

[0158] Through the above steps, the video processing system can flexibly determine the sample time interval and the key frame time interval according to different application scenarios and system conditions, and perform corresponding data processing tasks accordingly to optimize the processing and transmission of the video stream.

[0159] As an optional solution, the above method also includes:

[0160] In response to the PTZ control start command, obtain the delay and frame rate corresponding to the PTZ;

[0161] Reduce the frame rate corresponding to the gimbal to the target frame rate;

[0162] In response to the pan / tilt control end instruction, the pan / tilt is controlled to stop executing the target action, and the target frame rate corresponding to the pan / tilt is restored to the frame rate.

[0163] Optionally, in an embodiment of the present application, the frame rate corresponding to the above-mentioned pan / tilt refers to the frame rate setting adopted by the pan / tilt device when transmitting video data, including but not limited to 24 frames / second, 30 frames / second or 60 frames / second, etc., which depends on the design and application requirements of the video surveillance system. For example, 30 frames / second may be used in security monitoring to obtain a smoother video effect.

[0164] Optionally, in an embodiment of the present application, the above-mentioned target frame rate refers to a frame rate value adjusted according to a specific application scenario or network condition. This value may be lower than or equal to the corresponding frame rate of the gimbal, including but not limited to reducing the frame rate from 60 frames / second to 30 frames / second to adapt to bandwidth limitations, or reducing the frame rate from 30 frames / second to 15 frames / second to reduce data transmission when network congestion is detected.

[0165] It should be noted that the operation of obtaining the latency and frame rate corresponding to the pan-tilt in response to the pan-tilt control start instruction may vary according to different communication protocols and pan-tilt models, and this application does not limit this. For example, the latency may be determined by analyzing the transmission time of the network data packet, and the frame rate may be the result of the pan-tilt automatic adjustment according to the current lighting conditions; or in a specific pan-tilt device, the latency and frame rate are manually set by the operator through the control panel. In addition, the process of obtaining these parameters may also involve the pan-tilt self-test program, which monitors and reports its status in real time. The decision to reduce the frame rate corresponding to the pan-tilt to the target frame rate can be adjusted according to a variety of factors, and this application does not limit this. For example, reducing the frame rate may be to adapt to bandwidth limitations and ensure stable transmission of video streams; it may also be to reduce the storage space requirements while maintaining video quality; or in a battery-powered mobile monitoring device, reducing the frame rate may be to extend the use time of the device. In addition, the specific operation of reducing the frame rate may include changing the encoding settings, adjusting the output settings of the camera, or dynamically adjusting in the video processing software.

[0166] Optionally, in an embodiment of the present application, the process of restoring the target frame rate corresponding to the above-mentioned gimbal to the frame rate refers to the operation of restoring the frame rate setting of the gimbal device to the original state or the starting state, which may occur after the gimbal control end instruction response, including but not limited to restoring the frame rate from the lowered target frame rate to the initial 30 frames / second, or restoring the frame rate to the default setting after the special monitoring task is completed.

[0167] It should be noted that the process of controlling the gimbal to stop executing the target action and restoring the target frame rate corresponding to the gimbal to the frame rate in response to the gimbal control end instruction can be implemented in multiple ways, and this application does not limit this. For example, stopping the gimbal action may be achieved by sending a specific stop instruction, and the restoration of the frame rate may involve restoring the preset value in the gimbal firmware; or in some systems, the stop instruction and frame rate adjustment are sent simultaneously through an integrated control signal. In addition, the frame rate restoration process may require the gimbal to perform self-calibration to ensure the consistency of the video output; or after the network conditions improve, the system may automatically adjust the frame rate back to the original setting to provide a smoother video experience.

[0168] For example, in an application scenario of a security monitoring system, the specific implementation process is as follows:

[0169] S1, when the operator of the monitoring center needs to remotely control the outdoor PTZ camera to adjust its position, the operator sends a PTZ control start command through the control interface of the monitoring system.

[0170] S2, after the PTZ receives the control start command, it starts to execute the preset action, such as rotating to a specific position or zooming. At the same time, the system responds to the PTZ control start command and obtains the latency and frame rate corresponding to the PTZ. The latency may be determined by measuring the time interval from the command issuance to the PTZ response, and the frame rate is the number of frames when the PTZ transmits video data, such as 30 frames per second.

[0171] S3, the system decides whether to adjust the frame rate to optimize video quality and reduce network congestion based on the real-time performance of the PTZ and the network status. If necessary, the system reduces the frame rate corresponding to the PTZ to the target frame rate. For example, if the current frame rate of the PTZ is 30 frames per second, but the network bandwidth is insufficient to support it, the system may reduce the frame rate to 20 frames per second.

[0172] S4, after reducing the frame rate, the system continues to monitor the video quality and network conditions to ensure that the adjusted frame rate can meet the monitoring needs. If the network conditions improve, the system may gradually increase the frame rate until it returns to the original frame rate or reaches a new optimal frame rate.

[0173] S5, after the operator completes the adjustment of the PTZ position, he sends a PTZ control end instruction. In response to the PTZ control end instruction, the system controls the PTZ to stop executing the target action. For example, if the PTZ is rotating, it will stop rotating and remain at the current position.

[0174] S6, after stopping the PTZ action, the system restores the target frame rate corresponding to the PTZ to the original frame rate. This step ensures that the video stream is restored to its original quality and smoothness. For example, if the frame rate was previously reduced from 30 frames / second to 20 frames / second, it will now be restored to 30 frames / second.

[0175] It should be noted that in some cases, the PTZ may need to perform some additional operations before stopping, such as autofocus or white balance adjustment, to ensure video quality. The system integrates these additional operations into the processing flow of the PTZ control end command. The system may also need to handle abnormal situations throughout the process, such as PTZ failure or network interruption. In this case, the system will start the error recovery process and try to reestablish the connection or switch to the backup PTZ. The system may also need to record logs of all PTZ control operations, including control start and end commands, frame rate adjustment history, and video quality indicators, for post-audit and performance analysis. Finally, the system may also need to integrate with other systems, such as alarm systems or access control systems, to achieve more complex linkage operations. For example, when the PTZ detects abnormal activity, it may automatically trigger an alarm and notify security personnel.

[0176] Through the above steps, the security monitoring system can flexibly control the PTZ camera, optimize the transmission of the video stream, and restore to the original settings after the operation is completed, ensuring the continuity and effectiveness of monitoring.

[0177] The present application is further explained below with reference to specific examples:

[0178] Figure 3 is a network topology block diagram of an optional PTZ control method according to an embodiment of the present application, such as Figure 3 As shown, in the application scenario of the PTZ monitoring device, the user can view the monitoring screen in real time through the terminal device (mobile phone APP or PC client software, etc.), and can also control the PTZ to move to the area of ​​interest during the process, including but not limited to: the terminal device and the PTZ monitoring device complete interactive communication through the relay (Relay) forwarding server. Among them, the relay forwarding server includes: a media forwarding server and a signaling forwarding server. The media forwarder mainly completes the forwarding of media data, and the signaling server mainly completes the control signaling data forwarding. The terminal device and the PTZ monitoring device complete interactive communication through a point-to-point (P2P, Peer to Peer) method. Before establishing a P2P channel, a STUN server (Simple Traversal of UDP throughNAT, a server component in a network protocol) and a signaling forwarding server are additionally required to obtain NAT information and assist in penetration (this part is not within the design of this application and will not be described in detail); after the P2P channel is established, the terminal device can directly complete the signaling and media data interaction with the PTZ monitoring device through the signaling transmission channel and the media transmission channel.

[0179] Therefore, this application is mainly to solve the problem that the video screen lags behind the pan-tilt operation after the media data is accumulated. Therefore, according to the above method, this application performs frame loss processing on the video backlog cache data in the pan-tilt monitoring equipment, media forwarding server, and terminal equipment based on the pan-tilt control command information and combined with the network evaluation information, and at the same time adjusts its encoding bit rate according to the cache situation sent by the monitoring device, thereby ensuring the real-time synchronization of the pan-tilt operation and the video screen.

[0180] The specific implementation process is described below:

[0181] Figure 4 is a module block diagram of an optional PTZ control method according to an embodiment of the present application, such as Figure 4 As shown, including:

[0182] PTZ monitoring equipment:

[0183] PTZ command parsing module: performs protocol parsing on the PTZ control commands received from the terminal device, including parsing the specific operation type (such as start rotation, end rotation), rotation direction (such as up, down, left, right, etc.) and PTZ rotation speed.

[0184] PTZ operation control module: controls the PTZ operation according to the specific PTZ control commands parsed by the PTZ parsing module.

[0185] PTZ detection scheduling control module: After receiving the PTZ control start command, the media history cache is cyclically cleared according to the set detection interval T and the encoding bit rate of the monitoring device is adjusted according to the strategy until the PTZ control end command is received, then the scheduling control is stopped.

[0186] Encoding module: caches the media data encoded by the device encoding unit. Each channel has an independent cache. If the device is a storage device and the channel is a digital channel, the media data is the data received from the lower-level device through the network. This module supports external control to execute the frame loss strategy.

[0187] Transcoding module: transcodes the data taken out from the encoding module, such as packetization and grouping according to the transmission protocol, and caches it. This module supports external control to execute the frame loss strategy.

[0188] Sending module: Sending identification and caching of data taken from the device transcoding module, such as identifying unsent data, and for reliable UDP transmission mode (this mode is the main mode of media transmission due to its good real-time and transmission performance), it is also necessary to identify data that has been sent but not received confirmation (i.e., in-transit data is also called flight data). This module supports external control to execute frame loss strategy.

[0189] Media forwarding server:

[0190] Forwarding module: Caches the forwarding data from the media forwarding server, mainly to prevent data loss when the forwarding rate from the PTZ monitoring device to the media forwarding server is greater than the forwarding rate from the media forwarding server to the terminal device. This module supports the execution of frame loss strategy based on auxiliary control frames.

[0191] Terminal equipment:

[0192] Receiving module: Caches the data received by the terminal device from the network. This module supports the execution of frame loss strategy based on auxiliary control frames.

[0193] Transcoding module: transcodes the data taken out from the terminal device receiving module, such as removing the protocol header and assembling packets according to the transmission protocol, and caches them for the decoding module to call. This module supports the execution of frame loss strategy based on auxiliary control frames.

[0194] Decoding module: caches and decodes the data taken out from the transcoding module of the terminal device. This module supports the execution of frame loss strategy based on auxiliary control frames.

[0195] Figure 5 is a flow chart of an optional PTZ control method according to an embodiment of the present application, such as Figure 5 As shown, including:

[0196] S1, when the PTZ monitoring device receives a PTZ control command, the PTZ parsing module is called to parse the PTZ control command to obtain information such as operation type, rotation direction, rotation speed, etc. When it is a PTZ start command, the PTZ control module is called to control the PTZ to rotate according to the control command; when it is a PTZ end command, the PTZ monitoring device is restored to the original encoding bit rate, and the original encoding bit rate is the encoding bit rate value of the monitoring device obtained when the PTZ rotation start command is received.

[0197] S2, when receiving the PTZ start command, sets the detection interval T according to the detection scheduling module. The detection interval T can be evaluated according to the encoding frame rate and I frame interval of the monitoring device (for example, if the frame rate is 25 and the I frame interval is 25, then the interval time of I frame generation is 1s). T should be less than the interval time of I frame generation to ensure that the frame loss strategy can be executed regularly to reduce the amount of media data as much as possible, so as to ensure the real-time performance of the video screen and PTZ operation.

[0198] S3, clear the media historical cache data according to the objective delay time and the encoding frame rate. The objective delay time is evaluated by the RTT (Round Trip Time) of the media transmission link, and the maximum value of all RTTs obtained regularly at historical moments is taken; and the number of objective delay frames is evaluated according to the encoding frame rate and the client delay time (for example, if the objective delay time is 200ms and the frame rate is 25, the number of objective delay frames is 5); and the strategy for clearing cache data is formulated based on the comparison result of the number of flight data frames in the monitoring device sending module (the number of data frames that have been sent from the PTZ monitoring device but have not yet been received or processed at the receiving end (such as the media forwarding server, terminal device)) and the number of objective delay frames (the objective delay time (such as the maximum value of RTT) is divided by the encoding frame rate, and the value obtained is the number of delay frames generated by video screen transmission under ideal conditions):

[0199] If the number of flight frames is equal to the number of objective delay frames, it means that the current network situation is basically the same as the evaluation. To ensure the real-time synchronization between the PTZ operation and the video screen, all non-key frame data in each buffer are cleared (the flight data frames in the sending buffer are not cleared, the same below);

[0200] If the number of flying frames is less than the number of objective delayed frames, it indicates that the network is getting better and the picture quality can be further guaranteed while ensuring real-time performance. Then some non-critical frames in each cache are cleared.

[0201] If the number of flying frames is greater than the number of objective delayed frames, it means that there is a bottleneck in the receiving end processing, and all non-key frame data in each cache should also be cleared;

[0202] In addition, for the strategy of clearing cache data:

[0203] The frame loss strategy is executed from the monitoring device encoding cache, monitoring device transcoding cache, monitoring device sending cache, media forwarding server forwarding cache (this link is not involved for P2P point-to-point), terminal device receiving cache, terminal device transcoding cache, and terminal device decoding cache in sequence, and it is necessary to ensure that the frames that are not lost can still be decoded normally (such as H264 / H265 encoding, a GOP series IPPPPPI format, and the P frames before the next I frame are discarded in sequence); at the same time, for the monitoring device encoding cache, monitoring device transcoding cache, and monitoring device sending cache, the frame loss strategy can be directly executed by calling the corresponding control module through the interface. For the media forwarding server forwarding cache, terminal device receiving cache, terminal device transcoding cache, and terminal device decoding cache, auxiliary control frames can be added to the monitoring device sending cache. After the corresponding modules receive the control frame, the specified frame loss strategy is executed.

[0204] S4. Based on the comparison result between the number of flight data frames and the number of objective delay frames, if the number of flight frames is greater than the number of objective delay frames, the encoding bit rate is appropriately reduced to ensure the real-time quality of the image, and the value before the initial change is maintained (the value set on the monitoring device before the initial bit rate adjustment, to ensure that it can be restored when the subsequent control is completed).

[0205] S5, wait for the detection interval T to be reached, and execute S2, S3, S4 in a loop until the PTZ end control command is received, then exit the loop, and restore the encoding bit rate of the monitoring device to the initial value.

[0206] Through the embodiments of the present application, according to the pan-tilt control command information and in combination with the network evaluation information, frame loss processing is performed on the video backlog cache data in the pan-tilt monitoring device, the media forwarding server, and the terminal device, and at the same time, the encoding bit rate is adjusted according to the cache situation sent by the monitoring device, thereby ensuring the real-time synchronization of the pan-tilt operation and the video screen. Combined with the actual application scenarios and network conditions, the problem that the amount of video data is much larger than the amount of pan-tilt signaling data, resulting in the video screen lagging behind the pan-tilt operation, is effectively solved. Combined with the characteristics of the pan-tilt monitoring device, according to the pan-tilt control command information and in combination with the network evaluation information, a corresponding frame loss strategy is formulated and executed to ensure the real-time synchronization of the pan-tilt operation and the video screen.

[0207] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0208] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0209] According to another aspect of the embodiments of the present application, a pan / tilt control device for implementing the pan / tilt control method is also provided. Figure 6 As shown, the device comprises:

[0210] The first control module 602 is used to control the pan-tilt control start instruction to execute a target action corresponding to the pan-tilt control start instruction in response to the pan-tilt control start instruction, and obtain a delay and a frame rate corresponding to the pan-tilt control, wherein the pan-tilt control start instruction is sent by the target terminal, the delay represents the transmission delay between the target terminal and the pan-tilt, and the frame rate represents the frame rate used by the pan-tilt to transmit data;

[0211] A determination module 604 is used to determine a target frame number based on the number of flight frames, the delay and the frame rate, wherein the number of flight frames represents the number of frames that the PTZ has sent to the target terminal and the number of frames that the target terminal has not yet determined to have received;

[0212] A clearing module 606 is used to clear non-key frame data corresponding to the target frame number in the first buffer data, wherein the first buffer data represents data to be sent by the PTZ to the target terminal, and the first buffer data includes non-key frame data;

[0213] The second control module 608 is used to control the gimbal to stop executing the target action in response to the gimbal control end instruction.

[0214] As an optional solution, the device is used to determine the target number of frames based on the number of flying frames, delay and frame rate in the following manner:

[0215] Get the number of flight frames;

[0216] Determine the number of delayed frames according to the ratio between the delay and the frame rate;

[0217] The target frame number is determined based on the relationship between the number of flying frames and the number of delayed frames.

[0218] As an optional solution, the device is used to determine the target frame number according to the value relationship between the number of flight frames and the number of delayed frames in the following manner:

[0219] When the number of flying frames is greater than or equal to the number of delayed frames, determining the target number of frames as the first number;

[0220] When the number of flying frames is less than the number of delayed frames, the target number of frames is determined to be a second number, wherein the second number is less than the first number.

[0221] As an optional solution, the above device is also used for:

[0222] After determining the target number of frames based on the number of flight frames, the delay and the frame rate, an auxiliary control frame is generated based on the target number of frames, wherein the auxiliary control frame is used to mark non-key frame data;

[0223] An auxiliary control frame is sent to the target terminal to enable the target terminal to clear non-key frame data in the second cache data, wherein the second cache data represents data that the target terminal has received from the PTZ, and the second cache data includes the non-key frame data.

[0224] As an optional solution, the above device is also used for:

[0225] In response to the gimbal control start instruction, control the gimbal to perform a target action corresponding to the gimbal control start instruction, and obtain a delay and a frame rate corresponding to the gimbal;

[0226] determining a target time interval based on the frame rate and a predetermined key frame interval;

[0227] The following steps are cyclically performed in the target time interval: determining the target frame number based on the number of flying frames, the delay and the frame rate; clearing the non-key frame data in the first buffer data;

[0228] In response to the gimbal control end instruction, the gimbal is controlled to stop executing the target action.

[0229] As an optional solution, the device is used to determine the target time interval based on the frame rate and the predetermined key frame interval in the following manner:

[0230] Obtaining a preset sample time interval;

[0231] Determine a key frame time interval based on the key frame interval and the frame rate;

[0232] Any time interval among the sample time intervals that is less than or equal to the key frame time interval is determined as the target time interval.

[0233] As an optional solution, the above device is also used for:

[0234] In response to the PTZ control start command, obtain the delay and frame rate corresponding to the PTZ;

[0235] Reduce the frame rate corresponding to the gimbal to the target frame rate;

[0236] In response to the pan / tilt control end instruction, the pan / tilt is controlled to stop executing the target action, and the target frame rate corresponding to the pan / tilt is restored to the frame rate.

[0237] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0238] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0239] According to one aspect of the present application, a computer program product is provided. The computer program product includes a computer program.

[0240] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0241] Figure 7 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0242] It should be noted that Figure 7 The computer system 700 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0243] like Figure 7 As shown, the computer system 700 includes a central processing unit 701 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 702 (ROM) or the program loaded from the storage part 708 to the random access memory 703 (RAM). Various programs and data required for system operation are also stored in the random access memory 703. The central processing unit 701, the read-only memory 702 and the random access memory 703 are connected to each other through a bus 704. The input / output interface 705 (Input / Output interface, i.e., I / O interface) is also connected to the bus 704.

[0244] The following components are connected to the input / output interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read therefrom is installed into the storage section 708 as needed.

[0245] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processor 701, various functions defined in the system of the present application are executed.

[0246] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processor 701, various functions provided by the embodiment of the present application are performed.

[0247] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned pan / tilt control method is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a terminal device as an example. Figure 8 As shown, the electronic device includes a memory 802 and a processor 804. The memory 802 stores a computer program, and the processor 804 is configured to execute the steps in any of the above method embodiments through the computer program.

[0248] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0249] Optionally, in this embodiment, the above-mentioned processor can be configured to execute the methods in each embodiment of the present application through a computer program.

[0250] Alternatively, a person skilled in the art may understand that: Figure 8 The structure shown is for illustration only. Figure 8 The structure of the electronic device is not limited. Figure 8 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 8 Different configurations shown.

[0251] Among them, the memory 802 can be used to store software programs and modules, such as the program instructions / modules corresponding to the pan-tilt control method and device in the embodiments of the present application. The processor 804 executes various functional applications and data processing by running the software programs and modules stored in the memory 802, that is, to implement the above-mentioned pan-tilt control method. The memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 802 may further include a memory remotely located relative to the processor 804, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 802 can be specifically, but not limited to, used to store information such as pan-tilt control instructions or data. As an example, such as Figure 8 As shown, the memory 802 may include but is not limited to the first control module 602, the determination module 604, the clearing module 606 and the second control module 608 in the PTZ control device. In addition, other module units in the PTZ control device may also be included but are not limited to, which will not be repeated in this example.

[0252] Optionally, the transmission device 806 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 806 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 806 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0253] In addition, the electronic device further includes: a display 808 for displaying the pan / tilt screen; and a connection bus 810 for connecting the various module components in the electronic device.

[0254] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. The nodes may form a peer-to-peer network, and any form of computing device, such as a server, terminal or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0255] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the gimbal control method provided in various optional implementations of the above-mentioned gimbal control aspect.

[0256] Optionally, in this embodiment, the above-mentioned computer-readable storage medium can be configured to store data for executing the methods in various embodiments of the present application.

[0257] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0258] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0259] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for one or more electronic devices to execute all or part of the steps of the methods described in each embodiment of the present application.

[0260] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0261] In the several embodiments provided in the present application, it should be understood that the disclosed application can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0262] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0263] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0264] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A pan / tilt control method, characterized in that: include: In response to a pan-tilt control start instruction, the pan-tilt is controlled to execute a target action corresponding to the pan-tilt control start instruction, and a delay and a frame rate corresponding to the pan-tilt are obtained, wherein the pan-tilt control start instruction is sent by a target terminal, the delay represents a transmission delay between the target terminal and the pan-tilt, and the frame rate represents a frame rate used by the pan-tilt to transmit data; Determining a target frame number based on the number of flying frames, the delay and the frame rate, wherein the number of flying frames represents the number of frames that the PTZ has sent to the target terminal and the number of frames that the target terminal has received is not determined; Clearing non-key frame data corresponding to the target frame quantity in first cache data, wherein the first cache data represents data to be sent by the PTZ to the target terminal, and the first cache data includes the non-key frame data; In response to a pan / tilt control end instruction, controlling the pan / tilt to stop executing the target action; The method also includes: determining the ratio of the delay and the frame rate as the number of delay frames, determining the target number of frames as a first number when the number of flying frames is greater than or equal to the number of delay frames, and determining the target number of frames as a second number when the number of flying frames is less than the number of delay frames, wherein the second number is less than the first number.

2. The method according to claim 1, characterized in that The determining of the target frame number based on the number of flying frames, the delay and the frame rate comprises: Obtain the number of flight frames; Determining the number of delayed frames according to a ratio between the delay and the frame rate; The target frame number is determined according to the value relationship between the flight frame number and the delayed frame number.

3. The method according to claim 1, characterized in that After determining the target number of frames based on the number of flying frames, the delay and the frame rate, the method further includes: generating an auxiliary control frame based on the target frame quantity, wherein the auxiliary control frame is used to mark the non-key frame data; The auxiliary control frame is sent to the target terminal so that the target terminal clears the non-key frame data in second cache data, wherein the second cache data represents data that the target terminal has received from the pan / tilt station, and the second cache data includes the non-key frame data.

4. The method according to claim 1, characterized in that The method further comprises: In response to the gimbal control start instruction, control the gimbal to perform a target action corresponding to the gimbal control start instruction, and obtain a delay and a frame rate corresponding to the gimbal; determining a target time interval based on the frame rate and a predetermined key frame interval; The following steps are cyclically performed in the target time interval: determining the target number of frames based on the number of flight frames, the delay and the frame rate; clearing the non-key frame data in the first buffer data; In response to the gimbal control end instruction, the gimbal is controlled to stop executing the target action.

5. The method according to claim 4, characterized in that The determining the target time interval based on the frame rate and a predetermined key frame interval comprises: Obtaining a preset sample time interval; determining a key frame time interval based on the key frame interval and the frame rate; Any time interval among the sample time intervals that is less than or equal to the key frame time interval is determined as the target time interval.

6. The method according to claim 4, characterized in that The method further comprises: In response to the pan / tilt control start instruction, obtaining a delay and a frame rate corresponding to the pan / tilt; Reducing the frame rate corresponding to the pan / tilt to a target frame rate; In response to the gimbal control end instruction, the gimbal is controlled to stop executing the target action, and the target frame rate corresponding to the gimbal is restored to the frame rate.

7. A pan / tilt control device, characterized in that: include: A first control module is used to control the pan-tilt control start instruction to control the pan-tilt to perform a target action corresponding to the pan-tilt control start instruction, and obtain a delay and a frame rate corresponding to the pan-tilt, wherein the pan-tilt control start instruction is sent by a target terminal, the delay represents a transmission delay between the target terminal and the pan-tilt, and the frame rate represents a frame rate used by the pan-tilt to transmit data; A determination module, configured to determine a target frame number based on the number of flight frames, the delay, and the frame rate, wherein the number of flight frames represents the number of data that the PTZ has sent to the target terminal but it is not determined whether the target terminal has received the data; a clearing module, used for clearing non-key frame data corresponding to the target frame quantity in the first cache data, wherein the first cache data represents data to be sent by the PTZ to the target terminal, and the first cache data includes the non-key frame data; A second control module, configured to control the pan / tilt platform to stop executing the target action in response to a pan / tilt platform control end instruction; The device is also used to: determine the ratio of the delay and the frame rate as the number of delayed frames, determine the target number of frames as a first number when the number of flying frames is greater than or equal to the number of delayed frames, and determine the target number of frames as a second number when the number of flying frames is less than the number of delayed frames, wherein the second number is less than the first number.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein the computer program can be executed by an electronic device to perform the method described in any one of claims 1 to 6.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.

Images