Frame rate adaptive adjustment method and device and video monitoring system

Abstract

The invention relates to a frame rate adaptive adjustment method and device and a video monitoring system. The method comprises the following steps: respectively enabling a first algorithm in a specified number of video transmission channels; according to the maximum video transmission frame rate supported when the target equipment operates the first algorithm and consumed resources of the target equipment, obtaining a residual video transmission frame rate which can be used when the first algorithm is operated; the number of video transmission channels triggered by a second algorithm in the target device is obtained, and the second algorithm is used for triggering the corresponding video transmission channel when it is judged that the image frame transmitted by each video transmission channel meets the detection requirement of the first algorithm; and dividing the residual video transmission frame rate by the number of the triggered video transmission channels to obtain a first video transmission frame rate when the first algorithm is operated. By adopting the method, the transmission frame rate of the video channel can be adaptively adjusted.

Description

Technical Field

[0001] The present application relates to the field of computer algorithms, and in particular to a frame rate adaptive adjustment method, device, and video surveillance system. Background Art

[0002] With the rapid development of intelligent video surveillance systems, back-end storage devices such as DVRs (DVRs) not only store video data but also increasingly incorporate intelligent analysis capabilities and support the activation of intelligent algorithms. When back-end DVRs support intelligent algorithms, the computing power of each device is fixed, as is the number of intelligent channels enabled for each intelligent algorithm. Consequently, the device's total frame rate for each intelligent algorithm is fixed. In real-world applications, the computing requirements for different video channels or video content within different time periods vary significantly. Therefore, fixing the frame rate and number of channels for each algorithm's intelligent analysis can lead to a waste of computing resources or even a shortage of computing resources.

[0003] Currently, no effective solution has been proposed to the problem that the frame rate of video channel transmission cannot be adaptively adjusted in related technologies. Summary of the Invention

[0004] Based on this, it is necessary to provide a frame rate adaptive adjustment method, device and video surveillance system that can solve the problem of being unable to adaptively adjust the frame rate of video channel transmission in order to solve the above technical problems.

[0005] In a first aspect, this embodiment provides a frame rate adaptive adjustment method, including:

[0006] enabling a first algorithm in a specified number of video transmission channels respectively;

[0007] Obtaining a remaining video transmission frame rate that can be used when running the first algorithm based on a maximum video transmission frame rate supported by the target device when running the first algorithm and consumed resources of the target device;

[0008] Obtaining the number of the video transmission channels in the target device that are triggered by a second algorithm, wherein the second algorithm is configured to trigger the corresponding video transmission channel when it is determined that the image frame transmitted to each of the video transmission channels meets the detection requirements of the first algorithm;

[0009] The remaining video transmission frame rate is divided by the number of the triggered video transmission channels to obtain a first video transmission frame rate when running the first algorithm.

[0010] In some embodiments, obtaining a remaining video transmission frame rate that can be used when running the first algorithm based on a maximum video transmission frame rate supported by a target device when running the first algorithm and consumed resources of the target device includes:

[0011] Obtaining a first correlation between a video transmission frame rate and consumed resources when the first algorithm is running;

[0012] Based on the first association relationship, obtaining a third video transmission frame rate corresponding to the consumed resources;

[0013] According to the difference between the maximum video transmission frame rate and the third video transmission frame rate, a remaining video transmission frame rate not consumed when the target device runs the first algorithm is obtained.

[0014] In some embodiments, obtaining a third video transmission frame rate corresponding to the consumed resources based on the first association relationship includes:

[0015] When the target device enables the first algorithm and runs the third algorithm, obtaining a second correlation relationship between a video transmission frame rate and consumed resources when the third algorithm is running;

[0016] Obtaining a conversion coefficient according to the first association relationship and the second association relationship;

[0017] Obtaining a fourth video transmission frame rate selected by a video transmission channel corresponding to the third algorithm;

[0018] The fourth video transmission frame rate is converted based on the conversion coefficient to obtain the third video transmission frame rate.

[0019] In some embodiments, after dividing the remaining video transmission frame rate by the number of triggered video transmission channels to obtain a first video transmission frame rate when running the first algorithm, the method includes:

[0020] Obtaining a second video transmission frame rate when the algorithm performance of the first algorithm meets a preset performance requirement;

[0021] When the second video transmission frame rate is less than or equal to the first video transmission frame rate, the image frame is transmitted based on the second video transmission frame rate to the module running the first algorithm in the target device.

[0022] In some embodiments, enabling the first algorithm in a specified number of video transmission channels respectively includes:

[0023] Obtain the sum of the number of video transmission channels opened by the target device and the specified number;

[0024] When the sum of the number of the opened video transmission channels and the specified number is less than or equal to the maximum number of channels that can be opened when the target device runs the first algorithm, the first algorithm is enabled in the specified number of the video transmission channels in response to a user instruction.

[0025] In some embodiments, when the sum of the number of the opened video transmission channels and the specified number is not less than the maximum number of channels that can be opened when the target device runs the first algorithm, a preset prompt message is generated in response to the user instruction and the first algorithm is refused to be enabled.

[0026] In a second aspect, this embodiment provides a frame rate adaptive adjustment device, including:

[0027] An enabling module, configured to enable the first algorithm in a specified number of video transmission channels respectively;

[0028] a frame rate acquisition module, configured to obtain a remaining video transmission frame rate that can be used when running the first algorithm based on a maximum video transmission frame rate supported by the target device when running the first algorithm and consumed resources of the target device;

[0029] A channel triggering module, configured to obtain the number of the video transmission channels in the target device triggered by the second algorithm;

[0030] The frame rate calculation module is used to divide the remaining video transmission frame rate by the number of the triggered video transmission channels to obtain a first video transmission frame rate when running the first algorithm.

[0031] In a third aspect, a video surveillance system is provided in this embodiment, including a video acquisition device and a target device for implementing the frame rate adaptive adjustment method described in the first aspect above; wherein,

[0032] The video acquisition device is used to obtain a video stream, and the video stream can be transmitted to the target device through a video transmission channel.

[0033] In a fourth aspect, a computer device is provided in this embodiment, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the frame rate adaptive adjustment method described in the first aspect is implemented.

[0034] In a fifth aspect, a computer-readable storage medium is provided in this embodiment, on which a computer program is stored. When the computer program is executed by a processor, the frame rate adaptive adjustment method described in the first aspect is implemented.

[0035] The above-mentioned frame rate adaptive adjustment method, device and video surveillance system calculate the first video transmission frame rate of the first algorithm in real time according to the number of channels triggered by the second algorithm and the resources consumed by the target device, thereby realizing adaptive adjustment of the frame rate and reducing the possibility of waste of computing power or insufficient computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A diagram illustrating an application environment of a frame rate adaptive adjustment method according to an embodiment;

[0037] Figure 2 1 is a flow chart of a method for adaptively adjusting a frame rate according to an embodiment;

[0038] Figure 3 1. A schematic flow chart of the steps of an intelligent frame rate adaptive adjustment method according to an embodiment;

[0039] Figure 4 is a structural block diagram of a frame rate adaptive adjustment device in one embodiment;

[0040] Figure 5 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0042] The frame rate adaptive adjustment method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. The target device 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The target device 102 receives the video stream and detects the video stream based on the enabled algorithm. The target device 102 can be a device that includes a video and image acquisition module and an algorithm execution module, such as a video recorder, a camera, etc.; the target device 102 can also be various personal computers, laptops, smart phones, tablets, etc. that are connected to video and image acquisition devices and have algorithm execution capabilities. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0043] In one embodiment, Figure 2 As shown, a frame rate adaptive adjustment method is provided, which is applied to Figure 1 The target device in the example is used as an example, which includes the following steps:

[0044] Step S202: enabling the first algorithm in a specified number of video transmission channels respectively.

[0045] The first algorithm is used to perform analysis, detection, and other operations on the video stream based on image frames. Optionally, the first algorithm can be an algorithm for behavior analysis, target tracking, vehicle identification, intelligent early warning, or the like. Optionally, the video transmission channels and the first algorithm to be enabled are determined based on user instructions. When the first algorithm is running, the video stream can be received through a specified number of video transmission channels.

[0046] Step S204 , obtaining a remaining video transmission frame rate that can be used when running the first algorithm according to the maximum video transmission frame rate supported by the target device when running the first algorithm and the consumed resources of the target device.

[0047] The maximum video transmission frame rate is the maximum number of image frames that the target device can process per second when only the first algorithm is enabled on the target device. The consumed resources of the target device are the hardware resources occupied by the target device when executing other programs and algorithms, including but not limited to CPU usage and memory usage.

[0048] Optionally, a maximum video transmission frame rate is obtained based on the total consumable resources of the target device, and the remaining consumable resources available for running the first algorithm on the target device are calculated based on the consumed resources of the target device; thereby, based on the relationship between the video transmission frame rate corresponding to the first algorithm and the consumed resources, a remaining video transmission frame rate corresponding to the remaining consumed resources is calculated. Alternatively, the video transmission frame rate supported by the first algorithm when running using the consumed resources of the target device is calculated, and the remaining video transmission frame rate is calculated based on the difference between the maximum video transmission frame rate and the video transmission frame rate corresponding to the consumed resources.

[0049] Step S206, obtaining the number of video transmission channels in the target device triggered by the second algorithm; the second algorithm is used to trigger the corresponding video transmission channel when the image frame transmitted by each video transmission channel meets the detection requirements of the first algorithm.

[0050] Among them, the triggered video transmission channel is used to transmit image frames to the module in the target device that executes the first algorithm. The second algorithm is used to screen whether the video stream or image frame corresponding to the video transmission channel needs to execute the first algorithm. The second algorithm can be motion detection, template matching, rule triggering algorithm, etc. It is understandable that if the image frame transmitted by the video transmission channel does not meet the detection requirements, the transmission of the image frame to the module in the target device that executes the first algorithm is stopped. The triggered video transmission channel in the target device changes in real time. Triggering the video transmission channel through the second algorithm can avoid all the image frames being transmitted by all the opened video transmission channels. Optionally, the resource consumption of the second algorithm is less than the resource consumption of the first algorithm.

[0051] Step S208: Divide the remaining video transmission frame rate by the number of triggered video transmission channels to obtain a first video transmission frame rate when running the first algorithm.

[0052] The remaining video transmission frame rate is the total number of video transmission frame rates that the target device can currently provide for the first algorithm. When the target device is not running any algorithms, the remaining video transmission frame rate is equal to the maximum video transmission frame rate supported by the target device. The remaining video transmission frame rate is allocated based on the number of triggered video transmission channels, thereby adaptively adjusting the video transmission channel frame rate of the first algorithm based on the actual computing power of the target device.

[0053] In the above-mentioned frame rate adaptive adjustment method, the second algorithm is used to adjust the number of channels (video transmission channels) processed by the first algorithm in real time, and given the actual resources and triggered video transmission channels, the video transmission frame rate corresponding to the triggered video transmission channels is adjusted, thereby avoiding the waste of computing resources or insufficient computing resources caused by fixed transmission channels or fixed frame rates, improving product competitiveness, and solving the problem of being unable to adaptively adjust the transmission frame rate of video channels.

[0054] In one embodiment, after dividing the remaining video transmission frame rate by the number of triggered video transmission channels to obtain the first video transmission frame rate when running the first algorithm, the method includes: obtaining the second video transmission frame rate when the algorithm performance of the first algorithm meets the preset performance requirements; when the second video transmission frame rate is less than or equal to the first video transmission frame rate, transmitting the image frame based on the second video transmission frame rate to the module running the first algorithm in the target device.

[0055] The second video transmission frame rate is the video transmission frame rate when the first algorithm performs better. The second video transmission frame rate can be obtained through experimentation or calculated based on the performance of the second algorithm and the target device. The performance of the first algorithm corresponding to the preset performance requirement can be set based on actual application needs.

[0056] If the second video transmission frame rate is less than or equal to the first video transmission frame rate, running the first algorithm based on the relatively larger first video transmission frame rate will result in poor algorithm performance. Therefore, the second video transmission frame rate is selected to transmit image frames to ensure the algorithm detection effect. It is understandable that if the first video transmission frame rate is less than the second video transmission frame rate, transmitting image frames based on the second video transmission frame rate may result in insufficient target device resources, resulting in weaker detection effect of the first algorithm. Therefore, the first video transmission frame rate is selected to transmit images.

[0057] Furthermore, obtaining a second video transmission frame rate when the algorithm performance of the first algorithm meets a preset performance requirement includes: adjusting the video transmission frame rate when the target device executes the first algorithm; and, if the algorithm performance meets the preset performance requirement, using the corresponding video transmission frame rate as the second video transmission frame rate. Optionally, the preset performance requirement indicates that the performance of the first algorithm has reached an optimal state. By obtaining the performance of the first algorithm at different video transmission frame rates, the video transmission frame rate when the algorithm performance reaches the optimal state is obtained, thereby obtaining the second video transmission frame rate.

[0058] In this embodiment, by comparing the first video transmission frame rate and the second video transmission frame rate, the image frame is transmitted to the module running the first algorithm using the first video transmission frame rate or the second video transmission frame rate according to the comparison result, and the detection effect of the first algorithm is guaranteed based on the reduced resources, which is close to the actual application scenarios and customer needs.

[0059] In one embodiment, based on the maximum video transmission frame rate supported when the target device runs the first algorithm and the consumed resources of the target device, the remaining video transmission frame rate that can be used when running the first algorithm is obtained, including: obtaining a first association relationship between the video transmission frame rate and the consumed resources when the first algorithm is running; based on the first association relationship, obtaining a third video transmission frame rate corresponding to the consumed resources; based on the difference between the maximum video transmission frame rate and the third video transmission frame rate, obtaining the remaining video transmission frame rate that is not consumed when the target device runs the first algorithm.

[0060] Specifically, for the same target device and a fixed number of enabled video transmission channels, the video transmission frame rate is positively correlated with resource consumption when the first algorithm is running. That is, when the number of video transmission channels is fixed and the video transmission frame rate increases, the corresponding resource consumption increases. Optionally, the first correlation relationship can be directly calculated based on the characteristics of the first algorithm, such as the CPU resources and memory resources required for the first algorithm to run. Alternatively, considering that the first correlation relationship may be different in different target devices, the first correlation relationship can also be obtained through experimental methods.

[0061] When the total transmission frame rate of the triggered video transmission channels is the third video transmission frame rate, the resources consumed by the target device during the execution of the first algorithm are equivalent to the currently consumed resources of the target device. Optionally, based on the number of currently triggered video transmission channels, a first association between the video transmission frame rate and the consumed resources required to run the first algorithm is obtained, and the consumed resources are mapped based on the first association to obtain the third video transmission frame rate. The third video transmission frame rate is subtracted from the maximum video transmission frame rate to obtain the remaining video transmission frame rate not consumed by the target device during the execution of the first algorithm.

[0062] In this embodiment, by evaluating the resource consumption of the first algorithm, the resources consumed by the target device are converted into the third video transmission frame rate when the first algorithm is running, so as to accurately calculate the remaining video transmission frame rate.

[0063] Furthermore, in one embodiment, based on the first association relationship, a third video transmission frame rate corresponding to the consumed resources is obtained, including: when the target device enables the first algorithm and runs the third algorithm, obtaining the second association relationship between the video transmission frame rate and the consumed resources when the third algorithm is running; obtaining a conversion coefficient based on the first association relationship and the second association relationship; obtaining a fourth video transmission frame rate selected for the video transmission channel corresponding to the third algorithm; and converting the fourth video transmission frame rate based on the conversion coefficient to obtain the third video transmission frame rate.

[0064] The third algorithm may be the second algorithm, or another algorithm other than the first and second algorithms. Optionally, the second association relationship may be directly calculated based on characteristics of the third algorithm, such as the CPU resources and memory resources required when the third algorithm is running; or the second association relationship may be obtained through experimental methods.

[0065] The conversion coefficient is used to indicate the ratio between the hardware resources required by the first algorithm and the hardware resources required by the second algorithm when the number of video transmission channels opened is the same and the algorithm performance meets the preset performance requirements. The conversion coefficient can be used to convert the video transmission frame rate when the second algorithm is running into the video transmission frame rate when the first algorithm is running.

[0066] Optionally, based on the total video transmission frame rate of the first algorithm supported by the target device, the number of video transmission channels that can be enabled when running the first algorithm and the corresponding video transmission frame rate consumed by each video transmission channel are obtained, and a first association relationship is determined based on this. A second association relationship is determined using the same method. Based on the first and second association relationships, the ratio of the hardware resources consumed by the first and second algorithms to enable a video transmission channel is obtained, thereby obtaining a conversion coefficient. If the first and second algorithms consume the same hardware resources, the conversion coefficient is 1. For ease of understanding, the total algorithm frame rate supported by the first algorithm is 64, the guaranteed minimum video transmission frame rate for each video transmission channel when the first algorithm is running is 4 frames, and a maximum of 16 video transmission channels can be enabled simultaneously. The total algorithm frame rate supported by the second algorithm is 32, the guaranteed minimum video transmission frame rate for each video transmission channel when the second algorithm is running is 4 frames, and a maximum of 8 video transmission channels can be enabled simultaneously. Therefore, the resources consumed by the second algorithm to enable one video transmission channel are equivalent to the resources consumed by the second algorithm to enable two video transmission channels, and the conversion coefficient is 2. When the second algorithm is enabled and one video transmission channel is already open, the fourth video transmission frame rate is 4 fps. Based on the conversion coefficient, the fourth video transmission frame rate is converted to a third video transmission frame rate of 8 fps. Furthermore, based on the difference between the first algorithm's maximum video transmission frame rate of 64 and the third video transmission frame rate of 8, the remaining video transmission frame rate not consumed when the target device runs the first algorithm is 58 fps. The frame rates and video transmission channels supported by the first and second algorithms are determined based on actual application requirements and are not limited here.

[0067] In this embodiment, when the target device runs at least two algorithms simultaneously, the fourth video transmission frame rate when the third algorithm is running can be quickly calculated to the third video transmission frame rate.

[0068] In one embodiment, enabling the first algorithm in a specified number of video transmission channels respectively includes: obtaining the sum of the number of video transmission channels that have been turned on by the target device and the specified number; and enabling the first algorithm in the specified number of video transmission channels in response to user instructions when the sum of the number of video transmission channels that have been turned on and the specified number is less than or equal to the maximum number of channels that can be turned on when the target device runs the first algorithm.

[0069] The maximum number of channels that can be enabled on the target device when running the first algorithm refers to the number of channels that can be enabled on the target device when running the first algorithm and the algorithm performance of the first algorithm meets the basic performance requirements. The specified number is the number of video transmission channels specified for enablement in step S201. The higher the performance requirements met by the first algorithm, the higher the video transmission frame rate of the video transmission channel, and the corresponding decrease in the number of channels that can be enabled on the target device.

[0070] User commands can be generated through a touch layer covering the terminal display screen, keys, trackballs or touchpads provided on the terminal housing, or external input devices such as a keyboard, touchpad or mouse.

[0071] Furthermore, in one embodiment, when the sum of the number of opened video transmission channels and the specified number is not less than the maximum number of channels that can be opened when the target device runs the first algorithm, a preset prompt message is generated in response to a user instruction, and the first algorithm is refused to be enabled.

[0072] If the sum of the number of enabled video transmission channels and the specified number is equal to the maximum number of channels, the first algorithm cannot be enabled. Optionally, a preset prompt message may be used to indicate to the user that the first algorithm has failed to be enabled.

[0073] In this embodiment, the number of video transmission channels that will be opened after enabling the first algorithm is obtained based on the sum of the number of video transmission channels already opened on the target device and the number of a specified number of video transmission channels. By comparing the number of video transmission channels and the maximum number of channels, it is determined whether to enable the first algorithm, thereby avoiding the situation where the target device cannot execute the first algorithm or the detection result of the first algorithm cannot meet user needs.

[0074] In related technologies, devices with intelligent detection capabilities, such as back-end storage DVRs, support the activation of intelligent algorithms. Taking face detection performed on a back-end storage DVR as an example, the face detection algorithm consists of multiple operators connected in series, sequentially performing the following: FD (Face Detection), FT (Face Tracking), FQ (Face Quality Assessment), FLL (Face Landmark Localization), FA (Face Attribute Analysis), FE (Face Feature Extraction), FC (Face Comparison), and FSELECT (Optimal Image Selection). The maximum frame rate for each intelligent algorithm is achieved by executing all operators on every frame. Therefore, the frame rate for each intelligent algorithm is fixed at the factory, meaning the number of channels enabled is fixed. This means the device will operate at the optimal or minimum video transmission frame rate. For example, a face recognition algorithm achieves optimal detection results at a video transmission frame rate of 12 frames. An 8-frame video transmission frame rate is the minimum frame rate guaranteed for the algorithm. While delayed tracking frames may occur, event loss is avoided. However, running the device at 12 frames reduces the number of video transmission channels enabled by the algorithm, while running the device at 8 frames wastes device resources.

[0075] Therefore, the total frame rate for each intelligent function is fixed. For example, a device capable of performing face detection, perimeter detection, and scene motion detection (SMD) can be used. When performing face detection, four video transmission channels can be enabled, each sending frames to the face detection algorithm at a frame rate of 12 frames. Once four face detection channels are enabled, other intelligent functions cannot be enabled. When performing perimeter detection, eight video transmission channels can be enabled, each sending frames to the face detection algorithm at a frame rate of 8 frames. When performing SMD, 16 video transmission channels can be enabled, each sending frames to the face detection algorithm at a frame rate of 2 frames. Similarly, after enabling eight perimeter detection or sixteen SMD channels, other intelligent functions cannot be enabled. To run multiple intelligent algorithms, related technologies enable different intelligent functions in a fixed ratio. For example, one face detection channel, two perimeter detection channels, and eight SMD channels can be enabled simultaneously, with a ratio of 1:2:4 for face detection: perimeter detection: SMD detection. However, in actual application scenarios, if the device's image acquisition module is set and turned on in a quiet channel, there will be a long period of time when no face is detected in the video stream transmitted by the image acquisition module, resulting in each frame of the input face detection algorithm running in vain. Running based on a fixed frame rate will waste the device's computing power.

[0076] Based on this, Figure 3 Provides an intelligent frame rate adaptive adjustment method. Figure 3 The method shown can be applied to devices with intelligent detection capabilities such as back-end storage hard disk recorders. Figure 3 As shown, the steps include:

[0077] Step S301: Receive an algorithm enablement request, wherein the enablement request may request to enable one or more algorithms, or request to enable one or more algorithms.

[0078] Step S302: Determine whether the number of algorithm channels to be enabled according to the algorithm enablement request is greater than MinFrameChnNum (minimum frame rate guaranteed channel number). If so, it is determined that the enablement cannot continue and the device has reached its upper limit; if not, the algorithm enablement is determined to be successful and step 303 is executed.

[0079] Step S303: Determine the video transmission channel that triggers motion detection.

[0080] After the algorithm is successfully activated, image frames are not immediately sent to the algorithm analysis module. Instead, they are transmitted to the algorithm analysis module only after the video stream corresponding to the video transmission channel triggers motion detection. Motion detection is used to identify and respond to various events or behaviors, and can be performed by a second algorithm.

[0081] Step S304 : Divide DevUnUsed (the remaining video transmission frame rate) supported by the current device by the number of video transmission channels that trigger motion detection to obtain a first video transmission frame rate.

[0082] When the device only runs the algorithm enabled in step 301, DevUnUsed is equal to DevMaxFrame (the total frame rate of the algorithm supported by the device), and DevMaxFrame is the maximum number of channels that can be opened when the device runs the algorithm in the above embodiment.

[0083] Step S305 , taking the minimum value between the first video transmission frame rate and the algorithm optimal frame rate as the algorithm sending frame rate, and repeating step S304 until a request to disable the enable is received.

[0084] The algorithm frame rate is not fixed, ranging between ChnAlgMinFrame (the algorithm's minimum guaranteed frame rate) and ChnAlgOptimalFrame (the algorithm's optimal frame rate). The minimum guaranteed frame rate is the frame rate at which the algorithm meets basic performance requirements. Running the algorithm at this frame rate may result in delayed tracking frames, but no events will be lost. While the algorithm performance is weaker than at the optimal frame rate, the overall effect is acceptable to users. The optimal frame rate is the frame rate at which the algorithm meets the preset performance requirements. Running the algorithm at this frame rate achieves the best algorithm detection results.

[0085] For ease of understanding, let's take the execution of SMD as an example. When executing SMD, DevMaxFrame (the total frame rate of the algorithm supported by the device) is 32fps; ChnAlgOptimalFrame (the optimal frame rate of the algorithm) is 2 fps; ChnAlgMinFrame (the minimum guaranteed frame rate of the algorithm) is 1 fps; OptimalFrameChnNum (the number of channels with the optimal frame rate) is 16; and MinFrameChnNum (the number of channels with the minimum guaranteed frame rate) is 32.

[0086] based on Figure 3The method shown adjusts the SMD frame rate as follows: after receiving the algorithm enable request, if the number of video transmission channels to be opened is greater than 32, it is determined that the algorithm cannot be opened. If the number of video transmission channels to be opened is less than 32, it is determined that the opening is successful. After opening, frames will not be sent to the algorithm analysis immediately. Only after the video transmission channel triggers the motion detection, the image frames will be sent to the algorithm. The video transmission frame rate is between ChnAlgMinFrame and OptimalFrameChnNum. For example, if the number of video transmission channels that trigger the motion detection is less than or equal to 16, the frame rate sent to the algorithm is 2 frames; if the number of video transmission channels that trigger the motion detection is greater than 16, the frame rate calculated by dividing DevUnUsed (remaining video transmission frame rate) supported by the current device by the number of video transmission channels that trigger the motion detection is transmitted. Image transmission is usually achieved at an integer frame rate.

[0087] In this embodiment, the video transmission frame rate is calculated according to the number of triggered and enabled video transmission channels, which can increase the number of enabled video transmission channels while ensuring the algorithm detection performance, thereby giving users more choices.

[0088] It should be understood that although the steps in the flowcharts of the various embodiments described above are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts of the various embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The order of execution of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps. For example, step S301 can be executed while steps S302 to S305 are being executed. Optionally, before executing the algorithm enable request, it can be determined whether the number of channels currently enabled is equal to the number of channels with a guaranteed frame rate. If so, the algorithm enable request is rejected; if not, the algorithm enable request is received.

[0089] Based on the same inventive concept, the embodiments of the present application further provide a frame rate adaptive adjustment device for implementing the aforementioned frame rate adaptive adjustment method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the frame rate adaptive adjustment device provided below can be found in the above-mentioned limitations of the frame rate adaptive adjustment method and will not be repeated here.

[0090] In one embodiment, Figure 4 As shown, a frame rate adaptive adjustment device is provided, including: an enabling module, a frame rate acquisition module, a channel triggering module and a frame rate calculation module, wherein:

[0091] An enabling module, configured to enable the first algorithm in a specified number of video transmission channels respectively;

[0092] a frame rate acquisition module, configured to obtain a remaining video transmission frame rate that can be used when running the first algorithm based on a maximum video transmission frame rate supported by the target device when running the first algorithm and consumed resources of the target device;

[0093] a channel triggering module, configured to obtain the number of video transmission channels in the target device that are triggered by the second algorithm;

[0094] The frame rate calculation module is used to divide the remaining video transmission frame rate by the number of triggered video transmission channels to obtain a first video transmission frame rate when running the first algorithm.

[0095] In some embodiments, the frame rate calculation module obtains the remaining video transmission frame rate that can be used when running the first algorithm based on the maximum video transmission frame rate supported by the target device when running the first algorithm and the consumed resources of the target device, including: obtaining a first association relationship between the video transmission frame rate and the consumed resources when the first algorithm is running; based on the first association relationship, obtaining a third video transmission frame rate corresponding to the consumed resources; and obtaining the remaining video transmission frame rate that is not consumed when the target device runs the first algorithm based on the difference between the maximum video transmission frame rate and the third video transmission frame rate.

[0096] Optionally, based on the first association relationship, a third video transmission frame rate corresponding to the consumed resources is obtained, including: when the target device enables the first algorithm and runs the third algorithm, obtaining the second association relationship between the video transmission frame rate and the consumed resources when the third algorithm is running; obtaining a conversion coefficient based on the first association relationship and the second association relationship; obtaining a fourth video transmission frame rate selected for the video transmission channel corresponding to the third algorithm; and converting the fourth video transmission frame rate based on the conversion coefficient to obtain a third video transmission frame rate.

[0097] In some embodiments, after the frame rate calculation module divides the remaining video transmission frame rate by the number of triggered video transmission channels to obtain the first video transmission frame rate when running the first algorithm, the execution method includes: obtaining the second video transmission frame rate when the algorithm performance of the first algorithm meets the preset performance requirements; when the second video transmission frame rate is less than or equal to the first video transmission frame rate, transmitting the image frame based on the second video transmission frame rate to the module running the first algorithm in the target device.

[0098] Optionally, the frame rate calculation module obtains the second video transmission frame rate when the algorithm performance of the first algorithm meets the preset performance requirements, including: adjusting the video transmission frame rate when the target device executes the first algorithm; when the first algorithm achieves optimal performance, the corresponding video transmission frame rate is used as the second video transmission frame rate.

[0099] In some embodiments, the enabling module enables the first algorithm in a specified number of video transmission channels, including: obtaining the sum of the number of video transmission channels enabled on the target device and the number of the specified number of video transmission channels; and enabling the first algorithm in the specified number of video transmission channels in response to a user instruction if the sum of the number of enabled video transmission channels and the number of the specified number of video transmission channels is less than or equal to the maximum number of channels that can be enabled when the target device runs the first algorithm. Optionally, the enabling module generates a preset prompt message in response to a user instruction and refuses to enable the first algorithm if the number of enabled video transmission channels is not less than the maximum number of channels that can be enabled when the target device runs the first algorithm.

[0100] Each module in the above-mentioned frame rate adaptive adjustment device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0101] Based on the same inventive concept, embodiments of the present application also provide a video surveillance system for implementing the aforementioned frame rate adaptive adjustment method. The video surveillance system includes a video capture device and a target device; the target device is configured to implement the steps described in each of the aforementioned method embodiments. The video capture device is configured to capture a video stream, which can be transmitted to the target device via a video transmission channel.

[0102] Optionally, the video acquisition device can acquire multiple independent video streams and transmit different video streams through different video transmission channels. The target device can execute at least one intelligent algorithm.

[0103] The implementation solution provided by the video surveillance system to solve the problem is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more video surveillance system embodiments provided below can refer to the limitations of the frame rate adaptive adjustment method above and will not be repeated here.

[0104] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store algorithms and image data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a frame rate adaptive adjustment method is implemented.

[0105] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0106] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0107] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0108] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0109] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0110] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A frame rate adaptive adjustment method, characterized in that: The method comprises: enabling a first algorithm in a specified number of video transmission channels respectively; Obtaining a remaining video transmission frame rate that can be used when running the first algorithm based on a maximum video transmission frame rate supported by the target device when running the first algorithm and consumed resources of the target device; Obtaining the number of the video transmission channels in the target device that are triggered by a second algorithm, wherein the second algorithm is configured to trigger the corresponding video transmission channel when it is determined that the image frame transmitted to each of the video transmission channels meets the detection requirements of the first algorithm; The remaining video transmission frame rate is divided by the number of the triggered video transmission channels to obtain a first video transmission frame rate when running the first algorithm.

2. The method according to claim 1, characterized in that Obtaining a remaining video transmission frame rate that can be used when running the first algorithm based on a maximum video transmission frame rate supported by the target device when running the first algorithm and consumed resources of the target device, including: Obtaining a first correlation between a video transmission frame rate and consumed resources when the first algorithm is running; Based on the first association relationship, obtaining a third video transmission frame rate corresponding to the consumed resources; According to the difference between the maximum video transmission frame rate and the third video transmission frame rate, a remaining video transmission frame rate not consumed when the target device runs the first algorithm is obtained.

3. The method according to claim 2, characterized in that Obtaining a third video transmission frame rate corresponding to the consumed resources based on the first association relationship includes: When the target device enables the first algorithm and runs the third algorithm, obtaining a second correlation relationship between a video transmission frame rate and consumed resources when the third algorithm is running; Obtaining a conversion coefficient according to the first association relationship and the second association relationship; Obtaining a fourth video transmission frame rate selected by a video transmission channel corresponding to the third algorithm; The fourth video transmission frame rate is converted based on the conversion coefficient to obtain the third video transmission frame rate.

4. The method according to claim 1, wherein After dividing the remaining video transmission frame rate by the number of the triggered video transmission channels to obtain a first video transmission frame rate when running the first algorithm, the method includes: Obtaining a second video transmission frame rate when the algorithm performance of the first algorithm meets a preset performance requirement; When the second video transmission frame rate is less than or equal to the first video transmission frame rate, the image frame is transmitted based on the second video transmission frame rate to the module running the first algorithm in the target device.

5. The method according to claim 1, wherein The first algorithm is enabled in a specified number of video transmission channels respectively, including: Obtaining the sum of the number of video transmission channels enabled on the target device and the number of the specified number of video transmission channels; When the sum of the number of the opened video transmission channels and the number of the specified number of video transmission channels is less than or equal to the maximum number of channels that can be opened when the target device runs the first algorithm, the first algorithm is enabled in the specified number of video transmission channels in response to user instructions.

6. The method according to claim 5, characterized in that When the sum of the number of the opened video transmission channels and the specified number of video transmission channels is not less than the maximum number of channels that can be opened when the target device runs the first algorithm, a preset prompt message is generated in response to the user instruction, and the first algorithm is refused to be enabled.

7. A frame rate adaptive adjustment device, characterized in that: The device comprises: An enabling module, configured to enable the first algorithm in a specified number of video transmission channels respectively; a frame rate acquisition module, configured to obtain a remaining video transmission frame rate that can be used when running the first algorithm based on a maximum video transmission frame rate supported by the target device when running the first algorithm and consumed resources of the target device; A channel triggering module, configured to obtain the number of the video transmission channels in the target device triggered by the second algorithm; The frame rate calculation module is used to divide the remaining video transmission frame rate by the number of the triggered video transmission channels to obtain a first video transmission frame rate when running the first algorithm.

8. A video surveillance system, characterized in that: A target device comprising a video acquisition device and a target device for implementing the method according to any one of claims 1 to 6; wherein, The video acquisition device is used to obtain a video stream, and the video stream can be transmitted to the target device through a video transmission channel.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images