Method, apparatus and device for reporting remaining resources and scheduling image analysis tasks

Abstract

The present application discloses a method, apparatus and device for reporting remaining resources and scheduling picture analysis tasks, which relate to the field of computer technology and are applied to management devices and picture analysis devices. On the one hand, the picture analysis device periodically determines the current remaining resources of the picture analysis device based on three factors: the remaining space of the buffer queue, the resolution of each picture in the buffer queue, and the number of target pictures processed within a historical preset time, and reports them to the management device. On the other hand, after receiving a picture stream analysis task, the management device determines the estimated resources required to execute the picture stream analysis task, and then accurately allocates the picture stream analysis task to each picture analysis device based on the remaining resources reported by each picture analysis device and the estimated resources required for the picture stream analysis task.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and provides a method, apparatus, and device for reporting remaining resources and scheduling picture analysis tasks. Background Art

[0002] With the development of computer vision technology, intelligent analysis technology is increasingly widely used in the security field. Intelligent analysis means that for a given picture stream, relevant attributes of target objects therein can be analyzed, such as the license plate number of a vehicle, the gender and age of a pedestrian, etc. At present, the application of picture stream analysis is becoming more and more extensive, but the requirements for analysis devices are also constantly increasing. It is difficult for a single picture analysis device to meet the actual application requirements. Therefore, a large picture analysis system composed of multiple picture analysis devices has emerged. In this picture analysis system, multiple picture analysis devices are uniformly managed by a scheduling center to coordinate multiple picture analysis devices to jointly complete the picture stream analysis task.

[0003] In the related art, the scheduling center expects to evenly allocate tasks to each device. However, since the tasks currently being processed by each picture analysis device are complex and variable, even if the same number of picture stream data is allocated to each device, due to the different contents of different pictures, such as large differences in parameters such as the number of targets and resolution, the device resources consumed by different pictures are different. Therefore, the scheduling management center cannot accurately know the resource consumption of each device, and thus cannot achieve balanced task allocation, resulting in the underutilization of resources of some picture analysis devices and the overloading of some devices, leading to untimely intelligent analysis and delayed reporting of analysis results. Summary of the Invention

[0004] Embodiments of the present application provide a method, apparatus, and device for reporting remaining resources and scheduling picture analysis tasks, which are used to accurately calculate the current remaining resources of a picture analysis device based on the information of pictures that have been processed by the picture analysis device within a preset time and the information of pictures to be processed, and to achieve balanced task allocation by the scheduling management center.

[0005] On the one hand, a method for reporting the remaining resources of a device is provided, which is applied to a picture analysis device. The method includes:

[0006] Determine a first coefficient according to the number of targets included in each processed picture within a historical preset time period and the total number of the pictures; the first coefficient is used to characterize the level of resource consumption per unit of the picture analysis device for processing the processed pictures, and the first coefficient is positively correlated with the number of targets;

[0007] Determine a second coefficient corresponding to each to-be-processed picture according to the resolution corresponding to each to-be-processed picture in the buffer queue, where the buffer queue is used to store pictures that have been allocated to the picture analysis device but not processed, the second coefficient is used to characterize the difficulty level of the picture analysis device in processing each to-be-processed picture, and the second coefficient is positively correlated with the resolution;

[0008] Determine the estimated resources required for the picture analysis device to process the to-be-processed pictures according to the obtained first coefficient and each second coefficient;

[0009] Determine the remaining resources of the picture analysis device according to the total resources of the picture analysis device and the estimated resources, and report the remaining resources to the management device.

[0010] On the one hand, a picture analysis task scheduling method is provided, which is applied to a management device. The method includes:

[0011] Receive the remaining resources reported by each picture analysis device, where the remaining resources are obtained by each picture analysis device based on the resources consumed for processing a single processed picture within a historical preset time period and the difficulty level of processing each to-be-processed picture in the cache queue of the picture analysis device;

[0012] Receive a picture stream analysis task, where the picture stream analysis task carries picture stream data including a plurality of to-be-processed pictures;

[0013] Determine the estimated resources required to complete the picture stream analysis task according to the resolution corresponding to each to-be-processed picture in the picture stream data and the total number of pictures;

[0014] Select at least one picture analysis device to execute the picture stream analysis task from the picture analysis devices according to the remaining resources of the picture analysis devices and the estimated resources, and send the picture stream analysis task to the at least one picture analysis device.

[0015] On the one hand, a remaining resource reporting device is provided, including:

[0016] A first coefficient determination unit, configured to determine a first coefficient according to the target quantity included in each processed picture and the total quantity of each picture within a historical preset time period, where the first coefficient is used to characterize the level of the average resource consumption of the picture analysis device for processing the processed pictures, and the first coefficient is positively correlated with the target quantity;

[0017] A second coefficient determination unit, configured to determine a second coefficient corresponding to each to-be-processed picture according to the resolution corresponding to each to-be-processed picture in the buffer queue, where the buffer queue is used to store pictures that have been allocated to the picture analysis device but not processed, the second coefficient is used to characterize the difficulty level of the picture analysis device in processing each to-be-processed picture, and the second coefficient is positively correlated with the resolution;

[0018] A resource estimation unit, configured to determine the estimated resources required for the picture analysis device to process each of the to-be-processed pictures according to the obtained first coefficient and each second coefficient;

[0019] A remaining resource determination unit, configured to determine the remaining resources of the picture analysis device according to the total resources of the picture analysis device and the estimated resources, and report the remaining resources to the management device.

[0020] Optionally, the first coefficient determination unit is specifically configured to:

[0021] Determine a target coefficient corresponding to each processed picture according to the number of targets included in each processed picture within a historical preset time period; the target coefficient is used to measure the level of resources consumed by the picture analysis device in processing each processed picture;

[0022] Obtain the first coefficient according to the target coefficients corresponding to each of the pictures and the total number of the pictures.

[0023] Optionally, the resource estimation unit is specifically configured to:

[0024] Determine the sub-estimated resources required for each of the to-be-processed pictures according to the second coefficient corresponding to each of the to-be-processed pictures and the first coefficient;

[0025] Obtain the estimated resources based on the sub-estimated resources required for each of the to-be-processed pictures.

[0026] Optionally, the remaining resource determination unit is specifically configured to:

[0027] Determine the available resources of the picture analysis device according to the total resources of the picture analysis device and the estimated resources;

[0028] Determine whether the buffer queue is full. If it is full, determine that the remaining resources are zero;

[0029] If there is free space in the buffer queue, obtain the remaining resources according to the available resources and the size of the free space in the buffer queue.

[0030] Optionally, the remaining resource determination unit is further configured to:

[0031] Receive the picture stream analysis task sent by the management device, where the picture stream analysis task carries picture stream data; analyze each to-be-processed picture in the picture stream data respectively, and send the analysis results corresponding to each to-be-processed picture obtained to the management device.

[0032] On the one hand, a picture analysis task scheduling device is provided, including:

[0033] A subscription unit, configured to receive the remaining resources reported by each picture analysis device, where the remaining resources are the resources required by each picture analysis device for processing a single processed picture based on its own situation within a historical preset time period, and the processing difficulty levels of each to-be-processed picture in the cache queue of the picture analysis device;

[0034] A task receiving unit, configured to receive a picture stream analysis task, where the picture stream analysis task carries picture stream data including a plurality of to-be-processed pictures;

[0035] A resource prediction unit, configured to determine the estimated resources required to complete the picture stream analysis task according to the resolution and the total number of pictures corresponding to each to-be-processed picture in the picture stream data;

[0036] A scheduling unit, configured to select at least one picture analysis device for executing the picture stream analysis task from the various picture analysis devices according to the remaining resources of the various picture analysis devices and the estimated resources, and send the picture stream analysis task to the at least one picture analysis device.

[0037] Optionally, the resource prediction unit is specifically configured to:

[0038] Determine a first coefficient of the picture stream analysis task according to a reference value of a preset target coefficient, where the first coefficient is used to characterize the level of the average resources consumed by the picture analysis device for processing each to-be-processed picture, and the reference value is determined based on a target quantity included in a plurality of processed pictures; determine a second coefficient corresponding to each to-be-processed picture according to the resolution of each to-be-processed picture, where the second coefficient is used to characterize the processing difficulty level of the picture analysis device for processing each to-be-processed picture, and the second coefficient is positively correlated with the resolution; determine the sub-estimated resources required for each to-be-processed picture according to the first coefficient and the second coefficient corresponding to each to-be-processed picture; determine the estimated resources of the picture stream analysis task based on the sub-estimated resources corresponding to each to-be-processed picture.

[0039] Optionally, the scheduling unit is specifically configured to:

[0040] Analyze the remaining resources and the estimated resources of the respective image analysis devices, and select at least one image analysis device to execute the image stream analysis task; generate a scheduling instruction, where the scheduling instruction indicates the image stream analysis subtasks allocated to each image analysis device among the at least one image analysis device; and based on the scheduling instruction, send the corresponding image stream analysis subtasks to each image analysis device among the at least one image analysis device respectively.

[0041] On the one hand, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of any of the above methods are implemented.

[0042] On the one hand, a computer storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the steps of any of the above methods are implemented.

[0043] On the one hand, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps of any of the above methods.

[0044] In the embodiments of the present application, the image analysis device determines the estimated resources required for the image analysis device to process all the to-be-processed images based on the number of targets in each processed image within a historical preset time and the resolution of each to-be-processed image. Furthermore, based on the total resources and the estimated resources of the image analysis device, the current remaining resources are determined and reported to the management device. In this way, through the processing capacity of the image analysis device in the recent time period, the estimated resources required for each subsequent to-be-processed image can be more accurately estimated, and then the remaining resources of each image analysis device can be accurately calculated. Correspondingly, after receiving the image stream analysis task, the management platform determines the estimated resources required for the task based on the resolution and the total number of to-be-processed images in the image stream data. According to the remaining resources of each image analysis device and the estimated resources required to complete the task, the image stream analysis task is assigned to at least one image analysis device, that is, after the management device determines the estimated resources required for the current image stream analysis task, according to the accurate remaining resource amounts reported by each image analysis device, it accurately knows the remaining processing capabilities of each image analysis device, so as to improve the accuracy and balance of the allocation of the image stream analysis task, improve the resource utilization rate of each image analysis device, and speed up the efficiency of the image analysis task. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] To more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings required for use in the description of the embodiments or the related art. Obviously, the drawings in the following description are only those of the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0046] Figure 1 A scenario diagram provided for an embodiment of the present application;

[0047] Figure 2 A schematic diagram of the system architecture provided for an embodiment of the present application;

[0048] Figure 3 A schematic flowchart of the method for reporting remaining resources and allocating picture stream analysis tasks provided for an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of a remaining resource reporting device provided for an embodiment of the present application;

[0050] Figure 5 A schematic diagram of the structure of a picture analysis task scheduling device provided for an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of a computer device provided for an embodiment of the present application. Detailed implementation manners

[0052] To make the objectives, technical solutions, and advantages of the present application clearer and more understandable, the following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of them. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application. Without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0053] The following briefly introduces the design concept of the embodiments of the present application.

[0054] Image analysis technology generally uses mathematical models and combines image processing techniques to analyze underlying features and upper-layer structures, so as to extract the target information required in the picture, such as the license plate number of a vehicle, the gender and age of a pedestrian, etc. At present, image analysis technology has been widely applied in various fields, and the application scenarios are also growing. It is difficult for a single image analysis device to meet the actual application needs. Multiple image analysis devices need to form a large-scale image analysis system, and the scheduling center uniformly manages multiple image analysis devices to jointly complete the image stream analysis task.

[0055] However, in the related technology, the device resources and the resources required for tasks are calculated according to the number of pictures. For example, an image analysis device can process X photos per second. Now there are Y photos waiting to be processed in the device buffer queue, and a total of Z photos in an image stream analysis task, that is, the calculation is carried out according to the rule that the resources consumed by all pictures are equal. However, the pictures in the real scenario are dynamically changing. Some pictures have multiple targets, and there are also differences in the size and resolution of the pictures. The resources required to process each different picture may vary greatly. Therefore, the remaining resources calculated in this way are inaccurate, and the scheduling management center cannot evenly allocate tasks based on the remaining resources of each image analysis device. Some device resources are not fully utilized, some devices are overloaded, and the tasks are not processed in a timely manner.

[0056] In view of this, the embodiments of the present application provide a method for reporting remaining resources and scheduling image analysis tasks. In the method for reporting remaining resources, the image analysis device determines the estimated resources required for the image analysis device to process all the pictures to be processed based on the number of targets in each processed picture within a historical preset time and the resolution of each picture to be processed. Furthermore, based on the total resources of the image analysis device and the estimated resources, the current remaining resources are determined and reported to the management device. In this way, through the processing capacity of the image analysis device in the recent time period, the estimated resources required for each subsequent picture to be processed can be more accurately estimated, and then the remaining resources of each image analysis device can be accurately calculated. Correspondingly, when scheduling the image analysis task, after receiving the image stream analysis task, the management platform determines the estimated resources required for the task based on the resolution and the total number of pictures to be processed in the image stream data, and distributes the image stream analysis task to at least one image analysis device according to the remaining resources of each image analysis device and the estimated resources required to complete the task. That is, after the management device determines the estimated resources required for the current image stream analysis task, it accurately knows the remaining processing capabilities of each image analysis device according to the accurate remaining resource amounts reported by each image analysis device, so as to improve the accuracy and balance of the distribution of the image stream analysis task, improve the resource utilization rate of each image analysis device, and speed up the efficiency of the image analysis task.

[0057] After introducing the design concept of the embodiments of the present application, the following briefly introduces some application scenarios applicable to the technical solutions of the embodiments of the present application. It should be noted that the following introduced application scenarios are only used to illustrate the embodiments of the present application rather than limit them. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0058] The solution provided by the embodiments of the present application can be applied to the picture stream analysis scenario. As Figure 1 shown, it is an application scenario diagram provided by the embodiments of the present application. In this scenario, it includes a terminal device 101, a picture analysis device 102, and a management device 103.

[0059] The terminal device 101 can be any device capable of providing pictures to be analyzed and issuing picture stream analysis tasks. For example, it can be an Internet Protocol Camera (IPC), a mobile phone, a laptop computer, etc.

[0060] The picture analysis device 102 is used to execute picture stream analysis tasks and calculate the remaining resources of the current device. For example, it can be implemented by a computer or a server with certain computing capabilities.

[0061] The management device 103 is a device for task scheduling of multiple picture analysis devices 102. The management device 103 can receive picture stream analysis tasks and, according to the remaining resources of each picture analysis device and the estimated resources required for the picture stream analysis task, allocate the picture stream analysis task to the picture analysis device 102. For example, it can be implemented by a computer or a server with certain computing capabilities.

[0062] In a possible implementation manner, each picture analysis device 102 uses the device remaining resource reporting method provided by the embodiments of the present application to periodically report its remaining resources to the management device 103. The terminal device 101 issues a picture stream analysis task to the management device 103, and this task contains the picture stream data to be analyzed, such as the picture stream collected by a certain terminal device 101. After receiving this task, the management device 103 can use the picture analysis task scheduling method provided by the embodiments of the present application to allocate the picture stream analysis task to at least one picture analysis device 102 according to the remaining resources of each picture analysis device 102 and the estimated resources required for the picture stream analysis task.

[0063] The terminal device 101, the picture analysis device 102, and the management device 103 can be directly or indirectly communicatively connected through one or more networks. The network can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network. Of course, it can also be other possible networks, and the embodiments of the present application do not limit this.

[0064] Of course, the method provided by the embodiments of the present application is not limited to Figure 1 the application scenarios shown, and can also be used in other possible application scenarios, which are not limited by the embodiments of the present application. For Figure 1 the functions that can be achieved by each device in the application scenarios shown will be described together in the subsequent method embodiments, and will not be elaborated here.

[0065] See Figure 2 shown in the schematic diagram of the system architecture provided by the embodiments of the present application. Among them, the terminal device 101 includes an input module 201, the management device 103 includes a transceiver module 202 and a scheduling module 203, and the picture analysis device 102 includes a buffer module 204 and an analysis module 205.

[0066] (1) Input module 201

[0067] The input module 201 is used to obtain picture stream data and send a picture stream analysis task to the management device 103.

[0068] Specifically, the picture stream data can be directly collected by the terminal device 101 through monitoring, or can be indirectly obtained by the terminal device 101 from the network. The picture stream analysis task can include the picture stream data to be analyzed and the task objective, etc.

[0069] (2) Transceiver module 202

[0070] The transceiver module 202 is used to receive the picture stream analysis task from the terminal device 101 and allocate the picture stream analysis task to each picture analysis device 102.

[0071] Specifically, the transceiver module 202 responds to the picture stream analysis task from the input module 201, notifies the scheduling module 203 to perform scheduling, and after receiving the scheduling instruction from the scheduling module 203, sends the picture stream analysis task to each picture analysis device 102 according to the scheduling instruction. Among them, the scheduling instruction is used to indicate which picture analysis devices are responsible for executing the task and the task amount allocated to each picture analysis device 102.

[0072] (3) Scheduling module 203

[0073] The scheduling module 203 is used to determine the estimated resources for the picture stream analysis task and distribute the picture stream analysis task to at least one picture analysis device 102 according to the remaining resources of each picture analysis device.

[0074] Specifically, the scheduling module 203 will notify each picture analysis device to report its current remaining resources to itself periodically. When receiving a picture stream analysis task, it first determines the first coefficient of the picture stream data according to the reference value in the module. There is no limit on the value-taking method of the reference value. For example, it can directly take 1, or it can take the average value of the target coefficients corresponding to each processed picture within a preset time; it determines the corresponding second coefficient according to the resolution of each picture in the picture stream data; and then determines the estimated resources for the picture stream analysis task based on the first coefficient and the second coefficient, and sends a scheduling instruction to the transceiver module according to the estimated resources and the remaining resources of each picture analysis device.

[0075] (4) Buffer module 204

[0076] The buffer module 204 is used to receive and store the picture stream data from the transceiver module 202.

[0077] (5) Analysis module 205

[0078] The analysis module 205 is used to analyze the pictures in the buffer module 204, periodically calculate the remaining resources of the current device, and report the calculated remaining resources to the scheduling module 202.

[0079] Specifically, the analysis module 205 will record the target coefficient of each processed picture in the historical preset time, determine the first coefficient based on the target coefficients of each processed picture and the total number of processed pictures, that is, predict the target coefficient of the pictures not yet processed in the buffer module 204 through the target coefficients of the processed pictures in the recent time; and then determine the device remaining resources according to the second coefficient corresponding to the resolution of each unprocessed picture in the buffer module 204, the first coefficient, and the remaining space of the buffer module 204 and report it to the scheduling module 203.

[0080] The method steps executed by the above-mentioned each module will be specifically introduced in the subsequent embodiments, so there will be no more elaboration here.

[0081] Please refer to Figure 3 , which is a schematic flowchart of the method for reporting remaining resources and allocating picture stream analysis tasks provided by the embodiment of the present application. The process of this method is introduced as follows.

[0082] Step 301: Determine the first coefficient according to the target quantity included in each processed picture and the total quantity of the pictures within the historical preset time period.

[0083] Among them, the first coefficient is used to characterize the level of resources consumed by the picture analysis device for processing the processed pictures on average. That is to say, the first coefficient can characterize the average computing power required to process a single picture within a historical preset time period, and the first coefficient is positively correlated with the target quantity.

[0084] In the embodiments of the present application, the more the number of targets in a picture, the more resources are required for the picture analysis device to analyze the picture. Therefore, a coefficient corresponding to the number of targets in the picture can be set. The more the number of targets, the larger the corresponding coefficient. For the pictures that have not been processed in the picture analysis device, the picture analysis device cannot directly obtain the number of targets in the pictures. Since the scene conditions in adjacent pictures will not change much within a very short time, the number of targets in the unprocessed pictures can be predicted according to the number of targets in the pictures that have been processed by the picture analysis device recently.

[0085] Specifically, the number of targets in a picture and its corresponding coefficient can be as shown in Table 1. Taking the number of targets being 1 as the standard, when the number of targets in the picture is 1, the target coefficient of this picture is 1; when the number of targets in the picture is less than 1, the corresponding parameter value should be less than 1, that is, a1 should take a value greater than 0 and less than 1; when the number of targets in the picture is greater than 1, the corresponding parameter value should be greater than 1, that is, a2 - a5 increase in turn, and the values taken are all greater than 1.

[0086] For example, a picture analysis device has processed a total of F pictures within a historical preset time. According to Table 1, the target coefficients corresponding to each picture are obtained as a x1 …a xF , and calculate the average coefficient a x within the historical preset time for each picture:

[0087] a x =(a x1 +a x2 +…+a xF ) / F

[0088] a x can be used as the first coefficient for each unprocessed picture in the picture analysis device.

[0089] Number of targets Coefficient 0 <![CDATA[a1]]> 1 1 2 <![CDATA[a2]]> 3 <![CDATA[a3]]> 4 <![CDATA[a4]]> 5 <![CDATA[a5]]> >5 <![CDATA[a6]]>

[0090] Table 1

[0091] Step 302: Determine the second coefficient corresponding to each to-be-processed picture according to the resolution corresponding to each to-be-processed picture in the buffer queue.

[0092] Since the resolution of an image also has a significant impact on the resources required to process the image, the larger the resolution of an image, the more resources are consumed when analyzing the image. To calculate more precisely the resources required for an image, a second coefficient related to the resolution can be set for each image to represent the difficulty level of processing the image, and the second coefficient is positively correlated with the resolution of the image. In specific applications, the resolution of an image and its corresponding coefficient can be as shown in Table 2. The value of the second coefficient is based on 1080P. When the image resolution is 1080P, that is, the image size is 1920x1080, the second coefficient corresponding to the image is 1; when the image resolution is less than 1080P. As shown in Table 2 below, where D1 represents the resolution corresponding to the D1 standard, that is, the image size is 704x576, and its corresponding number is b2, HD1 represents the resolution that is half of the resolution corresponding to the D1 standard, that is, Half D1, and DVGA represents the resolution that is twice the resolution corresponding to the VGA standard, that is, Double VGA. When it is HD1 and DVGA, that is, when the image sizes are 352x576 and 960x640 respectively, the relationship of their corresponding second coefficients b1 - b3 is 0 < b1 < b2 < b3 < 1; when the image resolution is greater than 1080P, for example, when the image resolution is 3296x2472, its corresponding second coefficient b4 > 1.

[0093] In a picture analysis device, the resolution of each unprocessed picture can be obtained when receiving the picture stream data sent by the management device. The management device carries the resolution of each picture to be processed in the picture stream data, so that the picture analysis device can obtain the resolution of each unprocessed picture accordingly, and then based on the resolution of each unprocessed picture, by looking up the table, obtain the second coefficient corresponding to each unprocessed picture.

[0094] Resolution parameter Picture size Coefficient "HD1" 352x576 <![CDATA[b1]]> "D1" 704x576 <![CDATA[b2]]> "DVGA" 960x640 <![CDATA[b3]]> "1080P" 1920x1080 1 "3296x2472" 3296x2472 <![CDATA[b4]]>

[0095] Table 2

[0096] Step 303: Determine the estimated resources required for the picture analysis device to process each picture to be processed according to the obtained first coefficient and each second coefficient.

[0097] In specific applications, to accurately calculate the total estimated resources of the pictures to be processed, it is necessary to first calculate the sub - estimated resources required for each picture to be processed, and then determine the estimated resources required to process all pictures to be processed based on each sub - estimated resource.

[0098] Among them, the sub - estimated resources of the x - th picture to be processed in the buffer queue of the picture analysis device can be calculated through the following calculation method:

[0099] R x =a x*b x

[0100] Among them, R x represents the sub-estimated resource of the x-th picture to be processed, and a x is the first coefficient of the x-th picture to be processed, and b x is the second coefficient corresponding to this picture. Taking a picture with a resolution of 1080p and a target number of 1 as the standard, R x can also be understood as the resource required to process the x-th picture to be processed is R x times the resource required to process a standard picture.

[0101] If there are a total of W pictures to be processed in the buffer queue, the total estimated resource Q1 of this picture analysis device is:

[0102] Q1 = (R1 + R2 + … + R w )

[0103] Step 304: Determine the remaining resources of the picture analysis device according to the total resources and estimated resources of the picture analysis device.

[0104] In specific applications, the total resources of a picture analysis device are determined by its hardware configuration. That is to say, the total resources of a picture analysis device are fixed and have been fixed when the device leaves the factory. For example, in a picture analysis device, there are N intelligent sub-cards that can perform picture analysis. Each intelligent sub-card can process P standard pictures with a target number of 1 and a resolution of 1080 per second. The total resources Q0 of this picture analysis device are:

[0105] Q0 = N * P

[0106] Furthermore, the remaining resources of the picture analysis device can be obtained according to the obtained total resources Q0 and the above-mentioned estimated resources Q1.

[0107] In practical applications, since the space of the buffer queue in a picture analysis device is limited, when determining the remaining resources of the picture analysis device, it should also be considered whether the current buffer queue of the picture analysis device is full. If the maximum threshold of the buffer queue is L and the number of pictures in the current queue is W, the following several situations may occur:

[0108] (1) When W = L, it means that the current picture analysis device has no free space to store new pictures. Therefore, even if a new picture task is distributed, it cannot be stored, and thus the remaining resources can be determined to be zero.

[0109] (2) When W < L, it indicates that the picture analysis device still has space to store picture stream data, but the space is limited. Therefore, the remaining resources can be determined based on the remaining space of the buffer queue, the total resources, and the estimated resources.

[0110] Specifically, the calculation method of the remaining resource M can be:

[0111] M = (Q0 - Q1) * (1 - W / L)

[0112] The remaining resource can also be divided into two parameters M1 and M2, where:

[0113] M1 = Q0 - Q1

[0114] M2 = L - W

[0115] Upload the two parameters M1 and M2, so that the management device can know the remaining resources of the device based on the two parameters M1 and M2.

[0116] Step 305: The image analysis device reports the remaining resources to the management device.

[0117] Step 306: The management device receives the image stream analysis task from the front-end device.

[0118] In a specific application, an image stream analysis task from a front-end device may include image stream data and the task objective of the task. For example, the task objective may be to analyze the license plate numbers of all vehicles in the image stream data. In addition, the image stream analysis task may also include information such as the size and resolution of each image in the image stream data.

[0119] Step 307: Determine the estimated resources required to complete the image stream analysis task according to the resolution corresponding to each to-be-processed image in the image stream data and the total number of images.

[0120] In the embodiment of the present application, since the target quantity of each to-be-processed image is unknown before analysis, the management device can obtain the first coefficient of the received image stream data according to the reference value of the preset target quantity, and determine the corresponding second coefficient according to the resolution of each image in the image stream data, and calculate the estimated resources required to complete the image analysis stream task in the same way as in the above step 304.

[0121] In a specific application, the management device can directly take the reference value as 1, or record the information of each completed image stream analysis task, find the most recent image stream analysis task from the same device with the same task objective, and calculate the average value of the target coefficients according to the target numbers and the total number of images in the image stream data of this task as the reference value of this task.

[0122] Step 308: Select at least one image analysis device for executing the image stream analysis task from each image analysis device according to the remaining resources and the estimated resources of each image analysis device.

[0123] In the embodiments of the present application, the scheduling module generates a scheduling instruction according to the estimated resources required for the current picture stream analysis task and the remaining resources of each picture analysis device, and sends the scheduling instruction to the transceiver module. The scheduling instruction is used to indicate the tasks assigned to each picture stream analysis device; the transceiver module receives the picture stream analysis task of the front-end device and sends the tasks assigned to each picture analysis device to the device according to the scheduling instruction.

[0124] In specific applications, the scheduling module can allocate picture stream analysis tasks by any method. For example, the management device can allocate more tasks to picture analysis devices with more remaining resources and fewer tasks to picture analysis devices with less remaining resources, so that the remaining resources of all picture analysis devices are as consistent as possible; the management device can also allocate all tasks to the few devices with the most remaining resources, so that the pictures currently processed by each picture analysis device are as likely as possible to come from the same picture stream, thereby making the error between the first coefficient of the picture analysis device and the actual target coefficient of each picture in the buffer queue smaller.

[0125] Step 309: The management device issues the picture stream analysis task to the picture analysis device.

[0126] In specific applications, each picture analysis device puts the received picture stream data into its own buffer queue and processes it sequentially; after all the pictures in the data stream are processed, the results are reported to the management device. At the same time, the management device subscribes to the remaining resources from each picture analysis device, that is, each picture analysis device will periodically update its remaining resources according to the above steps 301-304 and report them to the management device.

[0127] Please refer to Figure 4 , based on the same inventive concept, the embodiments of the present application also provide a remaining resource reporting device, which includes:

[0128] The first coefficient determination unit 401 is used to determine a first coefficient according to the target quantity included in each processed picture and the total quantity of each picture in a historical preset time period. The first coefficient is used to characterize the level of resource consumption per processed picture by the picture analysis device, and the first coefficient is positively correlated with the target quantity;

[0129] The second coefficient determination unit 402 is used to determine a second coefficient corresponding to each to-be-processed picture according to the resolution corresponding to each to-be-processed picture in the buffer queue. The buffer queue is used to store pictures that have been assigned to the picture analysis device but not processed. The second coefficient is used to characterize the difficulty level of the picture analysis device in processing each to-be-processed picture, and the second coefficient is positively correlated with the resolution;

[0130] A resource estimation unit 403, configured to determine the estimated resources required for the picture analysis device to process each picture to be processed according to the obtained first coefficient and each second coefficient;

[0131] A remaining resource determination unit 404, configured to determine the remaining resources of the picture analysis device according to the total resources and the estimated resources of the picture analysis device, and report the remaining resources to the management device.

[0132] Optionally, the first coefficient determination unit 401 is specifically configured to:

[0133] Determine the target coefficient corresponding to each processed picture according to the number of targets included in each processed picture within a historical preset time period; the target coefficient is used to measure the level of resources consumed by the picture analysis device to process each processed picture; obtain the first coefficient according to the target coefficient corresponding to each picture and the total number of each picture.

[0134] Optionally, the resource estimation unit 403 is specifically configured to:

[0135] Determine the sub-estimated resources required for each picture to be processed respectively according to the second coefficient and the first coefficient corresponding to each picture to be processed; obtain the estimated resources based on the sub-estimated resources required for each picture to be processed.

[0136] Optionally, the remaining resource determination unit 404 is specifically configured to:

[0137] Determine the available resources of the picture analysis device according to the total resources and the estimated resources of the picture analysis device; determine whether the buffer queue is full, if it is full, determine that the remaining resources are zero; if there is free space in the buffer queue, obtain the remaining resources according to the available resources and the size of the free space in the buffer queue.

[0138] Optionally, the remaining resource determination unit 404 is further configured to:

[0139] Receive a picture stream analysis task sent by the management device, where the picture stream analysis task carries picture stream data; analyze each picture to be processed in the picture stream data respectively, and send the analysis results corresponding to each picture to be processed obtained to the management device.

[0140] This device can be used to execute Figure 3 the method executed by the image analysis device in the embodiment shown, therefore, for the functions that can be realized by each functional module of this device, reference can be made to Figure 3 the description of the embodiment shown, and details are not repeated here.

[0141] Please refer to Figure 5, Based on the same inventive concept, an embodiment of the present application further provides a picture analysis task scheduling device, which includes:

[0142] A subscription unit 501, configured to receive the remaining resources reported by each picture analysis device, where the remaining resources are the resources consumed by each picture analysis device for processing a single processed picture based on its own situation within a historical preset time period, and the processing difficulty levels of each to-be-processed picture in the cache queue of the picture analysis device;

[0143] A task receiving unit 502, configured to receive a picture stream analysis task, where the picture stream analysis task carries picture stream data including a plurality of to-be-processed pictures;

[0144] A resource prediction unit 503, configured to determine the estimated resources required to complete the picture stream analysis task according to the resolution of each to-be-processed picture and the total number of pictures in the picture stream data;

[0145] A scheduling unit 504, configured to select at least one picture analysis device for executing the picture stream analysis task from each picture analysis device according to the remaining resources of each picture analysis device and the estimated resources, and send the picture stream analysis task to at least one picture analysis device.

[0146] Optionally, the resource prediction unit 503 is specifically configured to:

[0147] Determine a first coefficient of the picture stream analysis task according to a reference value of a preset target coefficient, where the first coefficient is used to characterize the level of resources consumed by the picture analysis device for processing each to-be-processed picture on average; the reference value is determined based on a target number included in a plurality of processed pictures; determine a second coefficient corresponding to each to-be-processed picture according to the resolution of each to-be-processed picture, where the second coefficient is used to characterize the processing difficulty of the picture analysis device for processing each to-be-processed picture, and the second coefficient is positively correlated with the resolution; determine the sub-estimated resources required for each to-be-processed picture according to the first coefficient and the second coefficient corresponding to each to-be-processed picture; and determine the estimated resources of the picture stream analysis task based on the sub-estimated resources corresponding to each to-be-processed picture.

[0148] Optionally, the scheduling unit 504 is specifically configured to:

[0149] Select at least one picture analysis device for executing the picture stream analysis task according to the remaining resources and the estimated resources of each picture analysis device; generate a scheduling instruction, where the scheduling instruction indicates the picture stream analysis subtasks allocated to each picture analysis device in at least one picture analysis device; and send the corresponding picture stream analysis subtasks to each picture analysis device in at least one picture analysis device based on the scheduling instruction.

[0150] This device can be used to executeFigure 3 The method executed by the management device in the illustrated embodiment. Therefore, for the functions that can be achieved by each functional module of the device, reference can be made to Figure 3 the description of the illustrated embodiment, which will not be elaborated here.

[0151] Please refer to Figure 6 , based on the same inventive concept, an embodiment of the present application further provides a computer device 60, which may include a memory 601 and a processor 602.

[0152] The memory 601 is used to store the computer program executed by the processor 602. The memory 601 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function, etc.; the data storage area may store data created according to the use of the computer device, etc. The processor 602 may be a central processing unit (CPU) or a digital processing unit, etc. In the embodiment of the present application, the specific connection medium between the above-mentioned memory 601 and the processor 602 is not limited. In the embodiment of the present application Figure 6 , the memory 601 and the processor 602 are connected through a bus 603, and the bus 603 is represented by a thick line in Figure 6 . The connection manners between other components are only for illustrative purposes and are not limited thereto. The bus 603 may be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 6 only a thick line is used to represent it in

[0153] , but it does not mean that there is only one bus or one type of bus.

[0154] The processor 602, when calling the computer program stored in the memory 601, executes the method executed by the device in the embodiments shown in Figure 2 and Figure 3 .

[0155] In some possible embodiments, aspects of the methods provided in this application may also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to cause the computer device to execute the steps in the methods according to various exemplary embodiments of this application described above in this specification. For example, the computer device may execute the method performed by the device in the embodiment shown as Figures 2 to 4 shown in the embodiments.

[0156] The program product may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0157] Although the preferred embodiments of this application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0158] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and modifications.

Claims

1. A method for reporting remaining resources, characterized in that, Applied to an image analysis device, the method includes: Determine a first coefficient according to the number of targets included in each processed image and the total number of the processed images within a historical preset time period. The first coefficient is used to characterize the level of resource consumption for the image analysis device to process the processed images on average, and the first coefficient is positively correlated with the number of targets and negatively correlated with the total number. Determine a second coefficient corresponding to each to-be-processed image according to the resolution of each to-be-processed image in the buffer queue. The buffer queue is used to store images that have been allocated to the image analysis device but not processed. The second coefficient is used to characterize the difficulty level for the image analysis device to process each to-be-processed image, and the second coefficient is positively correlated with the resolution. Determine the estimated resources required for the image analysis device to process the to-be-processed images according to the obtained first coefficient and each second coefficient. Determine the remaining resources of the image analysis device according to the total resources of the image analysis device and the estimated resources, and report the remaining resources to the management device.

2. The method according to claim 1, wherein The step of determining a first coefficient according to the number of targets included in each processed image and the total number of the processed images within a historical preset time period includes: Correspondingly determine a target coefficient corresponding to each processed image according to the number of targets included in each processed image within a historical preset time period. The target coefficient is used to measure the level of resource consumption required for the image analysis device to process each processed image. Obtain the first coefficient according to the target coefficients corresponding to the processed images and the total number of the processed images.

3. The method according to claim 1, characterized in that, The step of determining the estimated resources required for the image analysis device to process the to-be-processed images according to the obtained first coefficient and each second coefficient includes: Respectively determine the sub-estimated resources required for each to-be-processed image according to the second coefficient corresponding to each to-be-processed image and the first coefficient. Obtain the estimated resources based on the sub-estimated resources required for each to-be-processed image.

4. The method according to claim 1, characterized in that, The step of determining the remaining resources of the image analysis device according to the total resources of the image analysis device and the estimated resources includes: Determine the available resources of the image analysis device according to the total resources of the image analysis device and the estimated resources. Determine whether the buffer queue is full. If it is full, determine that the remaining resources are zero. If there is free space in the buffer queue, obtain the remaining resources according to the available resources and the size of the free space in the buffer queue.

5. The method according to claim 1, characterized in that, The method further includes: Receive an image stream analysis task sent by the management device. The image stream analysis task carries image stream data. Analyze each to-be-processed image in the image stream data respectively, and send the analysis results corresponding to each to-be-processed image obtained to the management device.

6. A method for scheduling picture analysis tasks, characterized in that, Applied to a management device, the method includes: Receiving the remaining resources reported by each image analysis device, where the remaining resources are the resources that each image analysis device needs to consume based on its own consumption of a single processed image during a historical preset time period and the processing difficulty levels of the to-be-processed images in the cache queue of the image analysis device; Receiving an image stream analysis task, where the image stream analysis task carries image stream data including a plurality of to-be-processed images; Determining the estimated resources required to complete the image stream analysis task according to the resolutions and the total number of images corresponding to the to-be-processed images in the image stream data; Selecting at least one image analysis device to execute the image stream analysis task from the various image analysis devices according to the remaining resources of the various image analysis devices and the estimated resources, and sending the image stream analysis task to the at least one image analysis device.

7. The method according to claim 6, wherein Determining the estimated resources required to complete the image stream analysis task according to the resolutions and the total number of images corresponding to the to-be-processed images in the image stream data, including: Determining a first coefficient of the image stream analysis task according to a reference value of a preset target coefficient, where the first coefficient is used to characterize the level of resource consumption on average by the image analysis device for processing the to-be-processed images; the reference value is determined based on a target quantity included in a plurality of processed images; Determining a second coefficient corresponding to each of the to-be-processed images according to the resolutions of the to-be-processed images, where the second coefficient is used to characterize the processing difficulty level of the image analysis device for processing the to-be-processed images, and the second coefficient is positively correlated with the resolution; Determining the sub-estimated resources required for each of the to-be-processed images according to the first coefficient and the second coefficient corresponding to each of the to-be-processed images; Determining the estimated resources of the image stream analysis task based on the sub-estimated resources corresponding to each of the to-be-processed images.

8. The method according to claim 6, characterized in that, The management device includes a scheduling module and a transceiver module; Selecting at least one image analysis device to execute the image stream analysis task from the various image analysis devices according to the remaining resources of the various image analysis devices and the estimated resources, and sending the image stream analysis task to the at least one image analysis device, including: Selecting at least one image analysis device to execute the image stream analysis task through the scheduling module according to the remaining resources and the estimated resources of the various image analysis devices; Sending a scheduling instruction to the transceiver module through the scheduling module, where the scheduling instruction indicates the image stream analysis subtasks allocated to each image analysis device in the at least one image analysis device; Sending the corresponding image stream analysis subtasks to each image analysis device in the at least one image analysis device respectively through the transceiver module based on the scheduling instruction.

9. A remaining resource reporting device, characterized in that, Applied to an image analysis device, including: A first coefficient determination unit, configured to determine a first coefficient according to the number of targets included in each processed picture and the total number of the processed pictures within a historical preset time period, where the first coefficient is used to characterize the level of resource consumption on average by the picture analysis device for processing the processed pictures, and the first coefficient is positively correlated with the number of targets and negatively correlated with the total number; A second coefficient determination unit, configured to determine a second coefficient corresponding to each to-be-processed picture according to the resolution corresponding to each to-be-processed picture in a buffer queue, where the buffer queue is used to store pictures that have been allocated to the picture analysis device but not processed, and the second coefficient is used to characterize the difficulty level of the picture analysis device for processing each to-be-processed picture, and the second coefficient is positively correlated with the resolution; A resource estimation unit, configured to determine the estimated resources required by the picture analysis device for processing each to-be-processed picture according to the obtained first coefficient and each second coefficient; A remaining resource determination unit, configured to determine the remaining resources of the picture analysis device according to the total resources of the picture analysis device and the estimated resources, and report the remaining resources to a management device.

10. An apparatus for scheduling picture analysis tasks, characterized in that, Applied to a management device, including: A subscription unit, configured to receive the remaining resources reported by each picture analysis device, where the remaining resources are obtained by each picture analysis device based on the resources required for processing a single processed picture by itself within a historical preset time period and the difficulty level of processing each to-be-processed picture in the cache queue of the picture analysis device; A task receiving unit, configured to receive a picture stream analysis task, where the picture stream analysis task carries picture stream data including a plurality of to-be-processed pictures; A resource prediction unit, configured to determine the estimated resources required to complete the picture stream analysis task according to the resolution corresponding to each to-be-processed picture in the picture stream data and the total number of pictures; A scheduling unit, configured to select at least one picture analysis device for executing the picture stream analysis task from the various picture analysis devices according to the remaining resources of the various picture analysis devices and the estimated resources, and send the picture stream analysis task to the at least one picture analysis device.

11. A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that when the processor executes the computer program, the steps of the method according to any one of claims 1-5 or 6-8 are implemented.

12. A computer storage medium, on which computer program instructions are stored, characterized in that when the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1-5 or 6-8 are implemented.

Images