Computing power resource allocation method, computer device, and storage medium

Abstract

The application discloses a computing power resource allocation method, a computer device and a storage medium. The method comprises the following steps: obtaining a target set of processing tasks to be executed, wherein the target set comprises K target objects in a target video, K is an integer greater than 1; using attribute information of each target object in the K target objects to perform processing priority sorting on each target object; and according to the processing priority sorting of each target object, allocating computing power resources to each target object to execute processing tasks for each target object. The above scheme can reasonably allocate computing power resources and improve the use efficiency of the computing power resources.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method for allocating computing resources, a computer device, and a computer-readable storage medium. Background Technology

[0002] With the continuous development of image processing technology, it has been applied to various application scenarios, such as intelligent transportation, surveillance, and medical imaging. Taking target recognition tasks in surveillance scenarios as an example, target recognition tasks typically require significant computing resources. In existing technologies, the limited computing resources are usually evenly distributed among each target object to perform the target recognition task.

[0003] Due to limited computing resources, insufficient computing power may lead to inaccurate results in target recognition tasks. Furthermore, allocating the same computing resources to the same target object without any changes will waste these resources, resulting in low efficiency in image processing. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a method for allocating computing resources, computer equipment, and storage media, which can rationally allocate computing resources and improve the efficiency of computing resource utilization.

[0005] To address the aforementioned issues, the first aspect of this application provides a method for allocating computing resources. This method includes: obtaining a target set for processing tasks, the target set including K target objects in a target video, where K is an integer greater than 1; sorting the processing priorities of each target object using attribute information of each of the K target objects; and allocating computing resources to each target object according to the sorting of processing priorities to perform processing tasks on each target object.

[0006] To address the aforementioned problems, a second aspect of this application provides a computer device comprising a memory and a processor coupled to each other, wherein the memory stores program data and the processor executes the program data to implement any step of the aforementioned method for allocating computing resources.

[0007] To address the aforementioned problems, a third aspect of this application provides a computer-readable storage medium storing program data executable by a processor, the program data being used to implement any step of the aforementioned method for allocating computing resources.

[0008] The above scheme obtains a target set for the processing tasks, which includes K target objects in the target video. It then performs a comprehensive analysis of the attribute information of each of these K target objects to prioritize their processing. This allows for the determination of the importance or priority of each target object in relation to the processing task. Next, according to the priority of each target object, computing resources are allocated to each object to execute the processing task. Different computing resources can be allocated to target objects with different processing priorities, ensuring that more important target objects receive priority and more computing resources, while other target objects receive less. This approach allows for the rational allocation of computing resources to each target object when computing resources are limited, improving the accuracy of processing tasks and reducing waste. It also improves the efficiency of computing resource utilization by rationally allocating limited computing resources to target objects with different processing priorities. Furthermore, the ability to rationally allocate computing resources to target objects with different processing priorities also improves the overall processing efficiency of the target object processing tasks. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in this application, the accompanying drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Among them:

[0010] Figure 1 This is a flowchart illustrating an embodiment of the computing power resource allocation method of this application;

[0011] Figure 2 This is a flowchart illustrating another embodiment of the computing resource allocation method of this application;

[0012] Figure 3 This application Figure 1 A flowchart illustrating an embodiment of step S12;

[0013] Figure 4 This application Figure 3 A flowchart illustrating an embodiment of step S121;

[0014] Figure 5 This application Figure 3 A flowchart illustrating another embodiment of step S121;

[0015] Figure 6 This application Figure 1 A flowchart illustrating an embodiment of step S13;

[0016] Figure 7 This is a schematic diagram of the structure of an embodiment of the computing power resource allocation device of this application;

[0017] Figure 8 This is a schematic diagram of the structure of an embodiment of the computer device of this application;

[0018] Figure 9 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] The terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0021] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0023] Through long-term research, the inventors of this application have discovered that when allocating computing resources for processing target objects in a video stream, taking target recognition as an example, the same computing resources are usually allocated to each target object in each frame of the image to perform the processing task.

[0024] However, for moving targets, limited computing resources may cause errors in target recognition, while for stationary targets, allocating the same computing resources will result in repeated target recognition processing, which will waste computing resources.

[0025] To address the aforementioned technical problems, this application provides the following embodiments, which are described in detail below.

[0026] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the computing resource allocation method of this application. The method may include the following steps:

[0027] S11: Obtain the target set for the processing task to be performed. The target set includes K target objects in the target video, where K is an integer greater than 1.

[0028] The target video can be a video stream obtained by shooting a target area using a camera device. The specific content of the aforementioned target area is not limited; those skilled in the art can set it according to actual needs. For example, the target area can be, but is not limited to, any one or more of the following: shopping mall, store, entrance / exit of a certain area, street venue, hospital, station, waiting room, airport, etc.; the aforementioned target video can contain one or more video image frames.

[0029] In some implementations, each frame of the target video may contain multiple target objects. This application uses the example of containing multiple or more target objects for illustration, but does not impose any limitations on this.

[0030] In some implementations, the target object can be any subject that may move relative to the camera device. For example, the target object can be a vehicle, a person, an animal, or even a license plate, a vehicle body, a face, etc.

[0031] In some implementations, the target object can be a target of interest in the application scenario. In actual application, the specific physical object referred to by the target object can be determined according to actual needs. For example, in scenarios such as banks and shopping malls, the target object can refer to people, faces, etc.; in scenarios such as roads and bridges, the target object can refer to vehicles and pedestrians; in scenarios such as forests and grasslands, the target object can refer to animals. This application does not impose any restrictions on this.

[0032] In some implementations, each image frame in the target video can be acquired sequentially, and each target object in each image frame can be identified. The target objects contained in each image frame are then used as the target set for the processing task. The target set includes K target objects in the target video, where K is an integer greater than 1. The algorithm for identifying each target object in each image frame can be a target recognition algorithm, capable of detecting the bounding box, contour features, and other information of each target object in each image frame.

[0033] In some implementations, the K target objects can be target objects in the same image frame or target objects in multiple image frames. This application will use the example of K target objects being target objects in the same image frame for illustration.

[0034] In some implementations, the processing task is to process each target object contained in a series of consecutive image frames. For example, the object state of each target object can be obtained, wherein the object state of the target object can change over time or space (e.g., the position, pose / posture, behavior, etc. of the target object). That is, in each image frame, the target object may have different object states. Detecting each frame requires a lot of computing resources, such as performing target localization, target recognition, target tracking, pose / posture detection, etc., on a series of consecutive images.

[0035] For example, the processing tasks can include target recognition, target localization, target tracking, target behavior recognition, intent recognition, violation detection, collision warning, and other related tasks. It is understood that the processing tasks referred to in this application may also include other processing tasks that consume computing resources to process the target object, and this application does not impose any restrictions on this.

[0036] Taking a person as the target object, the processing tasks can include target localization, target tracking, posture / pose detection, human body recognition, and target behavior recognition. Taking a vehicle as the target, the processing tasks can include vehicle recognition, license plate recognition, and vehicle attribute recognition (such as vehicle model and body color). This application does not limit the target object or the processing tasks.

[0037] As an example, let's take the task of target behavior recognition as an example. For a target, such as a person, the behavior of the target may be different in each frame of a series of image frames. The task of detecting human behavior is usually quite computationally intensive. Similar processing tasks can adopt the computational power allocation strategy described above in this application.

[0038] S12: Utilize the attribute information of each of the K target objects to sort the processing priority of each target object.

[0039] In order to complete the processing tasks of the target object under the condition of limited computing resources, the target object can be prioritized for processing, so that subsequent processing tasks of the target object are allocated computing power according to this priority.

[0040] It can obtain the attribute information of each of the K target objects. The attribute information can be the factors that affect the processing task of the target object. These factors can be modeled as the attribute information of the target object to be processed in the processing task. Different attribute state values ​​can be assigned to each factor, that is, the possible attribute states of each factor can be modeled. Thus, the processing priority of each target object can be sorted according to the magnitude of the attribute state value corresponding to the attribute state.

[0041] In some implementations, the attribute information includes at least M attributes, where M is an integer greater than 1. The M attributes may include at least one of the following: attribute information including computational power consumption factors for performing the processing task; attribute information including processing result factors for performing the processing task (e.g., behavior identification); attribute information including object state factors of the target object for performing the processing task (e.g., position, posture); and attribute information including historical processing information for performing the processing task. This application does not limit the attribute information.

[0042] For example, the computational power consumption factor for performing a processing task can be the motion state. As an example, performing a target localization task on a target object can be affected by the target's motion state / motion form, and the computational resources required for localization of target objects in different motion states will vary. Motion state / motion form and other factors can be used as attribute information, for example, for target object recognition. For instance, the localization of vehicles / people can be affected by the size or distance of the target in an image frame. Target objects that are closer receive more attention, and the distance of the target is more important to the target localization processing task, while the target's attributes (such as clothing) are less important. Factors influencing the degree of importance (such as distance) can also be used as attribute information.

[0043] It should be noted that the required attribute information and its priority may differ for different processing tasks. The attribute values ​​and priorities modeled within the attribute information may also differ. Attribute information is task-dependent; different processing tasks may involve completely different, partially the same, or completely identical attribute information. This application does not impose any restrictions on this.

[0044] In some implementations, the processing priority of each target object can be ranked based on the importance of multiple attribute information. For example, each attribute information can contain multiple attribute states, and each attribute state corresponds to an attribute state value, thereby obtaining the attribute values ​​of each target object's attribute information. The processing priority of each target object can be ranked according to the attribute values ​​of its attribute information. For example, the target objects can be prioritized based on the value of the first attribute. Then, among target objects with the same value of the first attribute, the processing priority can be prioritized based on the value of the second attribute. Furthermore, among target objects with the same values ​​of both the first and second attributes, the processing priority can be prioritized based on the value of the third attribute. And so on, ultimately obtaining a processing priority ranking based on all attribute information.

[0045] In some embodiments, please refer to Figure 2 Before step S12, that is, before prioritizing the processing of each target object using the attribute information of each of the K target objects, the following steps may be included:

[0046] S111: Sort the M attribute information based on the importance of each attribute information; the importance is the degree of importance of the corresponding attribute information relative to the processing task.

[0047] The factors influencing the fulfillment of the processing task's requirements can be modeled as attribute information within the processing task. Since these attributes are related to the processing task, the importance of each of the M attributes relative to the processing task can be obtained as its corresponding importance score. Alternatively, the computational power consumption of each attribute relative to the processing task can be obtained as its corresponding importance score. Furthermore, the M attributes can be sorted according to their importance scores to obtain their attribute ranking.

[0048] In some implementations, taking the processing task as an example of a target localization task, this processing task can be a task that locates various target objects. The attribute information includes at least one of the following: motion state, historical processing status, and current device distance, where the current device distance is the distance between the target object and the camera device, and the historical processing status indicates the status of the target object in historical image frames during which the processing task was performed. The attribute information can be ordered in the following order: motion state, historical processing status, and current device distance.

[0049] In some implementations, taking the processing task as a behavior recognition task as an example, this processing task can identify the behavior of each target object, such as whether the target object is engaging in dangerous behavior. Attribute information may include at least one of the following: target behavior type, vehicle distance, number of vehicles, etc., where vehicle distance is the distance between the target vehicle and the target object, and the number of vehicles is the number of target vehicles, where the target vehicles are those that pose a safety threat to the target object. The attribute information can be ordered as follows: target behavior type, vehicle distance, and number of vehicles.

[0050] S112: Perform state modeling on the sorted attribute information, and define the attribute state value for each attribute state according to the importance of each attribute state in each attribute information; the importance is the degree of importance of the corresponding attribute state relative to the processing task.

[0051] After sorting the M attribute information, state modeling can be performed on the sorted attribute information, that is, to obtain the attribute state contained in each attribute information, and to obtain the importance of each attribute state relative to the processing task or the computing power consumption, so as to define the attribute state value for each attribute state.

[0052] Among them, the attribute state values ​​corresponding to each attribute state of the attribute information can be the same, different, or partially the same, and this application does not impose any restrictions on this.

[0053] In some implementations, the task is located with the goal of processing the task. The attribute information includes at least one of the following: motion state, historical processing information, and current device distance.

[0054] The motion state includes attribute states such as turning, going straight, and stopping. Turning can include left or right turn. State modeling of the motion state allows for the definition of different attribute state values ​​for each attribute state. The attribute state values ​​corresponding to the motion state can be discrete values. The attribute state values ​​for each attribute state can be defined as a, b, and c, where the values ​​of a, b, and c decrease sequentially, and a, b, and c are natural numbers greater than or equal to 0. For example, a, b, and c are 2, 1, and 0, respectively. This application does not impose restrictions on the attribute state values ​​corresponding to different motion states.

[0055] When the target object is in the "turning" attribute state, the target object's position changes significantly and is accompanied by a change in posture. Using auxiliary positioning may lead to a large error. Therefore, the attribute state can be set to have the highest priority for the target positioning task, and the attribute state value of the "turning" attribute state can be defined as 2.

[0056] When the target object is in the "straight ahead" attribute state, the position of the target object changes, but the pose remains unchanged. Using auxiliary positioning will not introduce a large error, and the attribute state value of the "straight ahead" attribute state can be defined as 1.

[0057] When the target object is in the "stopped" attribute state, that is, the target object is stationary, the position of the target object does not change, and the posture does not change, accurate positioning can be obtained by using auxiliary positioning. The attribute state value of the "stopped" attribute state can be defined as 0.

[0058] The historical processing status includes whether the target object was processed using the first preset processing method in the historical image frame, that is, whether target recognition was used to process the task in the historical image frame. The historical image frame can be the previous image frame of the current image frame. Since different preset processing methods require different amounts of computing resources, different historical processing statuses correspond to different attribute state values, which can be represented by discrete values.

[0059] For example, the attribute states of historical processing can be defined as d and e respectively for "the first preset processing method was used in the historical image frame" and "the first preset processing method was not used in the historical image frame", where d is less than e and d and e are natural numbers greater than or equal to 0. For example, d and e are 0 and 1 respectively. This application does not restrict the attribute state values ​​corresponding to different attribute states of historical processing.

[0060] Regarding the current device distance of a target object, the level of attention given to each target object varies within an image frame. For example, a target object closer to the camera in an image frame receives more attention than one further away. Since targets closer to the camera are typically of greater interest to the localization task, and due to perspective (objects appearing larger when closer and smaller when farther away) in an image frame, the pixel changes in the image frame will be more pronounced when the target object moves the same distance on the ground plane. Therefore, more frequent localization is required. Thus, targets closer to the camera can be given higher priority; that is, the closer the target object is to the camera in the image frame, the higher its distance attribute value.

[0061] The distance attribute value corresponding to the current device distance can be represented by a continuous value. The attribute status value corresponding to the current device distance of the target object is the distance between the target object and the preset position point in the current image frame of the target video. Here, the preset position point is the position point of the camera device, or the preset position point is the position point corresponding to the shooting area in the current image frame. The shooting area can be the center area of ​​the lower part of the image frame.

[0062] In some implementations, the attribute status value corresponding to the current device distance can be defined as the height of the target object in the current image frame, that is, it can be defined using the vertical coordinate on the image frame.

[0063] In some implementations, this application may also obtain other attribute information of the target object and attribute values ​​of various states corresponding to each attribute information. Various attribute information and attribute values ​​can be set according to specific needs or specific application scenarios, and this application does not limit them.

[0064] In some implementations, the attribute information of the target object includes at least one of the following: motion state, historical processing status, and current device distance, and the attribute values ​​corresponding to each attribute information can be represented as:

[0065] M={2,1,0}; F={1,0}; C={x|0≤x <H}。

[0066] Where M represents the attribute state value corresponding to the motion state, F represents the attribute state value corresponding to the historical processing situation, C represents the attribute state value corresponding to the current device distance, H represents the height of the current image frame, and x represents the attribute state value corresponding to the current device distance. The attribute state value can characterize the importance of the target object.

[0067] We perform state modeling on the possible states of the above-mentioned attribute information, so that we can subsequently rank them according to the importance of the attribute state values ​​of these states to the target localization task.

[0068] In some implementations, taking the task of behavior recognition as an example, the attribute information may include at least one of the following: target behavior type, vehicle distance, number of vehicles, etc. Similarly, attribute state values ​​can be defined for each attribute state based on the importance of each attribute state.

[0069] The attribute states of the target behavior type include: running a red light, climbing over a guardrail, falling, and walking normally. Different attribute states of the target behavior type correspond to different attribute state values. For example, the more dangerous the attribute state, the higher the attribute state value is defined, that is, the higher the priority. The attribute state value defined for running a red light can be 3, the attribute state value defined for climbing over a guardrail can be 2, the attribute state value defined for falling can be 1, and the attribute state value defined for walking normally can be 0.

[0070] In this context, the attribute status value corresponding to the vehicle distance of the target object is the distance description value between the target object and the target vehicle. The distance description value is inversely proportional to the distance. Vehicle distance can be represented as the distance to vehicles that may pose a safety threat to the target object. Since the closer the distance, the greater the danger, different distance description values ​​can be set for different vehicle distances. The distance description value is inversely proportional to the distance; that is, the closer the distance, the larger the attribute status value. The ratio of a preset distance (which can be considered a safe distance) to the vehicle distance can be obtained, and the attribute status value (distance description value) can be determined based on this ratio. Alternatively, the inverse ratio of the vehicle distance (e.g., 1 / vehicle distance) can be directly used as the distance description value or attribute status value. Alternatively, different distance ranges can be set with corresponding distance description values, each range corresponding to one distance description value, which can then be defined as the attribute status value corresponding to the vehicle distance.

[0071] In this application, the attribute state value corresponding to the number of vehicles in the target object is a quantity description value, which is directly proportional to the number of target vehicles. The number of vehicles can represent the number of vehicles that may pose a security threat to the target. Different attribute state values ​​can be set for different numbers, where the attribute state value or the quantity description value is directly proportional to the number of target vehicles; that is, the more vehicles there are, the larger the quantity description value. The number of target vehicles can be directly normalized, and the normalized value can be used as the quantity description value. Alternatively, based on the number range to which the number of target vehicles belongs, the quantity description value corresponding to each range can be obtained, with each range corresponding to a single quantity description value. For example, the quantity range 0-1 corresponds to a quantity description value (i.e., attribute state value) of 0, the quantity range 1-5 corresponds to an attribute state value of 1, and the quantity range 5-10 corresponds to an attribute state value of 2, etc. This application is not limited to these attribute information and attribute states for the target behavior recognition task.

[0072] In some embodiments, please refer to Figure 3 This embodiment can further extend step S12 of the above embodiment. Utilizing the attribute information of each of the K target objects, the processing priority of each target object is sorted. This embodiment may include the following steps:

[0073] S121: Obtain the attribute values ​​corresponding to the M attribute information of each target object; where the attribute values ​​represent the importance of the target object.

[0074] It can obtain the attribute states corresponding to M attribute information of each target object, and then use the attribute state value that matches each attribute state of the M attribute information as the attribute value of each attribute information. The attribute value includes the sum of the attribute state values ​​corresponding to each attribute state of the attribute information. For example, if the target object contains one attribute state, then the attribute state value matching that attribute state is used as the attribute value of each attribute information. If the target object contains multiple attribute states, then the sum of the attribute state values ​​matching the multiple attribute states is used as the attribute value of each attribute information.

[0075] In some implementations, the attribute information includes at least one of the following: motion state, historical processing status, and current device distance, where the current device distance is the distance between the target object and the camera device in the current image frame, and the historical processing status indicates the status of the target object performing processing tasks in historical image frames.

[0076] In some embodiments, please refer to Figure 4 Taking the task of target localization as an example, step S121 of the above embodiment can be further extended. Obtaining the attribute values ​​corresponding to the M attribute information of each target object in step S121 may include any one or more of the following steps:

[0077] S211: Obtain the attribute state of the motion state of each target object, and from the several attribute state values ​​corresponding to the attribute state of the motion state, obtain the attribute state value that matches the attribute state of the motion state of each target object, and use it as the attribute value corresponding to the motion state.

[0078] Among them, the values ​​of several attribute states corresponding to the motion state are all different.

[0079] For example, if the attribute state of the target object's motion state is "turning", then the attribute state value matching the motion state is 2; if the attribute state of the target object's motion state is "going straight", then the attribute state value matching the motion state is 1; and if the attribute state of the target object's motion state is "stopped", then the attribute state value matching the motion state is 0. In this way, the attribute values ​​corresponding to the motion state of each target object in each frame of the image can be obtained.

[0080] In some implementations, the motion state of a target object in the current image frame can be obtained using multiple image frames (the current image frame and at least one historical image frame). For example, scene matching can be performed on adjacent image frames (the current image frame and at least one historical image frame) to obtain the motion state of each target object between adjacent image frames. Alternatively, the position of the localization task or the position of the detection box of each target object in multiple image frames can be used, and the motion state of each target object can be obtained by the change in the position of the localization task or the change in the position of the detection box.

[0081] S212: Obtain the attribute status of the historical processing status of each target object, and from the several attribute status values ​​corresponding to the attribute status of the historical processing status, obtain the attribute status value that matches the attribute status of the historical processing status of each target object, and use it as the attribute value corresponding to the historical processing status.

[0082] For example, if the attribute status of the target object's historical processing is "the first preset processing method was used in the historical image frame", the attribute status value matching the target object's historical processing is 0. If the attribute status of the target object's historical processing is "the first preset processing method was not used in the historical image frame", the attribute status value matching the target object's historical processing is 1. In this way, the attribute status value corresponding to the historical processing of each target object can be obtained.

[0083] S213: Obtain the coordinate information of each target object, and use the distance between the coordinate information of each target object and the preset position point in the current image frame of the target video as the attribute value corresponding to the current device distance of each target object.

[0084] It can obtain the coordinate information of each target object in the current image frame of the target video, and can obtain the distance value between the coordinate information of each target object and the preset position point in the current image frame of the target video. For example, the vertical coordinate of the target object's coordinate information can be used as the distance value, and the distance value can be used as the attribute value corresponding to the current device distance of the target object.

[0085] Using the above method, the attribute values ​​of each target object in the image frame of the target image can be obtained.

[0086] In some embodiments, please refer to Figure 5 Taking a behavior recognition task as an example, step S121 of the above embodiment can be further extended. Obtaining the attribute values ​​corresponding to the M attribute information of each target object in step S121 may include any one or more of the following steps:

[0087] S221: Obtain the attribute state of the target behavior type of each target object, and from the several attribute state values ​​corresponding to the attribute state of the target behavior type, obtain the attribute state value that matches the attribute state of the target behavior type of each target object, and use it as the attribute value corresponding to the target behavior type.

[0088] This method can obtain the attribute status of the target behavior type of each target object in the current image frame of the target video. If the attribute status includes running a red light, then the attribute status value 3 matching running a red light is obtained as the attribute value; if the attribute status includes climbing over a guardrail, then the attribute status value 2 matching climbing over a guardrail is obtained as the attribute value; if the attribute status includes falling, then the attribute status value 1 matching falling is obtained as the attribute value; if the attribute status includes walking normally, then the attribute status value 0 matching walking normally is obtained as the attribute value. This method allows the acquisition of the attribute values ​​corresponding to the target behavior type of each target object.

[0089] S222: Obtain the distance between each target object and the target vehicle, and use the distance description value corresponding to the distance as the attribute value corresponding to the vehicle distance.

[0090] The system obtains the distance between the target object and the target vehicle in the current image frame of the target video, as well as the corresponding distance description value, and uses this distance description value as the attribute value corresponding to the vehicle distance. The target vehicle is the vehicle that poses a security threat to the target object.

[0091] S223: Obtain the number of target vehicles corresponding to each target object, and use the quantity description value corresponding to the quantity as the attribute value corresponding to the vehicle quantity.

[0092] The system retrieves the number of target vehicles corresponding to each target object in the current image frame of the target video, along with a corresponding quantity description value. This quantity description value is then used as the attribute value corresponding to the vehicle quantity. Here, a target vehicle is defined as a vehicle that poses a security threat to the target object.

[0093] S122: Sort the processing priority of each target object according to the attribute values ​​corresponding to the attribute information of each target object.

[0094] There are M attributes, and the M attributes of the target object have different attribute values.

[0095] In some implementations, the attribute ranking of M attribute information is obtained. The attribute ranking can be the same as the ranking of the importance of the M attribute information described above. In some application scenarios, the attribute ranking can also be obtained based on the following steps: Each attribute information can be assigned a different computational power consumption level. For example, the computational power consumption level is related to the importance of the attribute information to the processing task; the higher the importance to the processing task, the higher the computational power consumption level of the attribute information. For example, if the computational power required for processing a certain attribute information varies, the computational power consumption level of the attribute information will also be different. For example, motion states consume more computational power for processing tasks, and the set computational power consumption level is also higher. When ranking multiple attribute information, the computational power consumption level of each attribute information with respect to the processing task can be obtained, and the multiple attribute information can be ranked using the computational power consumption levels corresponding to each attribute information.

[0096] In some implementations, the attribute information that ranks first in the current attribute sort can be used as the target attribute information, that is, the motion state can be used as the target attribute information. Then, according to the magnitude of the attribute value corresponding to the target attribute information, the objects to be processed in this round are prioritized, where the objects to be processed in the first round are the target objects. The objects to be processed with the same attribute value corresponding to the target attribute information are found from the priority sort and are used as the objects to be processed in the new round. The next attribute information in the current attribute sort is used as the target attribute information, and the steps of using the attribute information that ranks first in the current attribute sort as the target attribute information and subsequent steps are repeated until the attribute values ​​corresponding to the target attribute information are all different, or the target attribute is the last attribute information in the current attribute sort, that is, the sorting of all attribute information is completed.

[0097] For example, taking the task of locating the target object as the task of processing, the motion state can be used as the target attribute information. The target objects are sorted according to the motion attribute values ​​of the motion state. If multiple target objects have the same motion attribute value, they are sorted according to the processing attribute value of the historical processing situation. If multiple target objects have the same motion attribute value and processing attribute value, they are sorted according to the distance attribute value of the current device distance. This process continues until all attribute information is sorted. The final sort is used as the priority sort for processing each target object.

[0098] As an example, the current image frame includes multiple target objects. The attribute information and values ​​of each target object are obtained. For instance, the attribute values ​​for target A are represented as: M=2, F=1, C=300; the attribute values ​​for target B are represented as: M=2, F=1, C=200; the attribute values ​​for target L are represented as: M=0, F=0, C=50, etc. The processing priority order for each target object can be as follows:

[0099]

[0100] Table 1. Priority Ranking of Target Objects

[0101] In some implementations, taking the task of behavior recognition as an example, similarly, the targets can be prioritized based on the attribute values ​​of each attribute information of each target. The targets are sorted in turn according to the attribute values ​​of the target behavior type, vehicle distance, and vehicle quantity, until the attribute values ​​of each target object are different or the processing priority sorting of the last attribute information (vehicle quantity) is completed, thus obtaining the processing priority sorting of each target object.

[0102] S13: Sort the processing priorities of each target object and allocate computing resources to each target object to perform processing tasks for each target object.

[0103] By prioritizing the processing of each target object, the importance of each target object in the image frame of the target video can be determined. Since some target objects do not require precise processing tasks, while others do, or different target objects can be assigned different computational resources, computational resources can be allocated to each target object in the current image frame to perform processing tasks. Different computational resources can be allocated to different target objects; for example, the top few target objects can be selected based on priority and allocated the first set of computational resources, while other target objects can be allocated the second set of computational resources. This application does not impose any restrictions on this approach.

[0104] In some implementations, taking the processing task of a target object as an example of a target localization task, the computational resources required for the processing task can vary depending on the different motion states of the target object. More computational resources can be allocated to a moving target object, while less can be allocated to a stationary moving target object. Localization processing can include localization tasks with different computational resources, such as target localization and assisted localization. Target localization can be described as accurately locating the target object in the current image frame, while assisted localization can be described as estimating the position of the target object using information such as the target object's detection bounding box and historical trajectory. In other words, assisted localization can include at least one of target recognition and target tracking.

[0105] In some embodiments, please refer to Figure 6 Step S13 can be further extended. In step S13, according to the processing priority of each target object, computing resources are allocated to each target object to execute processing tasks. This may include the following steps:

[0106] S131: Sort the target objects according to their processing priority, select some target objects as the first target objects, and select the remaining target objects as the second target objects.

[0107] Based on the processing priority of each target object in the current image frame, a subset of target objects can be selected as first target objects, i.e., objects of higher importance that require more computing resources. The remaining target objects can then be designated as second target objects. The number of selected target objects can be set based on specific application scenarios, and this application does not impose any restrictions on this.

[0108] In some implementations, a first set number of target objects with the highest processing priority can be selected as the first target objects, and the remaining target objects can be selected as the second target objects.

[0109] In some implementations, in order to allocate different computing resources more finely, some of the second target objects can be selected as the third target objects, etc., and this application does not limit this.

[0110] S132: Allocate a first computing power resource to a first target object to perform a processing task on the first target object using a first preset processing method, and allocate a second computing power resource to a second target object to perform a processing task on the second target object using a second preset processing method, wherein the first computing power resource is greater than the second computing power resource.

[0111] Different computing resources can be allocated to target objects with different processing priorities. A first set of computing resources is allocated to the first target object to execute processing tasks using a first preset processing method. A second set of computing resources is allocated to the second target object to execute processing tasks using a second preset processing method. The amount of computing resources required for different preset processing methods varies, with the first set of computing resources requiring more than the second set.

[0112] In some implementations, the processing task is a target localization task. The first preset processing method is target localization, which involves accurately locating a first target object and identifying its depth, height, and position information. The second preset processing method is assisted localization, which requires less computational resources. Assisted localization can be described as estimating the position of the target object using its bounding box, historical trajectory, and other information. In other words, assisted localization can include at least one of target recognition and target tracking.

[0113] Since target localization requires precise positioning and thus requires high computing resources, it is not possible to perform real-time target localization for every frame of the image. Instead, more computing resources can be allocated to locate the first number of target objects before sorting, while the second target objects that are sorted later require less computing resources and can be allocated fewer computing resources for auxiliary localization.

[0114] Using the above method, target objects can be sorted according to the priority of attribute information and attribute values. Based on the available computing power resources, higher priority target objects can be selected for computing power allocation (such as targets to be located), while other target objects can be located using cached historical trajectory information and other information that requires less computing power resources.

[0115] Furthermore, by analyzing the processing task requirements under the constraint of computing resources, and using the attribute information and attribute values ​​of the target objects to comprehensively analyze the target objects in the current image frame, the priority ranking of the target objects that need to be ranked is obtained. Then, based on the computing resources, a preset number of target objects are selected for target localization. By applying limited computing resources to the target objects that most need target localization, the target objects that are more important to the overall localization task are prioritized for localization, while other target objects use auxiliary localization methods with less computing resources, which can improve the overall target localization efficiency.

[0116] In some implementations, the processing task is a behavior recognition task. The first preset processing method is target behavior recognition, which means accurately performing the target behavior recognition task on the target object. The second preset processing method is auxiliary behavior recognition, which consumes less computing resources. Auxiliary behavior recognition includes using the target object's historical behavior recognition results to perform target behavior recognition on the target object.

[0117] After prioritizing the processing of attribute information for each target object, different computing resources can be allocated to targets with different priorities for target behavior recognition tasks. For targets with lower priorities, fewer computing resources can be allocated for target behavior recognition tasks. This can reasonably allocate computing resources, reduce waste of computing resources, and improve the recognition efficiency of target behavior recognition tasks.

[0118] In this embodiment, a target set for the processing task is obtained, comprising K target objects in the target video. The attribute information of each of the K target objects is comprehensively analyzed to prioritize their processing, thus determining their importance or priority in relation to the processing task. Then, computing resources are allocated to each target object according to their processing priority. Different computing resources can be allocated to target objects with different processing priorities, ensuring that more important target objects receive priority and more computing resources, while other target objects receive less. This approach allows for the rational allocation of computing resources to each target object when resources are limited, improving the accuracy of processing tasks and reducing waste. It also improves the efficiency of computing resource utilization. Furthermore, the rational allocation of different computing resources to target objects with different processing priorities further enhances the processing efficiency of the task.

[0119] Furthermore, since processing tasks can be performed on each target object in each image frame within the scope of computing resources without losing image frames, and real-time target localization can be performed on as many target objects as possible, the accuracy of processing tasks on target objects can be further improved.

[0120] In addition to the above embodiments, this application also provides a computing resource allocation device. Please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram of an embodiment of the computing power resource allocation device of this application.

[0121] The computing power resource allocation device 20 includes an acquisition module 21, a sorting module 22, and an allocation module 23. The acquisition module 21, the sorting module 22, and the allocation module 23 are connected together.

[0122] The acquisition module 21 is used to acquire the target set for the processing task to be performed. The target set includes K target objects in the target video, where K is an integer greater than 1.

[0123] The sorting module 22 is used to sort the processing priority of each target object by utilizing the attribute information of each target object among the K target objects.

[0124] The allocation module 23 is used to sort the processing priorities of each target object and allocate computing resources to each target object to perform processing tasks for each target object.

[0125] The specific implementation of this embodiment can be referred to the implementation process of the above embodiments, and will not be repeated here.

[0126] Regarding the above embodiments, this application provides a computer device; please refer to [link / reference]. Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device according to an embodiment of the present application. The computer device 30 includes a memory 31 and a processor 32, wherein the memory 31 and the processor 32 are coupled to each other. The memory 31 stores program data, and the processor 32 is used to execute the program data to implement the steps of any embodiment of the above-described method for allocating computing resources.

[0127] In this embodiment, processor 32 can also be referred to as CPU (Central Processing Unit). Processor 32 may be an integrated circuit chip with signal processing capabilities. Processor 32 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor can be a microprocessor, or processor 32 can be any conventional processor.

[0128] The specific implementation of this embodiment can be referred to the implementation process of the above embodiments, and will not be repeated here.

[0129] The methods described in the above embodiments can be implemented as computer programs; therefore, this application proposes a computer-readable storage medium. Please refer to [link to relevant documentation]. Figure 9 , Figure 9 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 40 stores program data 41 that can be executed by a processor. The program data 41 can be executed by the processor to implement the steps of any embodiment of the above-described method for allocating computing resources.

[0130] In this embodiment, the computer-readable storage medium 40 can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a medium that can store program data 41. Alternatively, it can be a server that stores the program data 41. The server can send the stored program data 41 to other devices for execution, or it can run the stored program data 41 itself.

[0131] The specific implementation of this embodiment can be referred to the implementation process of the above embodiments, and will not be repeated here.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0134] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application.

[0136] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, and thus stored in a computer-readable storage medium for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, this application is not limited to any particular hardware and software combination.

[0137] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for allocating computing resources, characterized in that, The method includes: Obtain the target set for which processing tasks need to be performed, wherein the target set includes K target objects in the target video, and K is an integer greater than 1; The M attribute information is sorted based on the importance of each attribute information; wherein, the importance is the degree of importance of the corresponding attribute information relative to the processing task, M is an integer greater than 1, the attribute information is a factor affecting the processing task of the target object, and the M attribute information includes at least two of the following: the computing power consumption factor of executing the processing task, the processing result factor of executing the processing task, the object state factor of the target object executing the processing task, and the historical processing status of executing the processing task. The computing power consumption factor includes the motion state, and the attribute state of the motion state includes: turning, going straight, and stopping. Different motion states correspond to different attribute state values. The sorted attribute information is modeled in a state, and an attribute state value is defined for each attribute state according to the importance of each attribute state in each attribute information; the importance is the degree of importance of the corresponding attribute state relative to the processing task. Using the attribute status values ​​corresponding to the attribute information of each of the K target objects, the processing priority of each target object is sorted. According to the processing priority of each target object, computing resources are allocated to each target object to execute the processing task for each target object, including: According to the processing priority of each target object, some of the target objects are selected as the first target objects, and the remaining target objects are selected as the second target objects. The first target object is allocated a first computing power resource to perform the processing task on the first target object using a first preset processing method, and the second target object is allocated a second computing power resource to perform the processing task on the second target object using a second preset processing method, wherein the first computing power resource is greater than the second computing power resource.

2. The method according to claim 1, characterized in that, The processing task is a target localization task; the attribute information includes at least one of the following: motion state, historical processing status, and current device distance, wherein the current device distance is the distance between the target object and the camera device, and the historical processing status indicates the status of the target object performing the processing task in historical image frames; or The processing task is a behavior recognition task, and the attribute information includes at least one of the following: target behavior type, vehicle distance, and number of vehicles. The vehicle distance is the distance between the target vehicle and the target object, the number of vehicles is the number of target vehicles, and the target vehicles are vehicles that pose a security threat to the target object.

3. The method according to claim 2, characterized in that, The historical processing status includes whether the target object uses a first preset processing method to perform the processing task in historical image frames. Different preset processing methods require different amounts of computing resources, and different historical processing statuses correspond to different attribute state values. The attribute status value corresponding to the current device distance of the target object is the distance value between the target object and the preset position point in the current image frame of the target video; Wherein, the preset position point is the position point of the camera device, or the preset position point is the position point corresponding to the shooting area in the current image frame.

4. The method according to claim 2, characterized in that, The attribute states of the target behavior type include: running a red light, climbing over a guardrail, falling, and walking normally; different attribute states of the target behavior type correspond to different attribute state values; The attribute status value corresponding to the vehicle distance of the target object is the distance description value of the distance between the target object and the target vehicle, and the distance description value is inversely proportional to the distance. The attribute status value corresponding to the number of vehicles of the target object is a quantity description value of the number of target vehicles, and the quantity description value is proportional to the number of target vehicles.

5. The method according to claim 1, characterized in that, The step of prioritizing the processing of each of the K target objects by utilizing the attribute status values ​​corresponding to the attribute information of each target object includes: Obtain the attribute values ​​corresponding to M attribute information of each target object; wherein, the attribute values ​​represent the importance of the target object, and the attribute values ​​include the sum of the attribute state values ​​corresponding to each attribute state of the attribute information; M is an integer greater than 1; The target objects are sorted by processing priority according to the attribute values ​​corresponding to the attribute information of each target object.

6. The method according to claim 5, characterized in that, The step of prioritizing the processing of each target object according to the attribute values ​​corresponding to the M attribute information of each target object includes: The attribute sorting of the M attribute information is obtained; The attribute information that is first in the current attribute sorting is taken as the target attribute information; Based on the magnitude of the attribute values ​​corresponding to the target attribute information, the objects to be processed in this round are prioritized and sorted, wherein the objects to be processed in the first round are the target objects. Find the objects with the same attribute values ​​as the target attribute information from the priority sort and use them as the objects to be processed in the next round; The next attribute information in the current attribute sort is taken as the target attribute information, and the steps of taking the first attribute information in the current attribute sort as the target attribute information and subsequent steps are repeated until the attribute values ​​corresponding to the target attribute information are all different or the target attribute information is the last attribute information in the current attribute sort.

7. The method according to claim 6, characterized in that, The step of obtaining the attribute values ​​corresponding to the M attribute information of each target object includes: Obtain the attribute status corresponding to the M attribute information of each target object; The attribute state value that matches the attribute state of each of the M attribute information is taken as the attribute value of each attribute information.

8. The method according to claim 5, characterized in that, The processing task is a target localization task, and obtaining the attribute values ​​corresponding to the M attribute information of each target object includes any one or more of the following steps: Obtain the attribute state of the motion state of each target object, and from the several attribute state values ​​corresponding to the attribute state of the motion state, obtain the attribute state value that matches the attribute state of the motion state of each target object, and use it as the attribute value corresponding to the motion state. Obtain the attribute status of the historical processing status of each target object, and from the several attribute status values ​​corresponding to the attribute status of the historical processing status, obtain the attribute status value that matches the attribute status of the historical processing status of each target object, and use it as the attribute value corresponding to the historical processing status. Obtain the coordinate information of each target object, and use the distance between the coordinate information of each target object and a preset position point in the current image frame of the target video as the attribute value corresponding to the current device distance of each target object; or, The processing task is a behavior recognition task, and obtaining the attribute values ​​corresponding to the M attribute information of each target object includes any one or more of the following steps: Obtain the attribute state of the target behavior type of each target object, and from the several attribute state values ​​corresponding to the attribute state of the target behavior type, obtain the attribute state value that matches the attribute state of the target behavior type of each target object, and use it as the attribute value corresponding to the target behavior type; Obtain the distance between each target object and the target vehicle, and use the distance description value corresponding to the distance as the attribute value corresponding to the vehicle distance; Obtain the number of target vehicles corresponding to each target object, and use the quantity description value corresponding to the quantity as the attribute value corresponding to the vehicle quantity.

9. The method according to claim 1, characterized in that, Selecting a portion of the target objects as the first target objects includes: Select the target objects whose processing priority is ranked first preset number, and use them as the first target objects; And / or, the processing task is a target localization task, the first preset processing method is target localization, the second preset processing method is assisted localization, and the assisted localization includes at least one of target recognition and target tracking; or, the processing task is a behavior recognition task, the first preset processing method is target behavior recognition, the second preset processing method is assisted behavior recognition, and the assisted behavior recognition includes using the historical behavior recognition results of the target object to perform target behavior recognition on the target object.

10. The method according to claim 1, characterized in that, The process of obtaining the target set for the processing tasks to be performed includes: Sequentially acquire each image frame in the target video, and use each target object contained in each image frame as the target set for the processing task to be performed; The processing task refers to the processing of each target object contained in a series of consecutive image frames.

11. A computer device, characterized in that, The method includes a memory and a processor coupled to each other, the memory storing program data and the processor executing the program data to implement the steps of the method according to any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, The system stores program data that can be executed by a processor, the program data being used to implement the steps of the method according to any one of claims 1 to 10.

Images