A model scheduling method and device, and a storage medium

Abstract

Embodiments of the present application provide a model scheduling method and device, and a storage medium. A starting model can be specified from a plurality of models, and the starting model processes source data according to a preset processing frequency, which can guarantee the processing frequency of the source data. The time consumption of the starting model usually does not exhaust the trigger interval time thereof, and therefore, the available time periods generated by the starting model can be monitored, and the trigger time of each remaining model can be determined based on a preset scheduling strategy within the available time periods, so as to control each remaining model to complete the processing work of each source data in cooperation with the starting model. In this way, the remaining models do not need to be limited by the preset processing frequency, but can make full use of the fragmented time left during the execution of the starting model to support the time consumption of each remaining model, which enables the plurality of models to maintain the required model accuracy without the need to reduce the accuracy for saving the time consumption. Accordingly, the model accuracy and the processing frequency can be guaranteed at the same time, thereby effectively improving the quality of the processing result.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a model scheduling method, device and storage medium. Background Technology

[0002] With the widespread application of visual AI, it is difficult to meet application requirements by deploying a single model, especially in high-frame-rate and high-precision detection and recognition scenarios, such as passenger flow recognition. In such scenarios, the terminal needs to achieve high-precision detection and tracking, as well as data optimization and feature extraction. This often requires the deployment of multiple models to support increasingly complex application needs.

[0003] Currently, because the NPU (Neural-network Processing Unit) is exclusive, models must be invoked serially, meaning multiple models are executed sequentially. This leads to the need to compress model execution time to meet high frame rate requirements, resulting in reduced model accuracy. Conversely, to maintain high accuracy, the input frame rate must be reduced. In either case, the quality of the final processing result is unsatisfactory. Summary of the Invention

[0004] This application provides a model scheduling method, device, and storage medium to improve the quality of processing results in multi-model scenarios.

[0005] This application provides a model scheduling method, including:

[0006] The source data is processed by triggering the specified initial model according to the preset processing frequency;

[0007] The trigger interval time corresponding to the starting model is calculated based on the preset processing frequency to determine the available time period that the remaining models can occupy within the trigger interval time.

[0008] During the available time period generated by the initial model, the triggering time of each remaining model is determined based on a preset scheduling strategy, so as to control each remaining model to cooperate with the initial model to complete the processing of each source data.

[0009] This application also provides a computing device, including a memory, a processor, and a communication component;

[0010] The memory is used to store one or more computer instructions;

[0011] The processor is coupled to the memory and the communication component, and is used to execute the one or more computer instructions for:

[0012] The source data is processed by triggering the specified initial model according to the preset processing frequency;

[0013] The trigger interval time corresponding to the starting model is calculated based on the preset processing frequency to determine the available time period that the remaining models can occupy within the trigger interval time.

[0014] During the available time period generated by the initial model, the triggering time of each remaining model is determined based on a preset scheduling strategy, so as to control each remaining model to cooperate with the initial model to complete the processing of each source data.

[0015] This application also provides a computer-readable storage medium for storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the aforementioned model scheduling method.

[0016] In this embodiment, a starting model can be specified among multiple models, and the starting model processes the source data according to a preset processing frequency, ensuring the processing frequency of the source data. The starting model's execution time typically does not exhaust its trigger interval. Therefore, the available time periods generated by the starting model can be monitored, and within these available time periods, the triggering timing of each remaining model is determined based on a preset scheduling strategy. This controls each remaining model to cooperate with the starting model to complete the processing of each source data. In this way, the remaining models are no longer limited by the preset processing frequency, but can fully utilize the fragmented time freed up during the execution of the starting model to support the required processing time of each remaining model. This allows multiple models to maintain the required model accuracy without sacrificing accuracy to save time. Accordingly, both model accuracy and processing frequency can be guaranteed simultaneously, thereby effectively improving the quality of the processing results. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A flowchart illustrating a model scheduling method provided for an exemplary embodiment of this application;

[0019] Figure 2 A logical schematic diagram of a model scheduling scheme provided for an exemplary embodiment of this application;

[0020] Figure 3 A logical diagram illustrating an application scenario provided for an exemplary embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the structure of a computing device provided for another exemplary embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Currently, in multi-model scenarios, it is impossible to guarantee model accuracy and input frame rate, resulting in poor processing quality. To address this, some embodiments of this application specify a starting model among multiple models, and this starting model processes the source data according to a preset processing frequency. This ensures the processing frequency of the source data. The starting model's execution time typically does not exhaust its trigger interval. Therefore, the available time periods generated by the starting model can be monitored, and within these available time periods, the triggering timing of each remaining model can be determined based on a preset scheduling strategy. This controls each remaining model to cooperate with the starting model to complete the processing of each source data. In this way, the remaining models are no longer limited by the preset processing frequency, but can fully utilize the fragmented time freed up during the execution of the starting model to support the required execution time of each remaining model. This allows multiple models to maintain the required model accuracy without sacrificing accuracy to save time. Accordingly, both model accuracy and processing frequency can be guaranteed simultaneously, thereby effectively improving the quality of the processing results.

[0024] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0025] Figure 1 A flowchart illustrating a model scheduling method provided in an exemplary embodiment of this application is shown below. Figure 2 This is a logical schematic diagram of a model scheduling scheme provided for an exemplary embodiment of this application. The method can be executed by a model scheduling device, which can be implemented as a combination of software and / or hardware, and can be integrated into a computing device. (Reference) Figure 1 The method may include:

[0026] Step 100: Trigger the specified starting model to process the source data according to the preset processing frequency;

[0027] Step 101: Calculate the trigger interval time corresponding to the starting model based on the preset processing frequency, so as to determine the available time period that the remaining models can occupy within the trigger interval time.

[0028] Step 102: During the available time period generated by the initial model, determine the triggering time of each remaining model based on the preset scheduling strategy, so as to control each remaining model to cooperate with the initial model to complete the processing of each source data.

[0029] The model scheduling method provided in this embodiment can be applied to multi-model scenarios, that is, scenarios where multiple models are used to meet application requirements. For example, in a passenger flow recognition scenario, multiple models such as a human detection model, a face detection model, a human feature extraction model, and a face feature extraction model can be used to meet passenger flow recognition needs. Of course, this is only an example, and this embodiment can also be applied to other multi-model scenarios. The models in this embodiment can be large models such as neural network models and machine learning models, or other types of models. This embodiment does not limit the type of model. In addition, the model scheduling scheme provided in this embodiment is mainly for situations where the processor on which the model depends has exclusive access.

[0030] refer to Figure 1 and Figure 2 In step 100, the source data can be processed by a specified starting model according to a preset processing frequency. In this embodiment, a starting model can be specified among multiple models to be scheduled. Generally, any model that takes the source data as input parameters can be specified as the starting model. Of course, other factors such as time consumption and priority can also be considered to determine the starting model. This embodiment does not limit the selection scheme of the starting model. In practical applications, a suitable model can be specified as the starting model as needed.

[0031] The preset processing frequency refers to the data processing frequency required by the application. For example, in image processing, the preset processing frequency can refer to the frame rate; in simulation, it can refer to the simulation frequency. The preset processing frequency limits the frequency of source data acquisition. For instance, the frame rate limits the frequency of extracting image frames from the input image stream. The initial model must strictly adhere to the preset processing frequency to ensure that the source data extracted according to the preset processing frequency can be processed successfully. It is worth noting that, to ensure the normal execution of multiple models, in this embodiment, the maximum execution time of a single model among the multiple models meets the requirements of the preset processing frequency.

[0032] In this embodiment, in step 100, source data can be extracted from the input data according to a preset processing frequency. For example, if the input data is an image stream, the source data can be image frames from the image stream. In step 100, the extracted source data can also be input into the starting model according to the preset processing frequency to trigger the starting model to process the received source data. The triggering frequency of the starting model will be consistent with the preset processing frequency; therefore, the triggering time of the starting model each time is known. In step 101, the triggering interval time corresponding to the starting model can be calculated based on the triggering time of the starting model each time, to determine the available time period that the remaining models can occupy within the triggering interval time. Specifically, the interval time between two adjacent triggering times of the starting model can be calculated according to the preset processing frequency and used as the triggering interval time. Furthermore, the processing time of the starting model itself is also known; therefore, the available time period that the remaining models can occupy within the triggering interval time can be determined based on the triggering interval time corresponding to the starting model and the processing time of the starting model. For example, if the frame rate is 10 frames / s and the initial model's processing time is 20ms, and the trigger interval is 101ms-200ms, then after deducting the initial model's processing time of 20ms, the remaining model's available time within that trigger interval will be 121ms-200ms, totaling 80ms.

[0033] Based on the initial model being triggered at a frequency, the initial model will continuously generate available time periods. During these available time periods, the processors on which multiple models depend are in an idle state. This embodiment proposes to make full use of these fragmented time periods of the processor and use these fragmented time periods to insert the remaining models among multiple models.

[0034] It should be understood that, for each source data, this embodiment will control each remaining model to work together with the starting model to complete the processing work. That is, for each source data, a complete model chain will be executed. However, in the model chain for different source data, the triggering time of each remaining model may be discrete or discontinuous.

[0035] Therefore, refer to Figure 1 and Figure 2In step 102, within the available time period generated by the initial model, the triggering timing of each remaining model can be determined based on a preset scheduling strategy to control each remaining model to cooperate with the initial model in completing the processing of each source data. The scheduling strategy may include, but is not limited to, prioritizing the triggering of models with lagging execution counts; prioritizing the triggering of models with higher priority; and / or prioritizing the triggering of models with longer execution times. It should be understood that only a few exemplary strategy dimensions are provided here, and this embodiment is not limited to these. Furthermore, priorities can be configured between different strategy dimensions, and priority triggering conditions can be added to each strategy dimension. For example, under the strategy dimension of "prioritizing the triggering of models with lagging execution counts," the condition "the selected model's execution time cannot exceed the remaining time period in the current available time period" can be added as a priority triggering condition. Thus, under this strategy dimension, triggerable models will be selected from models whose execution time is less than the remaining time period in the current available time period. The scheduling strategy in this embodiment can be set according to actual needs and can be maintained by adding, modifying, or deleting as needed.

[0036] In this way, the fragmented idle time of the processing on which multiple models depend can be fully utilized. During the execution of the initial model case with the preset processing frequency, the remaining models in multiple models can be interspersed and triggered, ensuring that the processing work for each source data is completed completely.

[0037] refer to Figure 1 and Figure 2 Considering the potential contextual relationships between multiple models—that is, later models may need to rely on the processing results of earlier models as input parameters—this embodiment can also use a database or similar method to store the processing results of multiple models for later models to use as needed. This embodiment does not limit the type of database used; for example, an SQLite database or a MySQL database can be used. The processing results stored in the database can be indexed using model identifiers; however, this embodiment is not limited to this.

[0038] In this embodiment, auxiliary operations can also be configured for the model as needed. For special models associated with auxiliary operations, the special model and its corresponding auxiliary operations can be executed together, and the results of the auxiliary operations can be included as part of the processing results of the special model. For example, for a human detection model, a matting operation can be associated with it. In this way, the matting data obtained by the matting operation and the detection results of the human detection model can be saved together as the processing results corresponding to the human detection model. The time spent on the auxiliary operations can be included in the time spent on the associated model.

[0039] As mentioned earlier, in this embodiment, multiple models will be controlled to execute a complete model chain for each source data. Therefore, in this embodiment, the processing progress of each source data can be monitored separately; the model processing result corresponding to the target source data that has completed all model processing is output. Optionally, in this embodiment, processing progress information can be maintained for each source data separately. After each model completes the processing of the source data, a flag representing the model can be added to the processing progress information of the corresponding source data to indicate that the model has completed processing the source data. Thus, by detecting whether the processing progress information corresponding to each source data contains the flags corresponding to all models, if so, it can be determined that the processing of that source data has been completed. Of course, this is only an example; other methods can also be used to monitor the processing progress of source data in this embodiment, and this embodiment is not limited to this.

[0040] Accordingly, in this embodiment, a starting model can be specified among multiple models, and the starting model processes the source data according to a preset processing frequency, ensuring the processing frequency of the source data. The starting model's execution time typically does not exhaust its trigger interval. Therefore, the available time periods generated by the starting model can be monitored, and within these available time periods, the triggering timing of each remaining model can be determined based on a preset scheduling strategy. This controls each remaining model to cooperate with the starting model to complete the processing of each source data. In this way, the remaining models are no longer limited by the preset processing frequency, but can fully utilize the fragmented time freed up during the execution of the starting model to support the required execution time of each remaining model. This allows multiple models to maintain the required model accuracy without sacrificing accuracy to save time. Therefore, both model accuracy and processing frequency can be guaranteed simultaneously, effectively improving the quality of the processing results.

[0041] In the above or below embodiments, various implementation methods can be used to schedule each remaining model within the available time period generated by the initial model.

[0042] In one optional implementation, multiple scheduling tasks with a sequential relationship can be created. The multiple models to be scheduled are distributed in groups under these multiple scheduling tasks, with the initial model distributed under the initial scheduling task. Based on this, multiple scheduling tasks can be used to determine the triggering timing of each remaining model within the available time period generated by the initial model, according to a preset scheduling strategy.

[0043] In this implementation, multiple models can be distributed across multiple scheduling tasks according to various considerations such as the order requirements, contextual relationships, and priorities among the models. This embodiment does not limit the distribution scheme of multiple models; in practical applications, distribution can be carried out as needed, and the distribution scheme can be adjusted at any time. However, it should be understood that, for the sake of greater logical simplicity, this implementation assumes a sequential relationship between the multiple scheduling tasks. Thus, all models distributed under a earlier scheduling task will be triggered before all models distributed under a later scheduling task. This basic triggering order caused by the scheduling tasks should be fully considered during the distribution of multiple models.

[0044] In this implementation, a model group under a single scheduling task can contain one or more models. In this embodiment, priority, required processing time, and execution count can be configured for the models within the model group to provide a basis for the scheduling strategy of the scheduling task. In this embodiment, by default, each model processes the extracted source data sequentially according to the extraction order. Therefore, the execution count represents the amount of source data that the model has processed. The aforementioned strategy dimension of "prioritizing the triggering of models with lower execution counts" can be interpreted as prioritizing the complete processing of the earlier extracted source data. The execution count information supports the successful implementation of this scheduling dimension. Explanations of other scheduling information supporting other scheduling dimensions will not be elaborated here.

[0045] In addition, in this implementation, different scheduling tasks can be associated with different scheduling strategies. That is, different scheduling tasks can use a unified scheduling strategy or a different scheduling strategy. In this way, a more suitable scheduling strategy can be configured according to the scheduling needs of the model group under different scheduling tasks, so as to achieve better scheduling results.

[0046] Based on this, in determining the triggering timing of each remaining model using multiple scheduling tasks, we can:

[0047] After the model group under the initial scheduling task has finished executing, calculate the remaining time period in the available time period;

[0048] Send a trigger signal to the next scheduling task of the initial scheduling task to trigger the next scheduling task. The triggering timing of each model in the model group distributed under it is determined based on the scheduling strategy.

[0049] If the remaining time period is not consumed after the model group under the next scheduling task is completed, trigger signals will continue to be sent to subsequent scheduling tasks until the remaining time period is consumed.

[0050] In other words, within a single available time period, after a model group under a preceding scheduled task successfully completes execution, a trigger signal can be sent to its next scheduled task. If the remaining time period within the available time period is still sufficient to support the successful completion of the model group under that next scheduled task, a trigger signal can continue to be sent to the next scheduled task after that, until a model group under a subsequent scheduled task cannot be fully completed due to the exhaustion of available time periods, at which point no further trigger signals are sent. After the available time period is exhausted, the processors on which multiple models depend will default to triggering the initial model. After the initial model completes execution, a new available time period will be generated. Under the new available time period, each scheduled task can, according to the above scheduling logic, re-determine the execution timing of each remaining model.

[0051] Among the multiple scheduling tasks, the starting scheduling task is special. In this implementation, if the model group under the starting scheduling task contains only the starting model, the available time period is used as the remaining time period; if the model group under the starting scheduling task contains multiple models, the remaining time period after subtracting the time consumption of other models besides the starting model from the available time period is used as the remaining time period.

[0052] As can be seen, in this implementation, multiple scheduling tasks with sequential relationships can consume available time periods in turn. Under a single scheduling task, the triggering time of each model in the distributed model group can be determined according to the scheduling strategy. In this way, from a global perspective, the triggering time of each remaining model can be determined within the available time period generated by the initial model, and it can be ensured that each remaining model cooperates with the initial model to complete the processing work of each data source.

[0053] Furthermore, to avoid the problem of subsequent model groups not getting enough execution time due to earlier scheduled tasks frequently exhausting their available time slots, a relief strategy can be introduced in this implementation. For example, if the number of times a model under a scheduled task has been executed is lower than a specified standard, the scheduling task can be moved forward in the sequence to allow it more execution time. This allows for more flexible scheduling of multiple models.

[0054] It should be understood that, in addition to the above implementation methods, other implementation methods can also be used in this embodiment to schedule the remaining models within the available time period generated by the initial model. For example, the remaining models can be treated as a whole and globally scheduled according to a scheduling strategy, etc., but this embodiment is not limited to this.

[0055] In this embodiment, by creating multiple scheduling tasks with sequential relationships, multiple models can be grouped and scheduled. This makes the scheduling logic of multiple models more detailed, which can effectively improve the flexibility of model scheduling and reduce the logical complexity of model scheduling.

[0056] Figure 3 This is a logical diagram illustrating an application scenario provided for an exemplary embodiment of this application. Figure 3 Taking passenger flow recognition as an example, image streams can be acquired via cameras. Human detection models, face detection models, human feature extraction models, face feature extraction models, and face attribute detection models are used to address passenger flow recognition needs. (Reference) Figure 3 Two scheduling tasks can be created: the human detection model is distributed to the initial scheduling task, and the remaining four models are distributed to the second scheduling task.

[0057] Based on this Figure 3 The technical solution can be divided into four parts: data preprocessing, initial scheduling task, second scheduling task, and transmission processing.

[0058] In the data preprocessing section, the image stream acquired by the camera can be decapsulated, decoded, and processed. It can also determine whether the current image frame needs to be extracted based on the preset frame rate. If so, the extracted image frame data can be input into the initial scheduling task. If not, the current image frame can be released (which can be understood as ignoring it).

[0059] In the initial scheduling task, frame rate control can be implemented. Extracted image frames are received according to a preset frame rate, and the human detection model is triggered. If the human detection model detects a pedestrian, the associated auxiliary operation—masking—is executed, and the detection result and matting result are inserted into the database. If the human detection model does not detect a pedestrian, it continues to wait for the next image frame. After the model under the initial scheduling task completes execution, a trigger signal can be sent to the second scheduling task. It is worth noting that if the human detection model does not detect a pedestrian, the initial scheduling task can consider its model to have completed execution.

[0060] In the second scheduling task, the processing results generated under the initial scheduling task can be retrieved from the database in advance for later use. The second scheduling task can also calculate the remaining time period in the available time period. Upon receiving a trigger signal, the triggering timing of the four models under it can be determined based on the preset scheduling strategy. Figure 3 The document illustrates an exemplary scheduling strategy. The second scheduling task first checks whether any of the four models has not finished processing the previously loaded matting data (processing results under the initial scheduling task) based on the number of times the four models have been executed. If such a model exists, it is given priority to process the previously loaded matting data. If not, a reasonable triggering time can be determined for the four models based on the remaining time period. For example, models with higher priority or models with longer processing times can be triggered first.

[0061] For example, if the priorities of the four models are: face detection model > human feature extraction model = face feature extraction model = face attribute detection model; and the time consumption of the four models are: face detection model (60ms) > human feature extraction model (50ms) > face feature extraction model (20ms) > face attribute detection model (10ms), then with 80ms remaining, the face detection model can be triggered first, followed by the face feature extraction model, and the remaining time will be exhausted. In this case, after the second scheduled task starts again, the human feature extraction model can be triggered first, followed by the face attribute detection model. However, with 50ms remaining, the human feature extraction model can be triggered first, and the remaining time will be exhausted.

[0062] In the sending and processing section, the processing progress of each image frame can be queried. Once the image frame has been fully processed by the five models, the sending thread can be started. The sending thread can perform JPEG encoding on the processing results retrieved from the database and store the encoded image in OSS. OSS can return a storage address for the image to the sending thread. The sending thread can then package the storage address and the processing results retrieved from the database into an HTTP message and send it to the cloud for further processing.

[0063] As can be seen, in this exemplary passenger flow recognition scenario, the five models can utilize NPU resources in a time-sharing manner, thereby making full use of NPU resources. In addition, a high frame rate and high performance of each model can be maintained at the same time, thus resulting in high passenger flow recognition quality.

[0064] It should be noted that the execution subject of each step in the method provided in the above embodiments can be the same device, or the method can be executed by different devices. Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order; however, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel.

[0065] Figure 4 This is a schematic diagram of the structure of a computing device provided for another exemplary embodiment of this application. For example... Figure 4 As shown, the computing device includes: a memory 40, a processor 41, and a communication component 42.

[0066] Processor 41, coupled to memory 40, is used to execute computer programs in memory 40 for:

[0067] The source data is processed by triggering the specified initial model according to the preset processing frequency;

[0068] The trigger interval time corresponding to the starting model is calculated based on the preset processing frequency to determine the available time period that the remaining models can occupy within the trigger interval time.

[0069] During the available time period generated by the initial model, the triggering time of each remaining model is determined based on a preset scheduling strategy, so as to control each remaining model to cooperate with the initial model to complete the processing of each source data.

[0070] The processing results of each model are saved so that they can be used as needed in subsequent models.

[0071] In an alternative embodiment, processor 41 may also be used for:

[0072] Multiple scheduling tasks with a sequential relationship are created, wherein multiple models to be scheduled are distributed in groups under the multiple scheduling tasks, and the starting model is distributed under the starting scheduling task;

[0073] Within the available time period generated by the initial model, determining the triggering timing of each remaining model based on a preset scheduling strategy includes:

[0074] Using the multiple scheduling tasks, the triggering time of each remaining model is determined based on a preset scheduling strategy within the available time period generated by the initial model.

[0075] In an optional embodiment, the processor 41, in the process of determining the triggering timing of each remaining model based on a preset scheduling strategy within the available time period generated by the initial model using the plurality of scheduling tasks, can be used to:

[0076] After the model group under the initial scheduling task has been executed, the remaining time period in the available time period is calculated;

[0077] Send a trigger signal to the next scheduling task of the initial scheduling task to trigger the next scheduling task to determine the triggering timing of each model in its distributed model group based on the scheduling strategy.

[0078] If the remaining time period is not consumed after the model group under the next scheduling task is completed, trigger signals will continue to be sent to subsequent scheduling tasks until the remaining time period is consumed.

[0079] In an alternative embodiment, processor 41 may also be used for:

[0080] If the model group under the initial scheduling task contains only the initial model, then the available time period is taken as the remaining time period;

[0081] If the model group under the initial scheduling task contains multiple models, the remaining time period after subtracting the time consumption of other models besides the initial model from the available time period is taken as the remaining time period.

[0082] In an optional embodiment, the scheduling strategy includes:

[0083] Prioritize triggering models that have lagged behind in the number of executions;

[0084] Prioritize triggering models with higher priority; and / or

[0085] Prioritize triggering models that take longer to process.

[0086] In an optional embodiment, during the process of the processor 41 processing the source data by triggering a specified initial model according to a preset processing frequency, it can be used to:

[0087] Extract source data from input data according to the preset processing frequency;

[0088] The extracted source data is input into the starting model according to the preset processing frequency to trigger the starting model to process the received source data.

[0089] In an optional embodiment, the processor 41, in the process of calculating the trigger interval time corresponding to the starting model according to the preset processing frequency, and determining the available time period that the remaining models can occupy within the trigger interval time, can be used for:

[0090] The interval between two consecutive triggering times of the starting model is calculated based on the preset processing frequency and is used as the trigger interval.

[0091] Based on the trigger interval time corresponding to the starting model and the processing time of the starting model, determine the available time period that the remaining model can occupy within the trigger interval time.

[0092] In an alternative embodiment, processor 41 may also be used for:

[0093] Monitor the processing progress of the source data received by the initial model each time;

[0094] Output the model processing results corresponding to the target source data for which all model processing work has been completed.

[0095] In one optional embodiment, the source data is an image frame in an image stream, and the preset processing frequency is the frame rate.

[0096] In one alternative embodiment, the plurality of models includes multiple types of human body detection models, face detection models, human body feature extraction models, and face feature extraction models.

[0097] In an alternative embodiment, processor 41 may also be used for:

[0098] For special models with associated auxiliary operations, the associated special model and its corresponding auxiliary operations are executed.

[0099] The results of the auxiliary operation are used as part of the processing results of the special model.

[0100] In an optional embodiment, the maximum processing time of a single model among multiple models meets the requirements of the preset processing frequency.

[0101] Furthermore, such as Figure 4 As shown, the computing device also includes other components such as a power supply component 43. Figure 4 The diagram only shows some components and does not mean that the computing device includes only these components. Figure 4 The components shown.

[0102] It is worth noting that the technical details of the above-mentioned embodiments of the computing device can be referred to the relevant descriptions in the foregoing method system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0103] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by a computing device in the above method embodiments.

[0104] The above Figure 4 The memory in a computer is used to store computer programs and can be configured to store various other data to support operation on a computing platform. Examples of this data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disks, or optical disks.

[0105] The above Figure 4The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0106] The above Figure 4 The power supply component provides power to the various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0107] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0109] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0110] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0111] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0112] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0113] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0114] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0115] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A model scheduling method, comprising: The source data is processed by triggering the specified starting model according to the preset processing frequency, and the complete model chain is executed for each source data. The trigger interval time corresponding to the starting model is calculated based on the preset processing frequency to determine the available time period that the remaining models in the model chain can occupy within the trigger interval time. Within the multiple available time periods generated by the initial model, the triggering time of each remaining model is determined based on a preset scheduling strategy, so as to control each remaining model to cooperate with the initial model to complete the processing of each source data. For any given source data, the triggering timing of each remaining model is discrete.

2. The method according to claim 1, further comprising: Multiple scheduling tasks with a sequential relationship are created, wherein multiple models to be scheduled are distributed in groups under the multiple scheduling tasks, and the starting model is distributed under the starting scheduling task; Within the available time period generated by the initial model, determining the triggering timing of each remaining model based on a preset scheduling strategy includes: Using the multiple scheduling tasks, the triggering time of each remaining model is determined based on a preset scheduling strategy within the available time period generated by the initial model.

3. The method according to claim 2, wherein determining the triggering timing of each remaining model based on a preset scheduling strategy within the available time period generated by the initial model using the plurality of scheduling tasks includes: After the model group under the initial scheduling task has been executed, the remaining time period in the available time period is calculated; Send a trigger signal to the next scheduling task of the initial scheduling task to trigger the next scheduling task to determine the triggering timing of each model in its distributed model group based on the scheduling strategy. If the remaining time period is not consumed after the model group under the next scheduling task is completed, trigger signals will continue to be sent to subsequent scheduling tasks until the remaining time period is consumed.

4. The method according to claim 3, further comprising: If the model group under the initial scheduling task contains only the initial model, then the available time period is taken as the remaining time period; If the model group under the initial scheduling task contains multiple models, the remaining time period after subtracting the time consumption of other models besides the initial model from the available time period is taken as the remaining time period.

5. The method according to claim 1, wherein the scheduling strategy includes: Prioritize triggering models that have lagged behind in the number of executions; Prioritize triggering models with higher priority. and / or Prioritize triggering models that take longer to process.

6. The method according to claim 1, wherein triggering a specified starting model to process the source data according to a preset processing frequency comprises: Extract source data from input data according to the preset processing frequency; The extracted source data is input into the starting model according to the preset processing frequency to trigger the starting model to process the received source data.

7. The method according to claim 1, wherein calculating the trigger interval time corresponding to the starting model based on the preset processing frequency to determine the available time period that the remaining model can occupy within the trigger interval time includes: The interval between two consecutive triggering times of the starting model is calculated based on the preset processing frequency and is used as the trigger interval. Based on the trigger interval time corresponding to the starting model and the processing time of the starting model, determine the available time period that the remaining model can occupy within the trigger interval time.

8. The method according to claim 1, further comprising: Monitor the processing progress of the source data received by the initial model each time; Output the model processing results corresponding to the target source data for which all model processing work has been completed.

9. The method according to claim 1, wherein the source data is an image frame in an image stream, and the preset processing frequency is a frame rate.

10. The method according to claim 9, wherein the multiple models include a human body detection model, a face detection model, a human body feature extraction model, or a face feature extraction model.

11. The method according to claim 1, further comprising: For special models with associated auxiliary operations, the associated special model and its corresponding auxiliary operations are executed. The results of the auxiliary operation are used as part of the processing results of the special model.

12. The method according to claim 1, wherein the maximum processing time of a single model among multiple models meets the requirements of the preset processing frequency.

13. The method according to claim 1, further comprising: Save the processing results of each model so that they can be used as needed in subsequent models.

14. A computing device, comprising a memory, a processor, and communication components; The memory is used to store one or more computer instructions; The processor is coupled to the memory and the communication component, and is used to execute the one or more computer instructions for: The source data is processed by triggering the specified starting model according to the preset processing frequency, and the complete model chain is executed for each source data. The trigger interval time corresponding to the starting model is calculated based on the preset processing frequency to determine the available time period that the remaining models in the model chain can occupy within the trigger interval time. Within the multiple available time periods generated by the initial model, the triggering time of each remaining model is determined based on a preset scheduling strategy, so as to control each remaining model to cooperate with the initial model to complete the processing of each source data. For any given source data, the triggering timing of each remaining model is discrete.

15. A computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the model scheduling method according to any one of claims 1-13.

Images