Smooth frame rate scheduling method and smooth frame rate scheduling apparatus

Abstract

The present application provides a smooth frame rate scheduling method and a smooth frame rate scheduling device. The smooth frame rate scheduling method comprises the following steps. A first target frame rate is set for a first channel. Data of a first frame is received through the first channel, and a first current time corresponding to the data of the first frame is obtained. A first next scheduling time is determined according to the first target frame rate and the first current time. The first current time of the data of the first frame is compared with the first next scheduling time to confirm whether the first current time is earlier than the first next scheduling time. When it is confirmed that the first current time is earlier than the first next scheduling time, the processing of the data of the first frame is skipped. When it is confirmed that the first current time is equal to or later than the first next scheduling time, the data of the first frame is outputted, and the first next scheduling time is updated. Through the present application, the output frame rate is ensured to be stable, and smooth speed limiting is achieved, so as to increase the convenience of use.

Description

Technical Field

[0001] This invention relates to a scheduling method and scheduling apparatus for a computer operating system, and more particularly to a smooth frame rate scheduling method and smooth frame rate scheduling apparatus. Background Technology

[0002] Generally, in existing multi-channel visual perception systems, each camera typically operates independently with a fixed frame rate input. The system load fluctuates with the task, leading to uneven distribution of camera input frames and uneven allocation of system resources, thus affecting overall real-time performance and processing stability. Therefore, a new design is needed to address these issues. Summary of the Invention

[0003] This invention provides a smooth frame rate scheduling method and a smooth frame rate scheduling device, thereby ensuring stable output frame rate and achieving smooth rate limiting, thus increasing ease of use.

[0004] This invention provides a smooth frame rate scheduling method, comprising the following steps: Setting a first target frame rate for a first channel; receiving data of a first frame through the first channel and obtaining the first current time corresponding to the data of the first frame; determining a first next scheduling time based on the first target frame rate and the first current time; comparing the first current time of the data of the first frame with the first next scheduling time to confirm whether the first current time is earlier than the first next scheduling time; skipping the processing of the data of the first frame when the first current time is confirmed to be earlier than the first next scheduling time; and outputting the data of the first frame and updating the first next scheduling time when the first current time is confirmed to be equal to or later than the first next scheduling time.

[0005] This invention provides a smooth frame rate scheduling device, including a storage device and a processor. The storage device stores a program. The processor executes the program to perform the following steps: Setting a first target frame rate for a first channel; receiving data of a first frame through the first channel and obtaining a first current time corresponding to the data of the first frame; determining a first next scheduling time based on the first target frame rate and the first current time; comparing the first current time of the data of the first frame with the first next scheduling time to confirm whether the first current time is earlier than the first next scheduling time; skipping the processing of the data of the first frame when it is confirmed that the first current time is earlier than the first next scheduling time; and outputting the data of the first frame and updating the first next scheduling time when it is confirmed that the first current time is equal to or later than the first next scheduling time.

[0006] The smooth frame rate scheduling method and device disclosed in this invention can ensure stable output frame rate and achieve smooth rate limiting by skipping the processing of the first frame's data or discarding data whose current time is earlier than the next scheduling time. By achieving uniform input in a single channel through smooth rate limiting and stable output of multi-channel video through unified scheduling, this invention solves the video stability problems caused by uneven input frame time distribution due to changes in camera system load with task requirements and uneven system resource allocation. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of a smooth frame rate scheduling device according to an embodiment of the present invention.

[0008] Figure 2 This is a flowchart of a smooth frame rate scheduling method according to an embodiment of the present invention.

[0009] Figure 3 This is a flowchart of a smooth frame rate scheduling method according to another embodiment of the present invention.

[0010] Figure 4 This is a flowchart of a smooth frame rate scheduling method according to another embodiment of the present invention.

[0011] Figure 5 This is a diagram of a smooth frame rate scheduling architecture according to another embodiment of the present invention.

[0012] The annotations in the attached figures are explained as follows:

[0013] 100: Smooth frame rate scheduling device; 110: Storage device; 120: Processor; 130: Camera; S202~S212, S302~S308, S402~S412: Steps; S502: Independent scheduling structure; S504: Driver monitoring system (DMS); S506: Blind spot detection system (BSD); S508: Aspect ratio imaging system (AVM); S510: Next scheduling time for DMS; S512: Next scheduling time for BSD; S514: Next scheduling time for AVM. Detailed Implementation

[0014] The following description illustrates the implementation of the invention and is intended to describe the basic spirit of the invention, but is not intended to limit the invention. The actual scope of the invention must be referred to in the subsequent patent application.

[0015] It must be understood that the words “comprising” and “including” used in this specification are used to indicate the presence of specific technical features, values, method steps, work processes, elements and / or components, but do not preclude the addition of more technical features, values, method steps, work processes, elements, components, or any combination thereof.

[0016] In patent applications, the use of terms such as "first," "second," and "third" to modify elements in the patent application is not intended to indicate a priority order, a prior relationship, or that one element precedes another, or the chronological order of the execution of method steps, but only to distinguish elements with the same name.

[0017] In the embodiments listed below, the same or similar elements or components will be represented by the same reference numerals.

[0018] Figure 1 This is a schematic diagram of a smooth frame rate scheduling device according to an embodiment of the present invention. In this embodiment, the smooth frame rate scheduling device 100 can be applied to advanced driver assistance systems (ADAS), video acquisition platforms, and robot vision systems, but the embodiments of the present invention are not limited thereto. Please refer to... Figure 1 The smooth frame rate scheduling device 100 may include at least a storage device 110, a processor 120, and a camera device 130.

[0019] Storage device 110 can store programs. In some embodiments, storage device 110 may be a read-only memory, flash memory, floppy disk, hard disk, optical disk, USB flash drive, magnetic tape, database accessible by a network, or other storage media with the same function that can be easily conceived by those skilled in the art, but the embodiments of the present invention are not limited thereto.

[0020] The processor 120 can be connected to the storage device 110. The processor 120 can access the program stored in the storage device 110 to execute the smooth frame rate scheduling method of this embodiment.

[0021] In some embodiments, the processor 120 may be an integrated circuit, such as a micro control unit (MCU), microprocessor, digital signal processor (DSP), field programmable gate array (FPGA), application specific integrated circuit (ASIC), or logic circuit, but the embodiments of the present invention are not limited thereto.

[0022] Depending on the intended use and scenario, different devices can be selected for the photography equipment 130. In some embodiments, the photography equipment 130 can be an around view monitor (AVM), primarily used to eliminate blind spots around the vehicle, providing assistance for parking and handling complex road conditions. To achieve seamless stitching of 360-degree panoramic images, the AVM camera typically needs to be deployed with cameras at four positions: front, rear, left, and right of the vehicle. Fisheye lenses with ultra-wide-angle coverage are commonly used. The photography equipment 130 can also be a driver monitoring system (DMS) camera, primarily used to monitor the driver's fatigue and dangerous driving behaviors (such as closing eyes, yawning, smoking, making or receiving phone calls, etc.) in real time. The photography equipment 130 can also be a blind spot detection (BSD) camera, primarily used to monitor pedestrians, non-motorized vehicles, or other vehicles in the blind spots to the sides and rear of the vehicle, improving safety during lane changes and turns. However, the embodiments of the present invention are not limited to these.

[0023] Figure 2 The flowchart illustrates a smooth frame rate scheduling method according to an embodiment of the present invention. Please also refer to... Figure 1 and Figure 2 In step S202, a first target frame rate is set for the first channel. Figure 1 A channel is established between the processor 120 and the camera device 130 for data transmission, and the processor 120 sets different target frame rates on the channel according to different camera devices 130. For example, the processor 120 configures the target frame rate for the first channel. When the first camera device corresponding to the first channel is an AVM camera, the corresponding AI task is multi-category object detection, mainly including different types of motor vehicles, non-motor vehicles, pedestrians, and traffic signs, etc., and the corresponding first target frame rate can be set to 30fps, but this embodiment of the invention is not limited to this. When the first camera device corresponding to the first channel is a DMS camera, the corresponding AI task is driving behavior analysis, mainly including detecting the driver's eye state, fatigue, phone calls, smoking, and seat belts, and the first target frame rate can be set to 15fps, but this embodiment of the invention is not limited to this. When the first camera device corresponding to the first channel is a BSD camera, the corresponding AI task is peripheral object detection, mainly including different types of motor vehicles, non-motor vehicles, and pedestrians, and the first target frame rate can be set to 20fps, but this embodiment of the invention is not limited to this. Furthermore, the different target frame rates mentioned above are related to their corresponding channel priorities. Higher priority requirements allow for setting higher target frame rates. For example, a higher target frame rate can be set for the DMS camera channel of an engineering vehicle; a higher target frame rate can be set for the AVM camera channel of a private vehicle.

[0024] Next, step S204 is performed, where the data of the first frame is received through the first channel, and the first current time corresponding to the data of the first frame is obtained. Figure 1 The processor 120 can receive the data of the first frame through the first channel and obtain the first current time corresponding to the data of the first frame. That is, the processor 120 is connected to the first imaging device through the first channel to obtain the first current time corresponding to the data of the first frame.

[0025] Next, in step S206, based on the first target frame rate and the first current time, the first next scheduling time is determined dynamically using AI. Then, in step S208, the processor 120 compares the first current time of the first frame's data with the first next scheduling time to confirm whether the first current time is earlier than the first next scheduling time. If it is confirmed that the first current time is earlier than the first next scheduling time, the processor 120 can skip processing the first frame's data or discard the first frame's data (step S210).

[0026] More specifically, in step S210, if Figure 1 If the processor 120 in this embodiment determines that the data of the first frame has not yet reached the scheduling time, the processor 120 can skip processing the data of the first frame or discard the data of the first frame to ensure stable output frame rate and achieve smooth speed limiting. In addition, the discarded data of the first frame is a selective discard of redundant frame data under high frame rate input. The processor 120 can still ensure that the first channel can output the corresponding frame data according to the preset frame rate.

[0027] On the other hand, in step S208, when it is confirmed that the first current time is equal to or later than the first next scheduling time, Figure 1 The processor 120 in this embodiment can output the data of the first frame and update the first next scheduling time (step S212). In some embodiments, the processor 120 can input the data of the first frame into an artificial intelligence (AI) model, and dynamically adjust the video input frame rate or skip processing the data of the current frame or discard the data of the current frame according to the real-time load. In some embodiments, step S212 further includes adding a target time interval to the first current time as the first next scheduling time to update the first next scheduling time, wherein the target time interval is associated with a first target frame rate.

[0028] More specifically, in step S212, since the first current time is later than the first next scheduling time, the actual first next scheduling time is the first current time plus this target time interval, in order to update the first next scheduling time, wherein the target time interval is associated with the first target frame rate.

[0029] Furthermore, the updated first next scheduling time can be represented by the following formula (1).

[0030] next_dipatich_time=current_time+(1000 / max_fps), formula (1)

[0031] Where next_dipatich_time represents the first next scheduling time after the update, current_time represents the current time, (1000 / max_fps) represents the target time interval, and max_fps represents the first target frame rate.

[0032] Figure 3 This is a flowchart of a smooth frame rate scheduling method according to another embodiment of the present invention. In this embodiment, Figure 3 Steps S202~S212 and Figure 2 Steps S202 to S212 are the same or similar, and can be referred to Figure 2 The embodiments described herein will not be repeated here. Please refer to the following: Figures 1-3 In some embodiments, after outputting the data of the first frame and updating the first next scheduling time when the first current time is equal to or later than the first next scheduling time (step S212), the processor 120 may also take step S302 to obtain the time difference between the first current time and the first next scheduling time, and confirm whether the time difference between the first current time and the first next scheduling time exceeds a preset time threshold. The time threshold may be related to the target frame rate. For example, when the target frame rate is 30fps, the preset time threshold may be set to 60ms, but the embodiments of the present invention are not limited to this.

[0033] When the confirmed time difference exceeds a preset time threshold, it indicates that a clock error or delay has occurred, causing frame rate drift. At this time, the processor 120 can take step S304 to update the first next scheduling time, and use this updated first next scheduling time as the basis for subsequently discarding or outputting the data of the first frame. This ensures a long-term stable output frame rate and avoids data stacking or abnormal frame skipping. In some embodiments, the preset time threshold is, for example, 2 × (1000 / max_fps), but the embodiments of the present invention are not limited to this.

[0034] On the other hand, when the confirmed time difference does not exceed the preset time threshold, it indicates that no clock error or delay has caused frame rate drift. At this time, step S306 can be taken, that is, the processor 120 will not update the first next scheduling time, and will use this first next scheduling time as the basis for subsequently discarding or outputting the data of the first frame.

[0035] Next, in step S308, the first target frame rate is dynamically adjusted based on the average inference time. In one embodiment, Figure 1 The processor 120 can also dynamically adjust the first target frame rate based on the average inference time. Furthermore, the processor 120 can automatically reduce the input frame rate to maintain the immediacy of the processing pipeline, avoiding additional latency caused by frame backlog, by considering the processing time of the AI ​​inference task. The average inference time can be obtained, and the first target frame rate can be dynamically adjusted based on the average inference time. This ensures that the processor 120 maintains smooth, real-time operation under different task computational loads. In some embodiments, the adjusted first target frame rate can be calculated using the following formula (2).

[0036] fps_controls.max_fps=std::max(5, 1000 / avg_interfence_time_ms), formula (2)

[0037] Here, fps_controls.max_fps represents the first target frame rate to be adjusted, and avg_interfence_time_ms represents the average inference time.

[0038] Figure 4 This is a flowchart of a smooth frame rate scheduling method according to another embodiment of the present invention. In this embodiment, Figure 4 Steps S202~S212 and Figure 2 Steps S202 to S212 are the same or similar, and can be referred to Figure 2 The embodiments are described in detail here, so they will not be repeated here.

[0039] Please also refer to Figure 1 and Figure 4 In step S402, a second target frame rate is set for the second channel. At this time, the device has at least two channels. In step S404, the data of the second frame is received through the second channel, and the second current time corresponding to the data of the second frame is obtained. In step S406, based on the second target frame rate and the second current time, the second next scheduling time is determined by AI dynamic calculation.

[0040] In step S408, Figure 1In this embodiment, the processor 120 can determine whether the second current time is earlier than the second next scheduling time by comparing the second current time of the second frame data with the second next scheduling time. That is, whenever the processor 120 receives the second frame data, it can obtain the second current time corresponding to the second frame data. Step S408 determines whether the second current time is earlier than the second next scheduling time by comparing the second current time of the second frame data with the second next scheduling time.

[0041] When it is confirmed that the second current time is earlier than the second next scheduling time, proceed to step S410. That is, at this time, if Figure 1 If the processor 120 in this embodiment determines that the data of the second frame has not yet reached the scheduling time, the processor 120 can skip processing the data of the second frame or discard the data of the second frame to ensure stable output frame rate and achieve smooth speed limiting. In addition, the discarded data of the second frame is a selective discard of redundant frame data under high frame rate input. The processor 120 can still ensure that the second channel can output the corresponding frame data according to the preset frame rate.

[0042] When it is confirmed that the second current time is equal to or later than the second next scheduling time, proceed to step S412, output the data of the second frame, and update the second next scheduling time. That is, at this time, Figure 1 In this embodiment, the processor 120 can output the data of the second frame and update the second next scheduling time. In this embodiment, the way the processor 120 outputs the data of the second frame is the same as or similar to the way the processor 120 outputs the data of the first frame. For details on the way the processor 120 outputs the data of the first frame, please refer to the description of the way the processor 120 outputs the data of the first frame, and therefore will not be repeated here.

[0043] In some embodiments, step S412 further includes adding a target time interval to a second current time to obtain a second next scheduling time, thereby updating the second next scheduling time, wherein the target time interval is associated with a second target frame rate. Additionally, Figure 1 The method by which the processor 120 updates the second next scheduling time in the embodiment is the same as or similar to the method by which the processor 120 updates the first next scheduling time. For details, please refer to the description of the method by which the processor 120 updates the first next scheduling time and the formula (1), so it will not be repeated here.

[0044] In some embodiments, after the second current time is equal to or later than the second next scheduling time, the processor 120 obtains the time difference between the second current time and the second next scheduling time. Then, the processor 120 processes the time difference between the second current time and the second next scheduling time in the same or similar manner as it processes the time difference between the first current time and the first next scheduling time, as described in the above embodiments, and therefore will not be repeated here.

[0045] In one embodiment, the processor 120 dynamically adjusts the second target frame rate based on the average inference time. Furthermore, the processor 120 can combine the processing time of the AI ​​inference task to obtain the average inference time, and automatically reduce the input frame rate based on the average inference time to maintain the immediacy of the processing pipeline, avoid additional latency caused by frame backlog, and dynamically adjust the second target frame rate.

[0046] In some embodiments, when the priority of the first channel is higher than that of the second channel, the processor 120 can set the first target frame rate to be higher than the second target frame rate, and then prioritize outputting the relevant data of the first channel. For example, in urban driving scenarios, more attention is paid to vehicle surrounding detection. The first channel corresponds to the first camera device (AVM), and the second channel corresponds to the second camera device (DMS). Therefore, the priority of the first channel is higher than that of the second channel. Thus, the first target frame rate of the first channel can be set to 30fps, and the second target frame rate of the second channel can be set to 15fps. In some embodiments, the processor 120 can perform independent scheduling operations on the first and second channels, so that the first and second channels do not interfere with each other and achieve stable parallel multitasking. In some embodiments, the second channel can also correspond to the second camera device. In some embodiments, the second camera device can be any one of the front-view camera of the AVM camera, the DMS camera, and the BSD camera, but the embodiments of the present invention are not limited to these.

[0047] like Figure 5 The diagram illustrates a smooth frame rate scheduling architecture according to another embodiment of the present invention. The core of this architecture lies in constructing an independent scheduling structure S502 to solve resource contention and frame rate jitter problems during concurrent processing of multiple video streams. In this structure, for example, the first camera device corresponding to the first channel is a Driver Monitoring System (DMS) S504, the second camera device corresponding to the second channel is a Blind Spot Detection System (BSD) S506, and the third camera device corresponding to the third channel is a Panoramic Image System (AVM) S508, among other visual perception channels. Priorities are set according to different work scenarios. For example, in factory driving situations, where the driver's status is of greater concern, the Driver Monitoring System (DMS) is set to the highest priority, with a target frame rate of 30fps to better detect the driver's status and ensure work safety. The second priority is the Blind Spot Detection (BSD), with a target frame rate of 20fps. Less emphasis is placed on the Panoramic Image System (AVM), with a target frame rate of 15fps. The method for setting the target frame rate for each channel can be found in the description of the above embodiment, and will not be repeated here. However, the embodiments of the present invention are not limited to this, and may also include three or more channels.

[0048] Please also refer to Figure 2 , Figure 4 and Figure 5 For each of the aforementioned independent channels, the next scheduling time is calculated separately for each channel, for example, by using the methods described above. Figure 2 or Figure 4 Steps S206 and S406 are performed. The next scheduling time for DMS (S510), BSD (S512), and AVM (S514) is calculated respectively. By dynamically calculating the next scheduling time using AI, the frame rate of each channel is limited, effectively preventing the normal operation of other channels from being interfered with by the load fluctuation of a single channel, and ensuring the stability and independence of the output rhythm of each channel.

[0049] Next, the next scheduling time (S510, S512, S514) signals for each channel of DMS, BSD, and AVM are collected and sent to the underlying processor 120. The processor 120 performs overall coordination based on the received time of each channel and dynamically allocates multiple channels based on the total system bandwidth threshold to prevent bus congestion.

[0050] Please also refer to Figure 1 and Figure 5 In the above embodiments, Figure 1 The processor 120 can perform smooth frame rate scheduling operations on multiple channels (e.g., the first channel and the second channel). This smooth frame rate scheduling operation can be, for example, the processor 120 grouping and scheduling multiple channels based on a total system bandwidth threshold or dynamically allocating bandwidth to prevent bus congestion. Alternatively, the smooth frame rate scheduling operation can also dynamically adjust the video input frame rate based on the real-time load of the AI ​​task queue, or skip processing of the current frame data or discard the current frame data. For example, when AI processing time increases, the input frame rate is automatically reduced to maintain the real-time performance of the processing pipeline and avoid additional latency caused by frame backlog. This achieves smooth, stable, and low-latency frame rate control for multi-channel input. However, the embodiments of the present invention are not limited to this. In some embodiments, the processor 120 can perform smooth frame rate scheduling operations on three or more channels (e.g., the first channel, the second channel, and the third channel), for example, dynamically adjusting the video input frame rate based on the real-time load of the AI ​​task queue, or skipping processing of the current frame data or discarding the current frame data. Furthermore, the operation of smoothing frame rate scheduling for the third channel and multiple channels is the same as or similar to the operation of smoothing frame rate scheduling for the first channel or the second channel. Please refer to the description in the above embodiment, so it will not be repeated here.

[0051] In one embodiment, for the forward-looking camera in the AVM, there is often a situation where frames are dropped due to insufficient computing power. When the detection frame rate is lower than the input frame rate, the system needs to call the predictive frame interpolation mechanism, such as using the Kalman filter algorithm, to output a continuous and smooth target trajectory.

[0052] In summary, the smooth frame rate scheduling method and apparatus disclosed in this invention set a first target frame rate for a first channel. Data of the first frame is received through the first channel, and the first current time corresponding to the data of the first frame is obtained. Based on the first target frame rate and the first current time, a first next scheduling time is determined. The first current time of the first frame data is compared with the first next scheduling time to confirm whether the first current time is earlier than the first next scheduling time. When the first current time is earlier than the first next scheduling time, processing of the data of the first frame is skipped. When the first current time is equal to or later than the first next scheduling time, the data of the first frame is output, and the first next scheduling time is updated. In this way, a stable output frame rate and smooth rate limiting can be achieved, increasing ease of use.

[0053] It is worth noting that, Figure 2 , Figure 3 , Figure 4 and Figure 5 The order of steps and architecture described above is for illustrative purposes only and is not intended to limit the order of steps in the embodiments of the present invention. The order of the steps described above can be changed by the user according to their needs. Furthermore, additional steps can be added or fewer steps can be used without departing from the spirit and scope of the present invention.

[0054] The above description is only a preferred embodiment of the present invention, but it is not intended to limit the scope of the present invention. Any person skilled in the art can make further improvements and changes on this basis without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the scope defined in the claims of this application.

Claims

1. A smooth frame rate scheduling method, characterized in that, include: Set the first target frame rate for the first channel; The data of the first frame is received through the first channel, and the first current time corresponding to the data of the first frame is obtained; The first next scheduling time is determined based on the first target frame rate and the first current time; Compare the first current time of the data in the first frame with the first next scheduling time to confirm whether the first current time is earlier than the first next scheduling time; When it is confirmed that the first current time is earlier than the first next scheduling time, the processing of the data for the first frame is skipped; When it is confirmed that the first current time is equal to or later than the first next scheduling time, the data of the first frame is output and the first next scheduling time is updated.

2. The smooth frame rate scheduling method according to claim 1, characterized in that, When it is confirmed that the first current time is earlier than the first next scheduling time, the step of skipping the processing of the data of the first frame also includes discarding the data of the first frame.

3. The smooth frame rate scheduling method according to claim 1, characterized in that, The steps to update the first next scheduling time include: Confirm whether the time difference between the first current time and the first next scheduling time exceeds a preset time threshold; When it is confirmed that the time difference exceeds the preset time threshold, the first next scheduling time is updated; and If it is confirmed that the time difference does not exceed the preset time threshold, the first next scheduling time will not be updated.

4. The smooth frame rate scheduling method according to claim 1, characterized in that, The steps to update the first next scheduling time include: The first current time is added to the target time interval to obtain the first next scheduling time, and the first next scheduling time is updated, wherein the target time interval is associated with the first target frame rate.

5. The smooth frame rate scheduling method according to claim 4, characterized in that, The updated first next scheduling time is shown in the following formula: next_dipatich_time=current_time+(1000 / max_fps), Where next_dipatich_time represents the updated first next scheduling time, current_time represents the current time, (1000 / max_fps) represents the target time interval, and max_fps represents the first target frame rate.

6. The smooth frame rate scheduling method according to claim 1, characterized in that, Also includes: Set a second target frame rate for the second channel; The second frame of data is received through the second channel, and the second current time corresponding to the data of the second frame is obtained; Based on the second target frame rate and the second current time, determine the second next scheduling time; Compare the second current time of the data in the second frame with the second next scheduling time to confirm whether the second current time is earlier than the second next scheduling time; When it is confirmed that the second current time is earlier than the second next scheduling time, the processing of the data of the second frame is skipped and the data of the second frame is discarded; When it is confirmed that the second current time is equal to or later than the second next scheduling time, the data of the second frame is output and the second next scheduling time is updated.

7. The smooth frame rate scheduling method according to claim 6, characterized in that, It also includes setting corresponding target frame rates for three or more channels; Based on the system bandwidth threshold, group scheduling or dynamic bandwidth allocation are performed on these three or more channels respectively; and The target frame rate of the three or more channels is dynamically adjusted based on the real-time load of the task queue.

8. The smooth frame rate scheduling method according to claim 6, characterized in that, Also includes: The first target frame rate and the second target frame rate are dynamically adjusted based on the average inference time.

9. The smooth frame rate scheduling method according to claim 8, characterized in that, The average inference time is calculated based on the processing time of multiple inference tasks.

10. The smooth frame rate scheduling method according to claim 6, characterized in that, When the priority of the first channel is higher than that of the second channel, the first target frame rate is greater than that of the second target frame rate.

11. A smooth frame rate scheduling device, characterized in that, include: Storage device, stored program; as well as The processor executes the program to perform: Set the first target frame rate for the first channel; The data of the first frame is received through the first channel, and the first current time corresponding to the data of the first frame is obtained; The first next scheduling time is determined based on the first target frame rate and the first current time; Compare the first current time of the data in the first frame with the first next scheduling time to confirm whether the first current time is earlier than the first next scheduling time; When it is confirmed that the first current time is earlier than the first next scheduling time, the processing of the data for the first frame is skipped; When it is confirmed that the first current time is equal to or later than the first next scheduling time, the data of the first frame is output and the first next scheduling time is updated.

12. The smooth frame rate scheduling device according to claim 11, characterized in that, The processor also performs the following actions: when it is confirmed that the first current time is earlier than the first next scheduling time, skip processing of the data for the first frame, and also discards the data for the first frame.

13. The smooth frame rate scheduling device according to claim 11, characterized in that, The steps to update the first next scheduling time include: Confirm whether the time difference between the first current time and the first next scheduling time exceeds a preset time threshold; When it is confirmed that the time difference exceeds the preset time threshold, the first next scheduling time is updated; and If it is confirmed that the time difference does not exceed the preset time threshold, the first next scheduling time will not be updated.

14. The smooth frame rate scheduling device according to claim 11, characterized in that, The processor also performs: The first current time is added to the target time interval to obtain the first next scheduling time, and the first next scheduling time is updated, wherein the target time interval is associated with the first target frame rate.

15. The smooth frame rate scheduling device according to claim 14, characterized in that, The updated first next scheduling time is shown in the following formula: next_dipatich_time=current_time+(1000 / max_fps), Where next_dipatich_time represents the updated first next scheduling time, current_time represents the current time, (1000 / max_fps) represents the target time interval, and max_fps represents the first target frame rate.

16. The smooth frame rate scheduling device according to claim 11, characterized in that, The processor also performs: Set a second target frame rate for the second channel; The second channel receives a second frame of data and obtains the second current time corresponding to the second frame of data. Based on the second target frame rate and the second current time, determine the second next scheduling time; Compare the second current time of the data in the second frame with the second next scheduling time to confirm whether the second current time is earlier than the second next scheduling time; When it is confirmed that the second current time is earlier than the second next scheduling time, the processing of the data of the second frame is skipped and the data of the second frame is discarded; as well as When it is confirmed that the second current time is equal to or later than the second next scheduling time, the second frame data is output and the second next scheduling time is updated.

17. The smooth frame rate scheduling device according to claim 16, characterized in that, The processor also performs: Set corresponding target frame rates for three or more channels; Based on the system bandwidth threshold, group scheduling or dynamic bandwidth allocation are performed on these three or more channels respectively; and The target frame rate of the three or more channels is dynamically adjusted based on the real-time load of the task queue.

18. The smooth frame rate scheduling device according to claim 16, characterized in that, The processor also performs: The first target frame rate and the second target frame rate are dynamically adjusted based on the average inference time.

19. The smooth frame rate scheduling device according to claim 18, characterized in that, The average inference time is calculated based on the processing time of multiple inference tasks.

20. The smooth frame rate scheduling device according to claim 16, characterized in that, When the priority of the first channel is higher than that of the second channel, the first target frame rate is greater than that of the second target frame rate.

Images