Intelligent terminal computing power distribution and processing control method based on vehicle-mounted video stream

Abstract

The application provides a kind of intelligent terminal computing power distribution and processing control method based on vehicle-mounted video stream, which comprises: generating video processing task based on vehicle-mounted video stream and user service intention, constructing task dependency graph based on the interdependence of each video processing task in input data area, output data use and execution timing;Topological sorting is carried out on the task dependency graph to generate a task execution sequence;Based on the computing complexity characteristics of each task in the task execution sequence and the total amount of schedulable computing power resources of the intelligent terminal, allocate corresponding computing power resource share for each task;According to the computing power resource share, generate binding instructions for processor core and acceleration unit to obtain task scheduling instruction set;After monitoring that each task in the task execution sequence completes its computing process, release the occupied computing power resources.The application improves the real-time response certainty and computing power resource utilization rate of the vehicle-mounted intelligent terminal in complex and variable road environment.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method for intelligent terminal computing power allocation and processing control based on vehicle-mounted video streams. Background Technology

[0002] In intelligent vehicle systems, to achieve real-time perception and business response to the vehicle's surroundings, existing technologies typically employ a fixed computing power allocation strategy to process in-vehicle video streams. Specifically, this method statically divides the computing resources of the in-vehicle terminal into multiple business modules according to a preset ratio, such as lane departure warning, forward collision detection, and traffic sign recognition. Each module independently calls upon its allocated computing power resources to analyze the video stream and outputs control commands or prompts based on its respective business logic.

[0003] However, because the aforementioned methods statically allocate computing resources, they cannot flexibly schedule them according to the dynamic changes in the actual business intent. This leads to delays or even failures in some critical business processes due to insufficient computing power in certain high-load scenarios (such as multiple advanced driver assistance functions being triggered simultaneously in congested urban areas). Conversely, in low-load scenarios (such as highway cruising where only basic perception needs to be maintained), a large amount of computing resources are idle and wasted. Therefore, this "one-size-fits-all" computing power allocation mechanism severely restricts the real-time response capability and resource utilization efficiency of in-vehicle intelligent terminals in complex and ever-changing road environments. Summary of the Invention

[0004] This invention provides a method for intelligent terminal computing power allocation and processing control based on vehicle video stream, which improves the real-time response determinism and computing resource utilization of vehicle intelligent terminals in complex and ever-changing road environments.

[0005] In a first aspect, the present invention provides a method for intelligent terminal computing power allocation and processing control based on in-vehicle video streams, comprising:

[0006] Based on the in-vehicle video stream and user business intent, task analysis is performed to generate video processing tasks. Based on the interdependencies of each video processing task in terms of input data area, output data purpose, and execution timing, a task dependency graph is constructed. Each video processing task corresponds to a visual analysis operation.

[0007] Perform topological sorting on the task dependency graph to generate a task execution sequence that satisfies the dependency constraints;

[0008] Based on the computational complexity characteristics of each task in the task execution sequence and the total schedulable computing power resources of the smart terminal, a corresponding share of computing power resources is allocated to each task.

[0009] Based on the computing power resource share, binding instructions for processor cores and acceleration units are generated to obtain a task scheduling instruction set. After the execution status of the task execution sequence is monitored based on the task scheduling instruction set and each task has completed its calculation process, the occupied computing power resources are released.

[0010] Secondly, the present invention also provides a computing power allocation and processing control system for intelligent terminals based on in-vehicle video streams, applied to the intelligent terminal computing power allocation and processing control method based on in-vehicle video streams as described in the first aspect; the intelligent terminal computing power allocation and processing control system based on in-vehicle video streams includes:

[0011] The video task analysis module is used to perform task analysis based on the vehicle video stream and user business intent, generate video processing tasks, and construct a task dependency graph based on the interdependencies of each video processing task in terms of input data area, output data purpose, and execution timing; each video processing task corresponds to a visual analysis operation.

[0012] The task topology sorting module is used to perform topological sorting on the task dependency graph and generate a task execution sequence that satisfies the dependency constraints.

[0013] The computing power allocation module is used to allocate a corresponding share of computing power resources to each task based on the computational complexity characteristics of each task in the task execution sequence and the total schedulable computing power resources of the smart terminal.

[0014] The task scheduling module is used to generate binding instructions for processor cores and acceleration units based on the computing power resource share, obtain a task scheduling instruction set, and release the occupied computing power resources after the execution status monitoring based on the task scheduling instruction set detects that each task in the task execution sequence has completed its calculation process.

[0015] Thirdly, the present invention also provides an electronic device, comprising: a memory for storing computer software programs; and a processor for reading and executing the computer software programs, thereby realizing the intelligent terminal computing power allocation and processing control method based on vehicle video stream as described above.

[0016] Fourthly, the present invention also provides a non-transitory computer-readable storage medium storing a computer software program, which, when executed by a processor, implements the above-described intelligent terminal computing power allocation and processing control method based on vehicle video stream.

[0017] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described intelligent terminal computing power allocation and processing control method based on in-vehicle video stream.

[0018] The intelligent terminal computing power allocation and processing control method based on in-vehicle video stream provided in this invention generates video processing tasks based on in-vehicle video stream and user business intent. It constructs a task dependency graph by combining the interdependencies of each task in input data area, output data purpose, and execution sequence. Therefore, it can accurately match the current business intent, clarify the task set and dependency logic associated with each visual analysis task, and break the limitations of independent and isolated business modules. By associating with user business intent, video processing tasks are no longer generated indiscriminately, but precisely meet the needs of the current actual driving scenario, avoiding the consumption of computing power by tasks unrelated to business intent from the source. The task dependency graph is topologically sorted to obtain a task execution sequence that satisfies dependency constraints and is orderly connected. Therefore, the sorting process solves the resource conflict problem caused by disordered access to computing power by various modules. By clarifying the order of task execution, it ensures that subsequent tasks are executed only after the prerequisite tasks are completed, avoiding computing power consumption caused by chaotic parallel task execution and ensuring the orderly use of computing power. By combining the computational complexity characteristics of each task in the task execution sequence with the total schedulable computing resources of the smart terminal, a corresponding share of computing resources is allocated to each task. This achieves an on-demand, dynamically adaptable computing resource allocation scheme, solving the pain point of fixed-ratio static allocation. Instead of a "one-size-fits-all" approach, computing power is dynamically adjusted based on the actual complexity of the task and the total resource availability, achieving precise matching between computing resources and task requirements. Binding instructions for processor cores and acceleration units are generated based on the computing resource shares, resulting in a task scheduling instruction set that precisely targets hardware resources. This ensures the effective execution of the computing resource allocation scheme, allowing allocated computing resources to accurately connect to corresponding hardware units. This avoids resource waste or call delays caused by a disconnect between computing resource allocation and hardware calls, improving computing power utilization efficiency. The task scheduling instruction set is executed, and the task execution status is monitored. Once all tasks are completed, the occupied computing resources are released, enabling dynamic recycling and reuse of computing resources. This solves the problem of idle computing power in low-load scenarios. Timely resource release after task completion allows idle computing power to be quickly allocated to new tasks, achieving the cyclical reuse of computing resources. Therefore, the embodiments of the present invention realize the rational allocation of computing resources in high-load scenarios and the efficient utilization in low-load scenarios, thereby improving the flexibility and efficiency of computing power scheduling, solving the problems of response delay in high-load scenarios and computing power waste in low-load scenarios caused by static allocation, and improving the real-time response determinism and computing power resource utilization of vehicle-mounted intelligent terminals in complex and ever-changing road environments. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the intelligent terminal computing power allocation and processing control method based on vehicle video stream provided in an embodiment of the present invention.

[0020] Figure 2This is a schematic diagram of the intelligent terminal computing power allocation and processing control system based on vehicle video stream provided in an embodiment of the present invention;

[0021] Figure 3 An embodiment diagram of the electronic device provided in this invention;

[0022] Figure 4 An embodiment diagram of a computer-readable storage medium provided in accordance with the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Optionally, see Figure 1 , Figure 1 This is a flowchart illustrating the intelligent terminal computing power allocation and processing control method based on in-vehicle video stream provided by the present invention. In this embodiment of the invention, the executing entity of the intelligent terminal computing power allocation and processing control method based on in-vehicle video stream is the in-vehicle terminal system. Therefore, the intelligent terminal computing power allocation and processing control method based on in-vehicle video stream includes:

[0025] Step 10: Perform task analysis based on the in-vehicle video stream and user business intent to generate video processing tasks, and construct a task dependency graph based on the interdependencies of each video processing task in terms of input data area, output data purpose, and execution timing.

[0026] Optionally, the vehicle terminal system acquires the vehicle video stream, which refers to a real-time continuous image sequence captured by the vehicle camera. This image sequence contains various visual information in the vehicle scene, such as road conditions, pedestrians around the vehicle, traffic signs, and visual images related to the vehicle's own status. The vehicle terminal system receives the vehicle video stream in real time through the video acquisition interface inside the vehicle.

[0027] Secondly, the in-vehicle terminal system identifies the user's business intent. User business intent refers to the specific business request initiated by the user through the in-vehicle terminal system that needs to be completed based on the in-vehicle video stream. The in-vehicle terminal system clarifies the user's business intent by parsing the user's input commands and identifying the business function modules triggered by the user. The user's input commands include, but are not limited to, voice control commands, touch operation commands, and physical button commands. The business function modules triggered by the user include, but are not limited to, the in-vehicle driver assistance module, the in-vehicle video monitoring module, and the in-vehicle scene recognition module. For example, if the user issues a voice command to "recognize the traffic sign ahead," the in-vehicle terminal system parses the command and clarifies that the user's business intent is to complete the recognition of the traffic sign ahead based on the in-vehicle video stream. If the user triggers the lane departure warning function in the in-vehicle driver assistance module through touch, the in-vehicle terminal system clarifies that the user's business intent is to complete lane line recognition and lane departure judgment based on the in-vehicle video stream, ensuring the accuracy of user business intent recognition.

[0028] Furthermore, the in-vehicle terminal system performs task analysis based on the acquired in-vehicle video stream and the identified user business intent, generating video processing tasks. During task analysis, the user's business intent is broken down into several independently executable sub-tasks that can collaboratively complete the overall business intent. Each sub-task corresponds to a specific requirement for processing the in-vehicle video stream. Subsequently, for each decomposed sub-task, the in-vehicle terminal system, combined with the specific content of the in-vehicle video stream, clarifies the corresponding video processing requirements, and then generates the corresponding video processing task.

[0029] Each video processing task corresponds to a visual analysis operation. The visual analysis operation refers to the processing and analysis of image sequences in the vehicle video stream, extracting effective visual information and completing specific visual judgments, including but not limited to image segmentation, target detection, feature extraction, image enhancement, target tracking, and image recognition. For example, if the user's business intent is forward collision warning in vehicle-assisted driving, the vehicle terminal system will break down this business intent into three sub-tasks: "forward vehicle detection," "forward pedestrian detection," and "vehicle-pedestrian distance calculation." These will generate three corresponding video processing tasks: "performing visual analysis operation for forward vehicle detection based on the vehicle video stream," "performing visual analysis operation for forward pedestrian detection based on the vehicle video stream," and "performing visual analysis operation for vehicle-pedestrian distance calculation based on the vehicle video stream." Each video processing task clearly defines the processing object as the vehicle video stream and the processing purpose as completing the corresponding sub-task, ensuring that the generated video processing tasks accurately match the user's business intent, and that each video processing task has clear functional boundaries and can be executed independently.

[0030] Furthermore, the vehicle terminal system constructs a task dependency graph based on the interdependencies of each video processing task in terms of input data area, output data purpose, and execution timing, as detailed in steps 101 to 104.

[0031] Step 20: Perform topological sorting on the task dependency graph to generate a task execution sequence that satisfies the dependency constraints.

[0032] Optionally, topological sorting refers to sorting all nodes in a directed acyclic graph so that for any two nodes connected by a directed edge in the graph, the task corresponding to the node that appears earlier in the graph can be executed before the task corresponding to the node that appears later in the graph. For the task dependency graph constructed in step 10, it is essentially a directed acyclic graph (because there is no circular dependency between video processing tasks; if there were a circular dependency, all tasks could not be executed, so the task dependency graph must be a directed acyclic graph). The purpose of topological sorting this graph is to eliminate dependency conflicts between tasks and ensure that the generated task execution sequence can satisfy the dependency constraints between all video processing tasks, that is, all dependent video processing tasks can be executed before the video processing tasks that depend on it.

[0033] Optionally, the specific process of the vehicle-mounted terminal system performing topology sorting is as follows:

[0034] The first step is for the vehicle terminal system to traverse the task dependency graph and identify all video processing tasks without predecessor nodes. A predecessor node refers to a video processing task that needs to be executed before the current task in the dependency relationship and whose output data provides input data for the current task. A video processing task without a predecessor node means that the task does not depend on any other video processing tasks and can be started and executed directly. All video processing tasks without predecessor nodes are selected as the starting task set for topological sorting.

[0035] The second step is for the vehicle terminal system to select any video processing task from the initial task set, add the task to the end of the task execution sequence, and delete the node corresponding to the video processing task in the task dependency graph, as well as all directed edges originating from that node, that is, delete all dependencies that depend on the task.

[0036] The third step is to check whether a new video processing task without a predecessor node appears in the task dependency graph after deleting the above nodes and directed edges. If so, add it to the starting task set.

[0037] Fourth, repeat steps two and three until all nodes in the task dependency graph are deleted. At this point, the generated task sequence is the task execution sequence that satisfies the dependency constraints. If, during the traversal, a remaining node is found in the task dependency graph, but no task without a predecessor node can be found, it indicates that the task dependency graph has a circular dependency. In this case, the vehicle terminal system will trigger an error message, informing the relevant system that there is a dependency error and that the video processing tasks and dependencies need to be re-examined to ensure that the topology sorting can be completed normally.

[0038] Finally, the vehicle-mounted terminal system verifies the generated task execution sequence. The verification involves checking whether the dependencies of all video processing tasks in the task execution sequence are satisfied. That is, for any video processing task in the task execution sequence, all video processing tasks that provide input data to it and need to be executed before it are placed before this task. If the verification passes, the task execution sequence is determined to be the final executable task execution sequence; if the verification fails, the topological sorting operation is re-executed until a task execution sequence that meets the dependency constraints is generated.

[0039] Step 30: Based on the computational complexity characteristics of each task in the task execution sequence and the total schedulable computing power resources of the smart terminal, allocate a corresponding share of computing power resources to each task.

[0040] Optionally, computational complexity characteristics refer to the specific characteristics of the amount of computing resources required for each video processing task during execution, reflecting the computational burden of the video processing task. The relevant parameters of computational complexity characteristics include, but are not limited to, the number of operations required for task execution, the amount of memory required for task execution, and the time cost of task execution. Different types of video processing tasks have different computational complexity characteristics. For example, the computational complexity of image recognition video processing tasks is usually higher than that of image enhancement video processing tasks because it requires a large number of feature extraction and matching operations in the image sequence.

[0041] The total schedulable computing resources of a smart terminal refer to the sum of all computing resources available for executing video processing tasks. The computing resources of an in-vehicle terminal system include, but are not limited to, the computing resources of the central processing unit, the computing resources of the graphics processing unit, and the computing resources of the dedicated acceleration chip. The total schedulable computing resources are the remaining computing resources that can be specifically used to execute the current video processing task sequence after deducting the computing resources required for the basic operation of the in-vehicle terminal system itself and the computing resources required for other non-video processing services. The in-vehicle terminal system calculates the current total schedulable computing resources by real-time monitoring of the resource occupancy of its processors and acceleration units, ensuring the accuracy of the total schedulable computing resources and avoiding unreasonable subsequent allocation due to statistical errors in computing resources.

[0042] Optionally, the vehicle terminal system allocates a corresponding share of computing resources to each task based on the computational complexity characteristics of each task in the task execution sequence and the total schedulable computing resources of the smart terminal, as in steps 301 to 304.

[0043] Step 40: Generate binding instructions for processor cores and acceleration units based on the computing power resource share to obtain a task scheduling instruction set. After the execution status monitoring based on the task scheduling instruction set shows that each task in the task execution sequence has completed its calculation process, release the occupied computing power resources.

[0044] Optionally, a processor core refers to a core unit in a computing chip such as a central processing unit or a graphics processing unit that has independent computing capabilities, and each processor core can independently execute computing tasks.

[0045] Acceleration units refer to hardware units specifically designed to accelerate specific types of computations, such as video processing acceleration units and neural network computation acceleration units. They can significantly improve the execution efficiency of corresponding types of video processing tasks. Binding instructions refer to generated instructions used to associate specific video processing tasks with designated processor cores or acceleration units. Their function is to explicitly specify which processor cores or acceleration units will execute each video processing task, ensuring that the share of computing resources allocated to each video processing task is accurately implemented and preventing computing resources from being occupied by other tasks.

[0046] The task scheduling instruction set refers to the set of binding instructions corresponding to all video processing tasks, as well as control instructions related to task execution. It is the core basis for the vehicle terminal system to schedule the execution of video processing tasks by each processor core and acceleration unit.

[0047] Optionally, the vehicle-mounted terminal system generates binding instructions for processor cores and acceleration units based on the allocated computing power resource share for each video processing task. The specific generation process is as follows: the vehicle-mounted terminal system determines the number and type of processor cores and acceleration units corresponding to the allocated computing power resource share for each video processing task, i.e., it determines how many processor cores and which types of processor cores each task requires, and whether, how many, and which types of acceleration units are needed. Subsequently, for each video processing task, a corresponding binding instruction is generated. The binding instruction clearly includes the identification information of the video processing task, the identification information of the processor cores to be bound, the identification information of the acceleration units to be bound, and the effective duration of the binding. That is, during the execution of the video processing task, the specified processor cores and acceleration units are dedicated to serving that task.

[0048] For example, if the computing power allocated to a certain image recognition video processing task corresponds to 2 central processing unit cores and 1 video processing acceleration unit, the vehicle terminal system generates a binding instruction to specify that the image recognition task is bound to the central processing unit cores identified as 1 and 2 and the video processing acceleration unit identified as A, and the binding validity period is the entire execution period of the image recognition task.

[0049] After generating the binding instructions corresponding to all video processing tasks, the vehicle terminal system integrates all binding instructions with task execution-related control instructions (including but not limited to task start instructions, task pause instructions, and task termination instructions) to obtain a task scheduling instruction set. Simultaneously, the task scheduling instruction set is verified. The verification checks whether the processor core and acceleration unit corresponding to each binding instruction are in an idle state, whether the content of the binding instruction is consistent with the computing power resource allocation requirements, and whether the control instructions are complete. If the verification passes, the task scheduling instruction set is determined to be the final executable instruction set; if the verification fails, the binding instructions are readjusted until a compliant task scheduling instruction set is generated.

[0050] Then, the vehicle-mounted terminal system executes the task scheduling instruction set and monitors the execution status of each video processing task in the task execution sequence in real time. Optionally, the specific monitoring process in this embodiment of the invention is as follows: the vehicle-mounted terminal system starts each video processing task through the task scheduling instruction set, and at the same time, through the status monitoring module inside the vehicle-mounted terminal system, it collects the execution progress of each video processing task, the operating status of the corresponding processor core and acceleration unit, and error information during task execution in real time, so as to determine whether each video processing task has completed its calculation process. Among them, the completion of the task calculation process means that the video processing task has completed the visual analysis operation of the vehicle video stream according to the preset processing requirements, generated a processing result that meets the requirements, and the processing result has been stored in the preset output area. At the same time, the processor core and acceleration unit corresponding to the task have stopped the operation for the task.

[0051] If a video processing task is detected as not having completed its calculation process and no error message is displayed, the vehicle terminal system will continue monitoring to ensure the task is executed normally. If an error message is detected for a video processing task, the vehicle terminal system will trigger an exception handling mechanism, suspend the execution of the task, adjust the binding instructions of the task according to the error message (such as rebinding an idle processor core or acceleration unit), restart the execution of the task, and continue monitoring.

[0052] Finally, once the vehicle-mounted terminal system detects through the status monitoring module that each video processing task in the task execution sequence has completed its calculation process, it immediately releases the computing resources occupied by each task. Optionally, the specific process of releasing computing resources in this embodiment of the invention is as follows: the vehicle-mounted terminal system generates a computing resource release instruction, which includes the identification information of the processor cores and acceleration units occupied by each completed task, as well as the scope of release (i.e., completely releasing all computing resources occupied by the task); subsequently, the vehicle-mounted terminal system executes the computing resource release instruction, releases the binding relationship between each completed task and the corresponding processor core and acceleration unit, stops the task from occupying the processor cores and acceleration units, and restores all released processor cores and acceleration units to an idle state, so that they can be rescheduled and used by other services of the vehicle-mounted terminal system or subsequent video processing tasks.

[0053] The embodiments of the present invention realize the rational allocation of computing resources in high-load scenarios and the efficient utilization in low-load scenarios, thereby improving the flexibility and efficiency of computing power scheduling. It solves the problems of response delay in high-load scenarios and computing power waste in low-load scenarios caused by static allocation, and improves the real-time response determinism and computing power resource utilization of vehicle-mounted intelligent terminals in complex and ever-changing road environments.

[0054] Optionally, the processes of steps 101 to 104 include:

[0055] Step 101: Determine the spatiotemporal boundary of the data source for each video processing task based on the spatial range and timestamp interval of the input data region specified for each video processing task in the video frame coordinate system.

[0056] Optionally, the input data region refers to the specific area of ​​the vehicle video stream data that needs to be read during the execution of each video processing task. This region is a part of the overall data of the vehicle video stream and only contains the relevant data required for the visual analysis operation of this video processing task. The video frame coordinate system refers to a two-dimensional coordinate system set for each video frame in the vehicle video stream, used to accurately locate the position of each pixel in the video frame. The origin of the video frame coordinate system is set to the upper left corner of the video frame, with the horizontal axis pointing to the right and the vertical axis pointing downwards. Each point in the coordinate system corresponds to a unique pixel position in the video frame, measured in pixels. The spatial range refers to the pixel area occupied by the input data region in the video frame coordinate system, defined by the horizontal and vertical coordinate ranges in the coordinate system, clarifying the specific position and size of the input data in a single video frame. The timestamp interval refers to the time range of the in-vehicle video stream corresponding to the input data area. A timestamp is a time marker added to each video frame in the in-vehicle video stream to identify the specific time the video frame was captured. The timestamp interval consists of a start timestamp and an end timestamp, defining the time range of the video frame sequence corresponding to the input data. The data source spatiotemporal boundary refers to the boundary information used to clarify the temporal and spatial ranges of the input data required for each video processing task. It is the core basis for distinguishing the input data sources for different video processing tasks and includes both temporal and spatial boundaries.

[0057] Optionally, the vehicle terminal system obtains the input data region information specified for each video processing task. This input data region information is specified by the vehicle terminal system for each video processing task when generating the video processing task. It clarifies that the input data required by the task comes from the specific part of the vehicle video stream, including the spatial range information of the input data region in the video frame coordinate system and the corresponding timestamp interval information.

[0058] Furthermore, the vehicle-mounted terminal system analyzes the spatial range of the input data area for each video processing task. Based on the range of the horizontal and vertical coordinates of the input data area in the video frame coordinate system, it determines the spatial boundary of the input data area. Specifically, the spatial boundary is represented by the coordinates of the top left, top right, bottom left, and bottom right vertices of the input data area. By accurately defining the specific spatial location and coverage of the input data area in the video frame coordinate system through the coordinates of these four vertices, the uniqueness and accuracy of the spatial boundary are ensured, avoiding ambiguity in the spatial range definition of different tasks.

[0059] Furthermore, the vehicle-mounted terminal system parses the timestamp interval of the input data region for each video processing task, extracting the start and end timestamps corresponding to the input data region to determine its temporal boundaries. These temporal boundaries clearly define the start and end times of the vehicle-mounted video stream required for the video processing task, defining the time range of the corresponding video frame sequence and ensuring the clear discernibility of the temporal boundaries. The vehicle-mounted terminal system then integrates the parsed spatial and temporal boundaries to form the spatiotemporal boundaries of the data source for each video processing task.

[0060] Step 102: Based on the spatiotemporal boundaries of the data source for each video processing task, identify whether there are shared input data regions between different video processing tasks, and obtain the shared region identification results.

[0061] Optionally, the shared input data area refers to two or more different video processing tasks whose specified input data areas overlap. The in-vehicle video stream data in the overlapping part can be read and used by these different video processing tasks simultaneously without repeated acquisition or storage.

[0062] Shared region identification results refer to the results obtained after the identification operation, which are used to determine whether there are shared input data regions between different video processing tasks, and the specific information of the shared input data regions. This includes information such as the task combination with shared input data regions, the spatiotemporal boundaries of the data source of the shared regions, and the size of the shared regions.

[0063] Optionally, the process of the vehicle-mounted terminal system performing the recognition operation of the shared input data area is as follows: First, select any video processing task from all video processing tasks as the current task and extract the spatiotemporal boundary of the data source of the current task; Second, select any other video processing task besides the current task as the comparison task and extract the spatiotemporal boundary of the data source of the comparison task; Third, compare the spatiotemporal boundaries of the data sources of the current task and the comparison task. First, compare the time boundaries of the two tasks to determine whether the timestamp intervals of the two tasks overlap. If the timestamp intervals of the two tasks do not overlap, that is, the timestamp interval of one task is completely before or after the timestamp interval of the other task, it can be directly determined that the two tasks do not share an input data area; if the timestamp intervals of the two tasks overlap, then further compare the spatial boundaries of the two tasks to determine whether the spatial ranges of the two tasks overlap. If there is overlap, it means determining whether the area defined by the spatial boundary of the current task intersects with the area defined by the spatial boundary of the comparison task. If there is an intersection, it is determined that the two tasks share a common input data area, and this intersection area is the shared input data area. At the same time, the spatiotemporal boundary of the data source of the shared area (i.e., the overlapping timestamp interval and the overlapping spatial range) and the corresponding identification information of the two tasks are recorded. If the spatial ranges do not overlap, it is determined that the two tasks do not share a common input data area. Fourth step, repeat the operations of the second and third steps, comparing the current task with all other video processing tasks one by one to complete the identification of the shared area of ​​the current task with all other tasks. Fifth step, change the current task, select the video processing task that is not the current task as the new current task, and repeat the operations of the second to fourth steps until all video processing tasks have completed pairwise comparisons.

[0064] Finally, the vehicle-mounted terminal system organizes all identified shared input data areas and their corresponding task combinations, removes duplicate records (i.e., cases where the same shared area is identified multiple times), clarifies all video processing tasks corresponding to each shared input data area, and forms a complete shared area identification result.

[0065] Step 103: Based on the shared input data region identified in the shared region identification results, and combined with the downstream consumption task set specified by the output data usage of each video processing task, the initial one-way usage orientation relationship between each video processing task is obtained.

[0066] Optionally, the output data purpose refers to the specific purpose of the output data generated after each video processing task is completed, i.e., which other video processing tasks or business modules the output data will be used for; the downstream consumption task set refers to the set of other video processing tasks that will use the output data of each video processing task, as explicitly specified by the output data purpose of each video processing task. Each element in the set is a specific video processing task. The downstream consumption task set is specified synchronously for each video processing task when the video processing task is generated, and corresponds to the output data purpose; the initial one-way purpose pointing relationship refers to the one-way relationship used to indicate that any video processing task can only be started after its predecessor task has completed its output. Here, the predecessor task refers to the video processing task that provides input data for the current task (including directly providing output data as input data, or tasks that share input data with the current task and need to be executed first). The direction of the initial one-way purpose pointing relationship is from the predecessor task to the current task, clearly reflecting the sequential execution constraints between tasks.

[0067] Optionally, the vehicle terminal system generates an initial one-way usage pointer relationship based on the above data. The specific generation process is as follows: First, it traverses all shared input data regions in the shared region recognition results. For each shared input data region, it extracts all video processing tasks corresponding to that shared region and forms a shared task group. Second, for each shared task group, the vehicle terminal system analyzes the output data usage of each video processing task in the group and the corresponding downstream consumption task set. It determines whether there is a downstream consumption relationship between the video processing tasks in the group, that is, whether the downstream consumption task set of a certain video processing task in the group contains other video processing tasks in the group. If so, it indicates that there is a downstream consumption relationship, and the video processing task is a prerequisite task for its downstream consumption task. An initial one-way usage pointer relationship is generated from the video processing task to its downstream consumption task, clearly indicating that the downstream consumption task can only be started after the video processing task has completed its output.

[0068] Third, if there is no downstream consumption relationship between video processing tasks within a shared task group, that is, the downstream consumption task set of all video processing tasks in the group does not include other tasks in the group, the vehicle terminal system determines whether the video stream data corresponding to the shared input data area needs to be processed by a certain video processing task before it can be used by other tasks (i.e., determine whether there is a task that preprocesses the shared input data area). If there is a preprocessing task, an initial one-way usage pointing relationship is generated from the preprocessing task to other tasks using the shared input data area, explicitly indicating that other tasks can only start after the preprocessing task has completed its output. If there is no preprocessing task, the initial one-way usage pointing relationship is not generated between the video processing tasks within the shared task group for the time being, and their execution order will be further determined in subsequent steps in combination with execution timing constraints.

[0069] The fourth step is to traverse all video processing tasks. For each video processing task, extract all downstream consumption tasks from its downstream consumption task set. For each downstream consumption task, if the downstream consumption task and the current video processing task are not in the same shared task group (i.e., they do not share an input data area), but the output data of the current video processing task is the input data of the downstream consumption task, then an initial one-way usage pointer relationship from the current video processing task to the downstream consumption task is directly generated, explicitly indicating that the downstream consumption task can only start after the current video processing task has completed its output. The fifth step is to organize all generated initial one-way usage pointer relationships, remove duplicate relationships (i.e., the same pointer relationship is generated multiple times between the same preceding task and the same downstream consumption task), and check for cyclic pointer relationships (i.e., A points to B, B points to C, and C points to A). If a cyclic pointer relationship exists, it means that the pointer relationship was generated incorrectly. The downstream consumption task set and the shared input data area need to be re-analyzed, and the initial one-way usage pointer relationship needs to be adjusted until there is no cyclic pointer relationship, forming a complete initial one-way usage pointer relationship between each video processing task.

[0070] Step 104: Based on the initial one-way purpose pointing relationship between each video processing task and the execution timing constraints of each video processing task in the original task sequence, construct a task dependency graph.

[0071] Optionally, the original task sequence refers to the initial task sequence generated based on the user's business intent decomposition order after the video processing tasks are generated, without considering the dependencies between tasks; the execution timing constraint refers to the implicit execution order requirements of each video processing task in the original task sequence. This constraint is set based on the logical order of the user's business intent and is used to ensure that the task execution meets the business requirements.

[0072] Optionally, the vehicle terminal system constructs a task dependency graph based on the initial one-way purpose orientation relationship between each video processing task and the execution timing constraints of each video processing task in the original task sequence, as described in steps 1041 to 1044.

[0073] The task dependency graph constructed in this embodiment of the invention can comprehensively reflect the data source association, output purpose association and timing constraints between various video processing tasks, providing a prerequisite guarantee for the rational allocation of computing resources and avoiding computing waste. It supports the flexibility and efficiency of computing power scheduling of vehicle terminal system, and improves the real-time response determinism and computing resource utilization of vehicle intelligent terminal in complex and ever-changing road environments.

[0074] Optionally, the process of steps 1041 to 1044 includes:

[0075] Step 1041: Based on the initial unidirectional purpose-direction relationship between each video processing task and the execution timing constraints of each video processing task in the original task sequence, perform temporal dependency analysis to obtain the temporal dependency relationship between each video processing task.

[0076] Optionally, temporal dependency analysis refers to the process of analyzing and determining the temporal execution relationship between various video processing tasks. Its purpose is to clarify the constraints on the execution sequence of each video processing task and ensure that the task execution conforms to the logical flow of the user's business intent. Temporal dependency relationship refers to the constraint relationship obtained through temporal dependency analysis, which is used to clarify the sequential execution order between various video processing tasks. This relationship clearly indicates that a certain video processing task must be started only after another video processing task has been completed, or that a certain video processing task can be executed in parallel with another video processing task. Together with the initial unidirectional purpose orientation relationship, it constitutes the core foundation of task dependency relationship.

[0077] Optionally, the specific process of performing time-series dependency analysis in the vehicle-mounted terminal system is as follows: First, number each video processing task according to its order in the original task sequence. The numbering order should be consistent with the execution timing constraints in the original task sequence; that is, the video processing task that appears earlier in the original task sequence has a lower number than the video processing task that appears later. This numbering directly reflects the execution timing constraints of the original task sequence. Second, extract all initial one-way purpose pointing relationships, and identify the two video processing tasks corresponding to each initial one-way purpose pointing relationship, namely the starting task (preceding task) and the ending task (current task) of the pointing relationship, thus clarifying the initial one-way purpose pointing relationship. The third step involves comparing the initial one-way purpose pointer with the starting and ending tasks in the original task sequence to determine if the execution order indicated by the initial one-way purpose pointer matches the execution timing constraints in the original task sequence. If the starting task's number in the original task sequence is less than the ending task's number, it indicates that the execution order indicated by the initial one-way purpose pointer matches the execution timing constraints of the original task sequence. At this point, it is preliminarily determined that there is a timing dependency between the starting and ending tasks, meaning the ending task can only start executing after the starting task has been completed. If the starting task's number in the original task sequence is less than the ending task's number, it indicates that the execution order indicated by the initial one-way purpose pointer matches the execution timing constraints of the original task sequence. In this case, it is preliminarily determined that there is a timing dependency between the starting and ending tasks, meaning the ending task can only start executing after the starting task has been completed. If the number in the column is greater than the number of the endpoint task, it indicates that the execution order indicated by the initial one-way purpose pointer relationship is inconsistent with the execution timing constraints of the original task sequence. At this point, the timing dependency between the two is not determined and will be handled in subsequent steps. The fourth step involves traversing all video processing tasks. For each pair of video processing tasks that do not have an initial one-way purpose pointer relationship, the timing dependency between them is determined by considering the execution timing constraints in the original task sequence. If the number of one video processing task in the original task sequence is less than the number of the other video processing task, and the two tasks have a logical sequential execution requirement (i.e., the execution result of the former task indirectly affects the latter), then the process is considered complete. If the input data or execution logic of a task is considered, then a temporal dependency relationship is determined between the two, meaning that the task with the larger number must be executed before it can start. If the two tasks are adjacent or spaced apart in the original task sequence, but there is no sequential execution requirement in the business logic, then it is determined that there is no temporal dependency relationship between them, and they can be executed in parallel according to the computing resources. The fifth step is to organize all the temporal dependencies obtained, clarify the preceding temporal tasks (tasks that must be executed before it) and the following temporal tasks (tasks that can be started after it is executed) corresponding to each video processing task, forming a complete temporal dependency relationship between each video processing task.

[0078] Step 1042: Based on the temporal dependencies between various video processing tasks, remove the unidirectional purpose pointing relationships that do not satisfy the directional constraints in the initial unidirectional purpose pointing relationships to obtain the target unidirectional purpose pointing relationship.

[0079] Optionally, directional constraints refer to the directional requirements for the sequential execution of various video processing tasks, determined based on temporal dependencies. In other words, the direction from a preceding task to a subsequent task is explicitly defined in the temporal dependency. This constraint is used to regulate the rationality of the direction of the initial unidirectional purpose pointing relationship and ensure that the initial unidirectional purpose pointing relationship is consistent with the temporal dependency. A unidirectional purpose pointing relationship that does not satisfy the directional constraints refers to a pointing relationship in the initial unidirectional purpose pointing relationship where the pointing direction of the starting task (preceding task) and the ending task (current task) is inconsistent with the sequential execution direction of the two in the temporal dependency. Such pointing relationships do not conform to the temporal logic of task execution and need to be eliminated.

[0080] Optionally, the vehicle terminal system performs the elimination operation and the generation process of the target one-way purpose pointing relationship: First, it iterates through all initial one-way purpose pointing relationships, extracts the starting task and ending task of each initial one-way purpose pointing relationship one by one, and clarifies the direction of the pointing relationship (starting task points to ending task); Second, for each initial one-way purpose pointing relationship, it queries the sequential execution direction between the starting task and the ending task in the temporal dependency relationship, that is, it determines whether the starting task is a predecessor temporal task of the ending task, or whether the ending task is a predecessor temporal task of the starting task; Third, if the query result is that the starting task is a predecessor temporal task of the ending task, it means that the direction of the initial one-way purpose pointing relationship is consistent with the direction constraint of the temporal dependency relationship, which conforms to the temporal logic of task execution, and the initial one-way purpose pointing relationship is then... If the query result indicates that the endpoint task is a preceding sequential task of the starting task, it means that the direction of the initial one-way purpose pointing relationship is inconsistent with the direction constraint of the temporal dependency relationship, which does not conform to the temporal logic of task execution. It belongs to a one-way purpose pointing relationship that does not satisfy the direction constraint, and the initial one-way purpose pointing relationship is removed. If the query result indicates that there is no temporal dependency relationship between the starting task and the endpoint task, and the direction of the initial one-way purpose pointing relationship does not violate the execution temporal constraint of the original task sequence, then the initial one-way purpose pointing relationship is retained. If it violates the constraint, it is removed. The fourth step is to organize all the retained initial one-way purpose pointing relationships, remove duplicate pointing relationships (i.e., the same pointing relationship retained multiple times between the same starting task and the same endpoint task), ensure the uniqueness and rationality of each pointing relationship, and finally form the target one-way purpose pointing relationship.

[0081] Step 1043: Determine the joint task pair by taking the output provider in the target one-way use pointing relationship as the predecessor task and the input consumer in the target one-way use pointing relationship as the successor task.

[0082] Optionally, the output provider refers to the video processing task that serves as the starting task in a one-way target-purpose relationship. The output data generated by this task after its execution provides input data support for the ending task in the same relationship, making it the primary provider of input data. The predecessor task refers to the video processing task that needs to be executed before other tasks in a task dependency relationship, and whose output data provides input data support for other tasks; it is the same task as the output provider. The input consumer refers to the video processing task that serves as the ending task in a one-way target-purpose relationship. This task needs to read and use the output data generated by the output provider during its execution, making it the primary consumer of input data. The successor task refers to the video processing task that needs to be started only after the predecessor task (preceding task) is completed in a task dependency relationship, and which depends on the output data of the predecessor task; it is the same task as the input consumer. A joint task pair refers to a task combination consisting of a predecessor task and a successor task. Each joint task pair corresponds to a set of explicit dependencies, meaning the predecessor task provides input data support for the execution of the successor task, and the successor task depends on the execution result of the predecessor task. The joint task pair is the basic unit for constructing a task dependency graph.

[0083] Optionally, the specific process by which the vehicle-mounted terminal system determines the joint task pair is as follows: First, it iterates through all target one-way purpose pointing relationships, analyzes the structure of each relationship, and clarifies the starting and ending tasks in each relationship. Second, for each target one-way purpose pointing relationship, it determines the starting task (output provider) as the predecessor task and the ending task (input consumer) as the successor task, clarifying the dependency between the predecessor and successor tasks. That is, the data output by the predecessor task after execution provides input data for the successor task, and the successor task can only start after the predecessor task has been completed. Third, it sets the determined... The first step involves combining the predecessor and successor tasks to form a set of joint task pairs. Each joint task pair corresponds to a unique one-way target purpose relationship. The joint task pairs clearly indicate the identification information of the predecessor and successor tasks, as well as the type of dependency relationship between them (dependency based on the purpose of the output data). The second step is to organize all generated joint task pairs, remove duplicate joint task pairs (i.e., the case where a task pair consisting of the same predecessor task and the same successor task is generated multiple times), ensure the uniqueness of each joint task pair, and sort out the one-way target purpose relationship corresponding to each joint task pair to clarify the association logic between the joint task pairs, and finally form a complete set of joint task pairs.

[0084] Step 1044: Construct a task dependency graph based on the directed dependency connections of the joint task pairs.

[0085] Optionally, the vehicle terminal system constructs a task dependency graph based on the directed dependency connections of the joint task pairs, as described in steps 10441 to 10444.

[0086] The task dependency graph constructed in this embodiment of the invention can accurately reflect the data source association, output purpose association and time sequence constraints between various video processing tasks, effectively avoiding problems such as chaotic dependencies and incorrect pointers, supporting the orderliness and efficiency of computing power scheduling of vehicle terminal system, thereby improving the real-time response determinism and computing power resource utilization of vehicle intelligent terminal in complex and ever-changing road environments.

[0087] Optionally, the process of steps 10441 to 10444 includes:

[0088] Step 10441: For each joint task pair, based on whether the input data region of each successor task in the joint task pair is completely covered by the output data region of its corresponding predecessor task, determine whether the directed dependency connection constitutes a data integrity dependency and obtain the dependency judgment result.

[0089] Optionally, the output data coverage area refers to the entire data area covered by the output data generated after the predecessor task is executed. This area clearly defines the specific range of the predecessor task's output data and corresponds one-to-one with the purpose of the predecessor task's output data. Data integrity dependency refers to the dependency type between the predecessor task and the successor task corresponding to the directed dependency connection, where the output data of the predecessor task can fully satisfy the input data requirements of the successor task. That is, all input data of the successor task can be obtained from the output data of its corresponding predecessor task. Dependency judgment result refers to the result obtained after the judgment operation, which is used to determine whether the directed dependency connection corresponding to each joint task pair constitutes a data integrity dependency. Each joint task pair corresponds to a dependency judgment result, including whether it constitutes a data integrity dependency or not.

[0090] Optionally, the vehicle-mounted terminal system performs the judgment operation and generates the dependent judgment result as follows: First, it iterates through all joint task pairs, extracts individual joint task pairs one by one, and clarifies the predecessor task and the corresponding successor task in the joint task pair; Second, for the extracted joint task pairs, it retrieves the input data area information of the successor task in the joint task pair, clarifies the specific range of the successor task's input data, including the spatiotemporal boundaries (timestamp interval and spatial range) corresponding to the input data; Third, it retrieves the output data coverage area information of the predecessor task in the joint task pair, clarifies the specific range of the predecessor task's output data, including the spatiotemporal boundaries (timestamp interval and spatial range) corresponding to the output data; Fourth, it performs a comprehensive comparison between the input data area of ​​the successor task and the output data coverage area of ​​the predecessor task, including the timestamp interval and spatial range of both, to determine whether the input data area of ​​the successor task is completely covered by the output data of the predecessor task. The first step involves determining whether the timestamp interval of the input data of the subsequent task is completely within the timestamp interval of the output data coverage area of ​​the predecessor task, and whether the spatial range of the input data of the subsequent task is completely within the spatial range of the output data coverage area of ​​the predecessor task. The fifth step involves determining whether the input data region of the subsequent task is completely covered by the output data coverage area of ​​its corresponding predecessor task, and recording the result if the comparison result shows that the input data region of the subsequent task is not completely covered by the output data coverage area of ​​its corresponding predecessor task (i.e., some input data exceeds the range of the output data coverage area of ​​the predecessor task), and recording the result if the corresponding directed dependency connection of the joint task does not constitute a data integrity dependency. The sixth step involves repeating steps one through five until all joint task pairs have completed the determination process, resulting in a dependency determination result set.

[0091] Step 10442: Based on the dependency judgment result, retain the directed dependency connections corresponding to the joint task pairs that satisfy the output covering the input condition as initial valid dependency edges, and based on the initial valid dependency edges and whether the output data of each video processing task requires immediate supply to subsequent tasks in terms of temporal continuity, obtain the temporal coupling attribute of each initial valid dependency edge.

[0092] Optionally, the output covering input condition refers to the condition under which the joint task constitutes a data integrity dependency on the corresponding directed dependency connection, that is, the input data region of the subsequent task is completely contained by the output data coverage region of its corresponding predecessor task. This condition is the core criterion for screening initial valid dependency edges. Immediate supply to subsequent tasks in terms of temporal continuity means that the output data of the predecessor task has a clear purpose and its generated output data must be transmitted to the corresponding subsequent task immediately after the output is completed without any time interval, ensuring that the subsequent task can obtain the input data in a timely manner and avoid affecting the real-time performance of task execution due to data transmission delay. The time coupling attribute refers to the attribute used to describe the degree of temporal coupling between the predecessor task and the subsequent task corresponding to the initial valid dependency edge, and this attribute directly reflects the urgency of the subsequent task's time requirement for the output data of the predecessor task.

[0093] Optionally, the vehicle-mounted terminal system performs the initial effective dependency edge screening and time-dependent coupling attribute determination process as follows: First, it iterates through the dependency judgment result set, extracting the dependency judgment result corresponding to each joint task pair one by one; Second, for each joint task pair, if its corresponding dependency judgment result constitutes a data integrity dependency, i.e., satisfies the output covering input condition, then the directed dependency connection corresponding to the joint task pair is retained, and this directed dependency connection is determined as an initial effective dependency edge; if its corresponding dependency judgment result does not constitute a data integrity dependency, i.e., does not satisfy the output covering input condition, then the directed dependency connection corresponding to the joint task pair is removed and not included in the initial effective dependency edge set; Third, after completing the screening of all joint task pairs, all retained initial effective dependency edges are organized to form an initial effective dependency edge set, clarifying the predecessor and successor tasks corresponding to each initial effective dependency edge; Fourth, iterates through the initial effective dependency edge set, extracting each individual initial effective dependency edge one by one, clarifying the predecessor task corresponding to each initial effective dependency edge; Fifth, it retrieves the initial effective dependency edge... The sixth step involves analyzing the usage information of the output data of the predecessor task corresponding to the effective dependency edge, and whether the usage of the output data of the predecessor task requires immediate supply to its corresponding successor task in terms of temporal continuity. If the usage of the output data of the predecessor task explicitly requires immediate supply to the corresponding successor task in terms of temporal continuity, then the temporal coupling attribute of the initial effective dependency edge is determined to be high coupling, that is, the temporal coupling degree between the predecessor task and the successor task is high, and the immediate supply of output data must be strictly guaranteed. If the usage of the output data of the predecessor task does not require immediate supply to the corresponding successor task in terms of temporal continuity, that is, it is allowed that the output data is transmitted to the successor task after a certain time interval after output is completed, then the temporal coupling attribute of the initial effective dependency edge is determined to be low coupling, that is, the temporal coupling degree between the predecessor task and the successor task is low. The seventh step involves repeating the operations from the fourth to the sixth step until the temporal coupling attribute of all initial effective dependency edges is determined, and recording the temporal coupling attribute corresponding to each initial effective dependency edge to form a complete set containing the initial effective dependency edges and their corresponding temporal coupling attributes.

[0094] Step 10443: Based on the time-dependent coupling attribute of each initial valid dependency edge, perform attribute type labeling on each initial valid dependency edge to obtain the target valid dependency edge.

[0095] Optionally, attribute type labeling refers to the operation of assigning a corresponding type identifier to each initial valid dependency edge based on its time-coupling attribute. This includes synchronous and asynchronous types, used to distinguish initial valid dependencies with different degrees of time coupling. The synchronous type refers to one of the attribute types corresponding to the initial valid dependency edge, corresponding to an initial valid dependency edge with a high time-coupling attribute. It indicates that the output data between the predecessor and successor tasks corresponding to this initial valid dependency edge needs to be supplied immediately, and the successor task needs to start execution immediately after the predecessor task finishes outputting the data. The two are synchronously connected in time. The asynchronous type refers to one of the attribute types corresponding to the initial valid dependency edge, corresponding to an initial valid dependency edge with a low time-coupling attribute. It indicates that the output data between the predecessor and successor tasks corresponding to this initial valid dependency edge does not need to be supplied immediately, and the successor task can start execution within a reasonable time interval after the predecessor task finishes outputting the data. The two can be asynchronously connected in time.

[0096] Optionally, the vehicle-mounted terminal system performs the following process for attribute type labeling and generation of target valid dependency edges: First, it iterates through the initial valid dependency edge set, extracting each initial valid dependency edge individually, and determining the time-coupling attribute (high coupling or low coupling) corresponding to each initial valid dependency edge. Second, for the extracted initial valid dependency edges, attribute type labeling is performed based on their time-coupling attribute. If the time-coupling attribute of the initial valid dependency edge is high coupling, it is labeled with a synchronous type attribute, thus clarifying that the initial valid dependency edge is a synchronous type dependency edge; if the time-coupling attribute of the initial valid dependency edge is low coupling, it is labeled with an asynchronous type attribute, thus clarifying that the initial valid dependency edge is a synchronous type dependency edge. The first step is to identify the initial valid dependency edge as an asynchronous dependency edge. The second step is to retain the initial valid dependency edge and its corresponding attribute type identifier after completing the attribute type labeling, and use it as a candidate target valid dependency edge. The third step is to repeat the operations from the first to the third step until all initial valid dependency edges have completed attribute type labeling, and obtain all candidate target valid dependency edges. The fourth step is to organize all candidate target valid dependency edges, clarify the attribute type, corresponding predecessor task and successor task of each candidate target valid dependency edge, remove duplicate candidate target valid dependency edges (i.e., dependency edges with the same start point, the same end point and the same attribute type), and obtain the target valid dependency edge set.

[0097] Step 10444: Construct a task dependency graph with each video processing task as a graph node and each effective dependency edge of the target as the connection edge between the graph nodes.

[0098] Optionally, the process of constructing the task dependency graph for the vehicle terminal system is as follows: First, treat each video processing task as an independent graph node, assigning a unique identifier to each node. This identifier must match the identifier of the corresponding video processing task to ensure a one-to-one correspondence between graph nodes and video processing tasks. Second, arrange the graph nodes according to the business logic relationships of each video processing task and the temporal constraints of the original task sequence, ensuring clear connections between nodes and avoiding overlapping connections that could obscure dependencies. Third, traverse the set of valid dependency edges for each target, extracting each valid dependency edge individually, identifying the predecessor and successor tasks corresponding to that edge, and determining the attribute type (synchronous or asynchronous) of the edge. Fourth, based on the target's... For each effective dependency edge, draw a connecting edge between the predecessor and successor tasks corresponding to the two corresponding graph nodes. The direction of the connecting edge is from the graph node corresponding to the predecessor task to the graph node corresponding to the successor task, ensuring that the direction of the connecting edge is consistent with the direction of the target effective dependency edge. In the fifth step, label the corresponding attribute type (synchronous or asynchronous) on the drawn connecting edge to clarify the time coupling type of the dependency relationship corresponding to the connecting edge. In the sixth step, repeat the operations from the third to the fifth step until all target effective dependency edges have been drawn and labeled with attribute types. In the seventh step, integrate all graph nodes and connecting edges, sort out the association logic between graph nodes, and ensure that the dependency relationship between all video processing tasks is clearly presented through graph nodes and connecting edges, and that the attribute type labeling of the connecting edges is accurate. Finally, a complete task dependency graph is constructed.

[0099] The task dependency graph constructed in this embodiment of the invention not only solves the problems of ambiguous dependency relationships and insufficient data support, but also incorporates the distinction of time coupling attributes, improving the rationality and flexibility of computing power scheduling, realizing the rational allocation of computing power in high-load scenarios and the efficient utilization of computing power in low-load scenarios, solving the problems of high-load response delay and low-load computing power waste caused by static allocation, and improving the real-time response determinism and computing power resource utilization of vehicle intelligent terminals in complex and ever-changing road environments.

[0100] Optionally, the processes of steps 301 to 303 include:

[0101] Step 301: Based on the visual analysis operation type corresponding to each video processing task in the task execution sequence and its position number in the sequence, obtain the computational load category sequence for each video processing task.

[0102] Optionally, the computational load category refers to the category obtained by classifying each video processing task according to the computational amount of the visual analysis operation type. It is used to distinguish the computational burden of different video processing tasks. The computational load category can be divided into three categories: high load, medium load, and low load. Among them, visual analysis operation types with large computational amounts (such as image recognition operations and object detection operations) correspond to the high load category, visual analysis operation types with medium computational amounts (such as feature extraction operations and object tracking operations) correspond to the medium load category, and visual analysis operation types with small computational amounts (such as image enhancement operations and image segmentation operations) correspond to the low load category.

[0103] Optionally, the vehicle-mounted terminal system executes the following process for generating the computational load category sequence: First, it traverses the task execution sequence, extracting each individual video processing task and identifying the corresponding visual analysis operation type. Second, for each extracted video processing task, based on its corresponding visual analysis operation type, it determines the computational load category of that task, i.e., whether the visual analysis operation type is computationally intensive (high, medium, or low), and classifies it accordingly into a high-load, medium-load, or low-load category. Third, it extracts the position number of the video processing task within the task execution sequence, clarifying its relative position within the sequence. The fourth step is to associate and record the computational load category of the video processing task with its corresponding position number, ensuring that the computational load category corresponds one-to-one with the video processing tasks in the task execution sequence. The fifth step is to repeat the operations from the first to the fourth step until the computational load category of all video processing tasks in the task execution sequence has been determined and associated with its position number. The sixth step is to arrange the computational load category corresponding to each video processing task in the task execution sequence in ascending order of its position number, forming a complete sequence of computational load categories. This sequence clearly shows the distribution of computational load for each task in the task execution sequence.

[0104] Step 302: Based on the computational load category of each video processing task and its adjacency relationship in the task execution sequence, the task execution sequence is divided into a continuous preset number of computational load segments.

[0105] Optionally, the adjacency relationship refers to the relationship between two video processing tasks with adjacent position numbers in the task execution sequence. That is, the video processing task with position number N and the video processing task with position number N+1 are adjacent tasks. The adjacency relationship is used to determine whether the computational load categories of consecutive tasks are consistent. The preset number is reasonably set based on the total schedulable computing power resources of the smart terminal, the length of the task execution sequence, and the real-time requirements of task execution. The preset number is a fixed value and can be adjusted according to the actual application scenario. The computational load segment refers to the consecutive task segment obtained after dividing the task execution sequence according to the computational load category and the adjacency relationship. Each computational load segment contains one or more video processing tasks, and the computational load categories of all video processing tasks in the segment are completely consistent. The computational load categories between different computational load segments can be the same or different, but the tasks within the segment must meet the requirements of continuity and consistent load categories.

[0106] Optionally, the vehicle-mounted terminal system performs the following process for dividing the computing load segment: First, it iterates through the computing load category sequence and simultaneously executes the corresponding video processing tasks in the associated task execution sequence, clarifying the computing load category and corresponding video processing task for each position in the computing load category sequence. Second, starting from the beginning of the computing load category sequence (i.e., the computing load category corresponding to the video processing task with the smallest position number), it compares the computing load categories of adjacent positions sequentially to determine if the computing load categories corresponding to the two adjacent positions are the same. Third, if the computing load categories of two adjacent positions are the same, the video processing tasks corresponding to these two positions are grouped into the same temporary segment, and the comparison of the computing load categories of the next adjacent position continues until a different computing load category is encountered. Fourth, if the computing load categories of two adjacent positions are different, the division of the current temporary segment ends, and the current temporary segment is determined as a candidate computing load segment. Then, starting from the next position with a different computing load category, the operations of steps two to three are repeated to divide a new temporary segment. Fifth, after completing the division of all temporary segments, the candidate computing loads are statistically analyzed. The sixth step involves determining whether the total number of candidate computational load segments equals the preset number. If the total number of candidate computational load segments equals the preset number, all candidate computational load segments are directly identified as the final computational load segments. If the total number of candidate computational load segments is less than the preset number, some candidate computational load segments containing a large number of video processing tasks are further split according to their adjacency. Each split segment must still meet the requirement of containing one or more consecutive video processing tasks with the same computational load category until the total number of computational load segments equals the preset number. If the total number of candidate computational load segments is greater than the preset number, adjacent candidate computational load segments with the same computational load category are merged. The merged segments must still meet the requirement of containing one or more consecutive video processing tasks with the same computational load category until the total number of computational load segments equals the preset number. The seventh step involves organizing all computational load segments after the division, clarifying the video processing tasks contained in each computational load segment, the corresponding computational load category, and the start and end position numbers of each computational load segment in the task execution sequence, forming a complete set of computational load segments.

[0107] Step 303: Based on the number of video processing tasks contained in each computing load segment and the computational complexity characteristics of each task, determine the cumulative computational complexity value of each computing load segment.

[0108] Optionally, the number of video processing tasks included in the computational load segment refers to the number of video processing tasks covered in each computational load segment. This number can be obtained by counting the number of position numbers in the execution sequence of the corresponding tasks within each computational load segment. The cumulative computational complexity value refers to the value obtained by summing up the computational complexity characteristics of all video processing tasks within each computational load segment. This value reflects the overall computational burden of the entire computational load segment.

[0109] Optionally, the vehicle-mounted terminal system performs the following process to determine the cumulative computational complexity value: First, it traverses the set of computational load segments, extracts each individual computational load segment, identifies all video processing tasks contained within the computational load segment, and determines the computational complexity characteristics corresponding to each video processing task; Second, it counts the number of video processing tasks contained within the computational load segment; Third, for each video processing task within the computational load segment, it extracts the core parameters (number of operations, memory usage, time overhead, etc.) corresponding to its computational complexity characteristics, and quantifies each core parameter, converting different types of core parameters into numerical forms that can be directly summarized. The fourth step is to sum the quantized computational complexity feature values ​​of all video processing tasks within the computational load segment. During the summation process, it is necessary to ensure that the quantized values ​​of all core parameters of each video processing task are included in the summation range, thus obtaining the preliminary cumulative computational complexity value of the computational load segment. The fifth step is to correct the preliminary cumulative computational complexity value based on the number of video processing tasks included in the computational load segment. The correction method is to perform a weighted calculation of the preliminary cumulative computational complexity value and the number of video processing tasks. The weighting coefficient is preset based on the degree of influence of the number of tasks on the overall computational load, ensuring that the corrected cumulative computational complexity value can accurately reflect the overall computational burden of the computational load segment. The sixth step is to determine the corrected value as the final cumulative computational complexity value of the computational load segment. The seventh step is to repeat the operations from the first to the sixth step until the cumulative computational complexity value of all computational load segments has been determined, forming a complete set of cumulative computational complexity values.

[0110] Step 304: Based on the total schedulable computing resources and the cumulative computational complexity of each computing load segment, allocate a corresponding share of computing resources to each task.

[0111] Optionally, the total schedulable computing resources refer to the sum of all computing resources available for executing video processing tasks. These computing resources include, but are not limited to, the computing resources of the central processing unit, the graphics processing unit, and dedicated acceleration chips. The total schedulable computing resources are the remaining computing resources specifically available for executing this video processing task sequence after deducting the computing resources required for the basic operation of the vehicle terminal system itself and other non-video processing services. Based on the total schedulable computing resources and the cumulative computational complexity value of each computational load segment, a corresponding share of computing resources is allocated to each task, as detailed in steps 3041 to 3044.

[0112] The embodiments of the present invention achieve precise matching between computing resources and task computing requirements, enabling tasks with high computing loads to obtain more computing resources support, and tasks with low computing loads to reasonably occupy computing resources. This avoids the waste of computing resources and the problem of response delay for high-load tasks, supports the reasonable allocation of computing resources in high-load scenarios and the efficient utilization of computing resources in low-load scenarios, and improves the real-time response determinism and computing resource utilization of vehicle intelligent terminals in complex and ever-changing road environments.

[0113] Optionally, the processes of steps 3041 to 3044 include:

[0114] Step 3041: Based on the cumulative computational complexity value of each computational load segment, generate the resource demand intensity value of each computational load segment, and construct the resource demand intensity sequence based on the resource demand intensity value of each computational load segment in the order of the computational load segments in the task execution sequence.

[0115] Optionally, the resource demand intensity sequence refers to the sequence formed by arranging the resource demand intensity values ​​corresponding to each computing load segment in the order of the computing load segments in the task execution sequence. This sequence is completely consistent with the arrangement order of the computing load segments and clearly presents the changing pattern of computing power resource demand intensity at different stages in the task execution sequence.

[0116] Optionally, the vehicle-mounted terminal system performs the following process: First, it iterates through all computational load segments, extracts each individual computational load segment, and clarifies the cumulative computational complexity value corresponding to that segment. Second, for each extracted computational load segment, a preset quantization conversion method is used to convert the cumulative computational complexity value of that segment into a resource demand intensity value. This quantization conversion method is based on the preset correspondence between the cumulative computational complexity value and the computing power resource demand, and can uniformly convert cumulative computational complexity values ​​of different ranges into comparable resource demand intensity values. During the conversion process, it is necessary to ensure that the cumulative computational complexity value is consistent with the cumulative computational complexity value. The computational complexity value is positively correlated with the resource demand intensity value; that is, the larger the cumulative computational complexity value, the larger the converted resource demand intensity value. The third step involves recording the resource demand intensity value corresponding to the computational load segment and associating it with the order of the computational load segment in the task execution sequence to ensure that the resource demand intensity value corresponds to the computational load segment. The fourth step involves repeating steps one through three until all computational load segments have completed the generation and sequential association of their resource demand intensity values. The fifth step involves arranging the resource demand intensity values ​​corresponding to each computational load segment in order of their execution sequence to obtain the resource demand intensity sequence.

[0117] Step 3042: Based on the difference in resource demand intensity values ​​between adjacent computing load segments in the resource demand intensity sequence, identify the jump positions where the intensity change amplitude is greater than the continuity threshold, obtain a set of intensity jump points, and cut the task execution sequence at the jump points based on the set of intensity jump points to obtain multiple resource scheduling intervals.

[0118] Optionally, adjacent computing load segments refer to two computing load segments that are sequentially adjacent in the resource demand intensity sequence, and also correspond to two computing load segments that are sequentially connected in the task execution sequence; the difference in resource demand intensity values ​​refers to the difference between the resource demand intensity values ​​corresponding to two adjacent computing load segments, calculated using absolute values, used to characterize the magnitude of change in computing power resource demand intensity between two adjacent computing load segments. The larger the difference, the more drastic the change in computing power resource demand intensity; the continuity threshold refers to a pre-set critical value used to determine whether the change in resource demand intensity between adjacent computing load segments is continuous. This threshold is reasonably set based on the stability requirements of computing power resource scheduling and the real-time requirements of task execution, and is a fixed value. The resource share allocation process remains unchanged. If the difference in resource demand intensity between adjacent computing load segments exceeds the threshold, it indicates a drastic change in resource demand intensity between them, representing a jump. The jump position refers to the location in the resource demand intensity sequence where the difference in resource demand intensity between adjacent computing load segments is greater than the continuity threshold. This position corresponds to the junction of two computing load segments in the task execution sequence. The resource scheduling interval refers to each independent segment obtained after segmentation. Each resource scheduling interval contains one or more consecutive computing load segments, and the difference in resource demand intensity between any adjacent computing load segments within the interval does not exceed the continuity threshold. That is, the computing power resource demand intensity within the interval is relatively stable, facilitating unified computing power resource scheduling and allocation.

[0119] Optionally, the vehicle-mounted terminal system performs the following process: First, it iterates through the resource demand intensity sequence, extracting the resource demand intensity values ​​corresponding to two adjacent computing load segments one by one. Second, it calculates the difference between the resource demand intensity values ​​of these two adjacent computing load segments, using absolute values ​​to obtain the intensity change amplitude between them. Third, it compares the calculated intensity change amplitude with a continuity threshold to determine whether the intensity change amplitude is greater than the continuity threshold. Fourth, if the intensity change amplitude is greater than the continuity threshold, the connection point between these two adjacent computing load segments is determined as a jump position, and this jump position is added to the intensity jump point set. If the intensity change amplitude is not greater than the continuity threshold, the resource demand intensity change between these two adjacent computing load segments is determined to be continuous, and does not constitute a jump position. The process is as follows: 1. Do not add to the intensity jump point set; 2. Repeat steps 1 to 4 until all adjacent computational load segments in the resource demand intensity sequence have been compared and jump positions identified, ultimately forming a complete intensity jump point set; 3. Use each jump position in the intensity jump point set as a cutting point to cut the task execution sequence, ensuring that each cutting point corresponds to the junction of two computational load segments in the task execution sequence without destroying the integrity of a single computational load segment; 4. After cutting, multiple independent segments are obtained, each segment being a resource scheduling interval. Each resource scheduling interval contains one or more continuous computational load segments, and each resource scheduling interval is verified to ensure that the difference in resource demand intensity values ​​between any adjacent computational load segments within the interval does not exceed the continuity threshold, resulting in multiple resource scheduling intervals.

[0120] Step 3043: Based on the task subset covered by each resource scheduling interval and the computational complexity characteristics of each task in the task execution sequence, determine the total cumulative computational complexity of each resource scheduling interval.

[0121] Optionally, the task subset covered by the resource scheduling interval refers to the set of all video processing tasks contained in each resource scheduling interval. This task subset is composed of the video processing tasks contained in all computing load segments within the resource scheduling interval and is a subset of the task execution sequence.

[0122] Optionally, the vehicle-mounted terminal system performs the following process to determine the total cumulative computational complexity of each resource scheduling interval: First, it iterates through the set of resource scheduling intervals, extracting each individual interval and identifying all computational load segments within that interval. Second, it sorts out the video processing tasks within each computational load segment, integrating all video processing tasks from all computational load segments to form a task subset covered by the resource scheduling interval, ensuring that the task subset is complete, non-repeating, and consists entirely of video processing tasks within the task execution sequence. Third, it extracts the computational complexity features corresponding to each video processing task in the task subset, clarifying the computational burden of each task. Fourth, it quantifies the computational complexity features of each video processing task in the task subset, converting different types of computational complexity feature parameters (number of operations, memory usage, time overhead, etc.) into directly summable numerical forms. The quantification standard is consistent with the aforementioned steps, ensuring data uniformity and comparability. Fifth, it... The computational complexity features of all video processing tasks in the task subset are accumulated after quantization. During the accumulation process, it is necessary to ensure that the quantized values ​​of all core parameters of each task are included in the accumulation range to obtain the initial cumulative computational complexity of the resource scheduling interval. The sixth step is to correct the initial cumulative computational complexity by combining the number of video processing tasks and the number of computational load segments contained in the resource scheduling interval and making a weighted adjustment. The weighting coefficient is preset based on the degree of influence of the number of tasks and the number of load segments on the overall computational load, ensuring that the corrected cumulative computational complexity can accurately reflect the overall computational burden and total computing power resource requirements of the resource scheduling interval. The seventh step is to determine the corrected value as the final cumulative computational complexity of the resource scheduling interval and record the value and the corresponding resource scheduling interval. The eighth step is to repeat the operations from the first step to the seventh step until the cumulative computational complexity of all resource scheduling intervals has been determined, resulting in a set of cumulative computational complexity.

[0123] Step 3044: Based on the total schedulable computing power resources and the total cumulative computational complexity of each resource scheduling interval, allocate a corresponding share of computing power resources to each task.

[0124] Optionally, the vehicle-mounted terminal system allocates a corresponding share of computing resources to each task based on the total amount of schedulable computing resources and the total cumulative computational complexity of each resource scheduling interval, as described in steps 30441 to 30445.

[0125] The embodiments of the present invention achieve precise matching of computing resources between different scheduling intervals and different tasks, enabling computing resources to be tilted towards tasks and intervals with high computing load and urgent needs, while avoiding the waste caused by low-load tasks occupying too much computing resources. This supports the rational allocation of computing power in high-load scenarios and the efficient utilization of computing power in low-load scenarios for vehicle terminal systems, thereby improving the real-time response determinism and computing resource utilization of vehicle intelligent terminals in complex and ever-changing road environments.

[0126] Optionally, the processes of steps 30441 to 30445 include:

[0127] Step 30441: Based on the total cumulative computational complexity of each resource scheduling interval, determine the basic share of each resource scheduling interval in the overall computing power resource allocation.

[0128] Optionally, the vehicle-mounted terminal system traverses all resource scheduling intervals, extracts the total cumulative computational complexity corresponding to each resource scheduling interval, and records the identification information and the corresponding total cumulative computational complexity of each resource scheduling interval. The second step involves summing the total cumulative computational complexity of all resource scheduling intervals to obtain the sum of the total cumulative computational complexity of all resource scheduling intervals, which corresponds to the total computational complexity requirement of the task execution sequence. The third step involves using a proportional calculation method for each resource scheduling interval, dividing the total cumulative computational complexity of that resource scheduling interval by the sum of the total cumulative computational complexity of all resource scheduling intervals to obtain the basic share ratio of that resource scheduling interval. The fourth step requires maintaining reasonable computational precision during the calculation process to ensure the accuracy of the basic share ratio. Simultaneously, the basic share ratios of all resource scheduling intervals are aggregated and verified to ensure that the sum of the basic share ratios of all resource scheduling intervals equals 100%, avoiding imbalances in computing power allocation caused by deviations in the total ratio. The fifth step involves recording the basic share ratio corresponding to each resource scheduling interval and associating the basic share ratio with the identification information of the resource scheduling interval to form a complete set of basic share ratios.

[0129] Step 30442: Based on the basic share ratio of each resource scheduling interval and the total amount of schedulable computing power resources, calculate the base amount of computing power resources that can be obtained in each resource scheduling interval, and evenly distribute the base amount of computing power resources in each resource scheduling interval to each video processing task covered by each resource scheduling interval to obtain the initial share of computing power resources for each video processing task.

[0130] Optionally, the vehicle-mounted terminal system traverses the set of basic share proportions, extracting the basic share proportion and identification information of each individual resource scheduling interval; secondly, for the extracted resource scheduling interval, the basic share proportion of the interval is multiplied by the total schedulable computing power resources to obtain the base amount of computing power resources that can be obtained for the resource scheduling interval; thirdly, the base amount of computing power resources corresponding to the resource scheduling interval is recorded, and associated with the identification information of the interval and the number of video processing tasks covered; fourthly, for the resource scheduling interval, a uniform allocation method is used to divide its base amount of computing power resources by the number of video processing tasks covered by the interval to obtain the initial share of computing power resources that each video processing task in the interval can obtain; fifthly... Step 1: If dividing the base computing power by the number of video processing tasks does not yield an integer result, reasonable precision must be maintained to ensure the operability of the initial computing power resource share. Simultaneously, record the initial computing power resource share for each video processing task and associate it with the task's identifier and the resource scheduling interval to which it belongs. Step 2: Repeat steps 1 to 5 until all resource scheduling intervals have completed the calculation of the base computing power, and all video processing tasks have obtained their corresponding initial computing power resource shares. Step 3: Summarize and verify all initial computing power resource shares to ensure that the sum of the initial computing power resource shares for all video processing tasks equals the total schedulable computing power resources. After successful verification, the initial computing power resource share set is obtained.

[0131] Step 30443: Based on the initial computing power resource share of each video processing task and the execution interval constraints between adjacent tasks in the task execution sequence, determine whether there is a risk of delay in the start of subsequent tasks due to insufficient computing power of the preceding task, and obtain the delay risk judgment result.

[0132] Optionally, the execution interval constraint refers to the pre-set maximum allowable execution interval between two adjacent tasks. This constraint is reasonably set based on the real-time requirements of task execution and the response requirements of the in-vehicle driving scenario. If the time interval between the completion of the preceding task and the start of the subsequent task exceeds this constraint, the start of the subsequent task is determined to be delayed. Insufficient computing power of the preceding task refers to the fact that the initial computing power resource share of the preceding task cannot meet the computing power requirements for the task to be completed within the specified time, resulting in the extended execution time of the preceding task, which in turn affects the start time of the subsequent task. Delay risk refers to the possibility that the start interval of the subsequent task exceeds the execution interval constraint due to insufficient computing power and extended execution time of the preceding task. Delay risk judgment result refers to the result obtained after the judgment operation to clarify whether each video processing task has its own start delay due to insufficient computing power of the preceding task, and to clarify the task combination with delay risk and the risk level. The result includes two situations: there is delay risk and there is no delay risk.

[0133] Optionally, the vehicle-mounted terminal system traverses the task execution sequence, extracting adjacent preceding and subsequent tasks one by one, clarifying their task identification information and execution order; the second step involves retrieving the initial computing power resource share and computational complexity characteristics of the preceding task, determining whether the initial computing power resource share of the preceding task is sufficient to meet the task's execution requirements within the specified time. The determination method is as follows: based on the computational complexity characteristics of the preceding task, determine the minimum computing power resource share required for the task's execution. If the initial computing power resource share of the preceding task is greater than or equal to the minimum computing power resource share, the preceding task is deemed to have sufficient computing power and can complete execution within the specified time; if the initial computing power resource share of the preceding task is less than the minimum computing power resource share, the preceding task is deemed to have insufficient computing power, and the execution time will exceed the specified time; the third step involves determining that if the preceding task has sufficient computing power, the subsequent task does not have the risk of startup delay due to insufficient computing power of the preceding task; if the preceding task has insufficient computing power, then... The first step involves calculating the execution time delay caused by insufficient computing power in the preceding task. This time delay is then combined with the normal execution interval between adjacent tasks to obtain the actual interval between the completion of the preceding task and the start of the subsequent task. The second step compares the actual interval with the execution interval constraint. If the actual interval exceeds the constraint, the subsequent task is considered to have a startup delay risk. The task combination (preceding and subsequent tasks) and risk level (determined by the difference between the actual interval and the execution interval constraint) are recorded. If the actual interval does not exceed the constraint, the subsequent task is considered to have no startup delay risk, and this result is recorded. The third step repeats steps one through four until all adjacent tasks in the task execution sequence have undergone delay risk assessment. The fourth step analyzes all assessment results to identify the subsequent tasks with delay risks, their corresponding preceding tasks, and the task combinations without delay risks, thus obtaining a set of delay risk assessment results.

[0134] Step 30444: Adjust the initial computing power resource share of each video processing task based on the latency risk assessment results to obtain the target computing power resource share of each video processing task.

[0135] Optionally, the vehicle-mounted terminal system iterates through the latency risk assessment result set, filters out task combinations (preceding and subsequent tasks) with latency risks, and identifies the preceding tasks with insufficient computing power and their corresponding latency risk levels. The second step involves retrieving the initial computing power resource share, computational complexity characteristics, and minimum computing power resource share required for the completion of the preceding task for each task with insufficient computing power. The system then calculates the additional computing power resource share needed for the preceding task, which equals the minimum computing power resource share required for the task's completion minus the initial computing power resource share. The third step involves extracting a corresponding amount of computing power resources from the remaining share of the total schedulable computing power resources and adding it to the initial computing power resource share of the preceding task. This updates the computing power resource share of the preceding task, ensuring that the updated computing power resource share meets the minimum computing power resource requirements for its completion, thus eliminating the latency delay in the startup of subsequent tasks caused by insufficient computing power in the preceding task. Fourth, if there are still remaining schedulable computing resources after supplementing the computing resource share of preceding tasks, the initial computing resource share of subsequent tasks can be appropriately increased according to the computational complexity characteristics of subsequent tasks to further optimize the execution efficiency of subsequent tasks; if there are no remaining schedulable computing resources after supplementing the computing resource share of preceding tasks, no further adjustments will be made to other tasks; Fifth, for task combinations that do not have delay risk in the delay risk assessment results, their corresponding initial computing resource shares do not need to be adjusted and are directly retained as candidate target computing resource shares; Sixth, all adjusted computing resource shares and unadjusted initial computing resource shares are summarized and verified to ensure that the sum of computing resource shares of all tasks equals the total schedulable computing resources and there are no new delay risks; Seventh, after verification, the final computing resource share determined for each task is determined as the target computing resource share, resulting in a target computing resource share set.

[0136] Step 30445: Based on the target computing power resource share of each video processing task and the total physical resource pool of the processor cores and acceleration units in the smart terminal, the computing power resource share of each task is mapped to an integer to obtain the computing power resource share of each video processing task.

[0137] Optionally, the vehicle-mounted terminal system iterates through the target computing power resource share set, extracting the target computing power resource share and task identification information for each video processing task. The second step involves retrieving the hardware computing power allocation granularity of the vehicle-mounted intelligent terminal for each extracted video processing task, determining whether the target computing power resource share for that task is an integer and whether it matches the hardware computing power allocation granularity. The third step involves using the target computing power resource share as the final computing power resource share for the task directly if it is an integer and divisible by the hardware computing power allocation granularity, without requiring additional mapping. The fourth step involves converting the target computing power resource share into an integer that matches the hardware computing power allocation granularity using rounding. The converted integer is the final computing power resource share for the task. During the conversion process, it must be ensured that the deviation between the converted computing power resource share and the target computing power resource share is within a reasonable range and does not affect the normal execution of the task. The fifth step involves determining the total computing power after integer mapping for multiple tasks. If the resource share exceeds the total physical resource pool (i.e., exceeds the total schedulable computing power resources), the integer computing power resource shares of some tasks are fine-tuned, prioritizing tasks with lower computational complexity and less sensitivity to computing power resource requirements, appropriately reducing their computing power resource shares until the sum of the computing power resource shares of all tasks equals the total physical resource pool (total schedulable computing power resources). If the total computing power resource share does not reach the total physical resource pool, the remaining computing power resource shares are allocated to tasks with higher computational complexity and more sensitive to computing power resource requirements according to the proportion of task computational complexity characteristics, ensuring full utilization of computing power resources. In the sixth step, the operations of steps one to five are repeated until all video processing tasks have completed integer mapping, obtaining the final computing power resource share for each task. In the seventh step, the final computing power resource share of each video processing task is initially associated with the corresponding processor core and acceleration unit to ensure that the computing power resource share can accurately match the hardware physical resources. At the same time, the final computing power resource share of each task and the associated hardware unit identifier are recorded to form a complete computing power resource share allocation result.

[0138] This invention achieves a precise match between computing resources and task computation requirements, as well as hardware physical resources. This ensures that tasks with high computational complexity receive sufficient computing power and avoid response delays, while also preventing waste caused by low computational complexity tasks consuming excessive computing resources. It improves the accurate alignment between video processing tasks and user business intentions, and avoids the ineffective occupation of computing resources by irrelevant tasks from the perspective of computing power allocation. This supports the rational allocation of computing power in high-load scenarios and the efficient utilization of computing power in low-load scenarios for vehicle terminal systems, thereby improving the real-time response determinism and computing resource utilization of vehicle intelligent terminals in complex and ever-changing road environments.

[0139] Figure 2 This is a schematic diagram of the intelligent terminal computing power allocation and processing control system based on vehicle video stream provided by the present invention. The intelligent terminal computing power allocation and processing control system based on vehicle video stream includes:

[0140] The video task analysis module 210 is used to perform task analysis based on the vehicle video stream and user business intent, generate video processing tasks, and construct a task dependency graph based on the interdependencies of each video processing task in terms of input data area, output data purpose, and execution timing; each video processing task corresponds to a visual analysis operation.

[0141] The task topology sorting module 220 is used to perform topological sorting on the task dependency graph and generate a task execution sequence that satisfies the dependency constraints.

[0142] The computing power allocation module 230 is used to allocate a corresponding share of computing power resources to each task based on the computational complexity characteristics of each task in the task execution sequence and the total schedulable computing power resources of the smart terminal.

[0143] The task scheduling module 240 is used to generate binding instructions for processor cores and acceleration units based on the share of computing resources, obtain a task scheduling instruction set, and release the occupied computing resources after the execution status monitoring based on the task scheduling instruction set detects that each task in the task execution sequence has completed its calculation process.

[0144] The embodiments of the present invention realize the rational allocation of computing resources in high-load scenarios and the efficient utilization in low-load scenarios, thereby improving the flexibility and efficiency of computing power scheduling. It solves the problems of response delay in high-load scenarios and computing power waste in low-load scenarios caused by static allocation, and improves the real-time response determinism and computing power resource utilization of vehicle-mounted intelligent terminals in complex and ever-changing road environments.

[0145] Please see Figure 3 , Figure 3 An embodiment diagram of an electronic device provided in accordance with the present invention. For example... Figure 3 As shown, an embodiment of the present invention provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it implements the processes of steps 10 to 40.

[0146] Please see Figure 4 , Figure 4 An embodiment diagram of a computer-readable storage medium provided in accordance with an embodiment of the present invention is shown. Figure 4As shown, this embodiment provides a computer-readable storage medium 400 on which a computer program 311 is stored. When the computer program 311 is executed by a processor, it implements the processes of steps 10 to 40.

[0147] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the intelligent terminal computing power allocation and processing control method based on vehicle video stream provided by the above methods, which includes steps 10 to 40.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent terminal computing power allocation and processing control based on vehicle-mounted video stream, characterized in that, include: Based on the in-vehicle video stream and user business intent, task analysis is performed to generate video processing tasks. Based on the interdependencies of each video processing task in terms of input data area, output data purpose, and execution timing, a task dependency graph is constructed. Each video processing task corresponds to a visual analysis operation. Perform topological sorting on the task dependency graph to generate a task execution sequence that satisfies the dependency constraints; Based on the computational complexity characteristics of each task in the task execution sequence and the total schedulable computing power resources of the smart terminal, a corresponding share of computing power resources is allocated to each task. Based on the computing power resource share, binding instructions for processor cores and acceleration units are generated to obtain a task scheduling instruction set. After the execution status of the task execution sequence is monitored based on the task scheduling instruction set and each task has completed its calculation process, the occupied computing power resources are released.

2. The intelligent terminal computing power allocation and processing control method based on vehicle-mounted video stream according to claim 1, characterized in that, The steps for constructing the task dependency graph include: Based on the spatial range and timestamp interval of the input data region specified for each video processing task in the video frame coordinate system, the spatiotemporal boundaries of the data source for each video processing task are determined. Based on the spatiotemporal boundaries of the data source for each video processing task, identify whether there is a shared input data region between different video processing tasks, and obtain the shared region identification result; Based on the shared input data region identified in the shared region identification results, and combined with the downstream consumption task set specified by the output data purpose of each video processing task, an initial one-way purpose pointing relationship between each video processing task is obtained; the initial one-way purpose pointing relationship is used to indicate that any video processing task can only be started after its preceding task has completed output. Based on the initial one-way purpose-direction relationship between each video processing task and the execution timing constraints of each video processing task in the original task sequence, the task dependency graph is constructed.

3. The intelligent terminal computing power allocation and processing control method based on vehicle-mounted video stream according to claim 2, characterized in that, The task dependency graph is constructed based on the initial unidirectional purpose-direction relationship between each video processing task and the execution timing constraints of each video processing task in the original task sequence, including: Based on the initial one-way purpose orientation relationship between each video processing task and the execution timing constraints of each video processing task in the original task sequence, a temporal dependency analysis is performed to obtain the temporal dependency relationship between each video processing task. Based on the temporal dependencies between various video processing tasks, the unidirectional purpose pointing relationships that do not satisfy the directional constraints in the initial unidirectional purpose pointing relationships are eliminated to obtain the target unidirectional purpose pointing relationship. Using the output provider in the target one-way purpose orientation relationship as the predecessor task and the input consumer in the target one-way purpose orientation relationship as the successor task, a joint task pair is determined; The task dependency graph is constructed based on the directed dependency connections of the joint task pairs.

4. The intelligent terminal computing power allocation and processing control method based on vehicle video stream according to claim 3, characterized in that, The construction of the task dependency graph based on the directed dependency connections corresponding to the joint task pairs includes: For each joint task pair, based on whether the input data region of each successor task in the joint task pair is completely covered by the output data region of its corresponding predecessor task, it is determined whether the directed dependency connection constitutes a data integrity dependency, and the dependency judgment result is obtained. Based on the dependency judgment result, the directed dependency connections corresponding to the joint task pairs that satisfy the output covering input conditions are retained as initial valid dependency edges. Based on the initial valid dependency edges and whether the output data of each video processing task requires immediate supply to subsequent tasks in terms of temporal continuity, the time-dependent coupling attribute of each initial valid dependency edge is obtained. The target valid dependency edge is obtained by annotating the attribute type of each initial valid dependency edge with the time-dependent coupling attribute; the attribute type includes synchronous type and asynchronous type. The task dependency graph is constructed by using each video processing task as a graph node and each effective dependency edge of the target as the connection edge between the graph nodes.

5. The intelligent terminal computing power allocation and processing control method based on vehicle-mounted video stream according to claim 1, characterized in that, Based on the computational complexity characteristics of each task in the task execution sequence and the total schedulable computing resources of the smart terminal, a corresponding share of computing resources is allocated to each task, including: Based on the visual analysis operation type corresponding to each video processing task in the task execution sequence and its position number in the sequence, a computational load category sequence for each video processing task is obtained. Based on the computational load category of each video processing task and its adjacency relationship in the task execution sequence, the task execution sequence is divided into a predetermined number of consecutive computational load segments; each computational load segment includes one or more consecutive video processing tasks with the same computational load category. Based on the number of video processing tasks contained in each computing load segment and the computational complexity characteristics of each task, the cumulative computational complexity value of each computing load segment is determined. Based on the total schedulable computing resources and the cumulative computational complexity of each computing load segment, a corresponding share of computing resources is allocated to each task.

6. The intelligent terminal computing power allocation and processing control method based on vehicle-mounted video stream according to claim 5, characterized in that, The allocation of corresponding computing resource shares to each task based on the total schedulable computing resources and the cumulative computational complexity value of each computing load segment includes: Based on the cumulative computational complexity value of each computational load segment, a resource demand intensity value for each computational load segment is generated, and a resource demand intensity sequence is constructed based on the resource demand intensity value of each computational load segment in the order of the computational load segments in the task execution sequence. Based on the difference in resource demand intensity values ​​between adjacent computing load segments in the resource demand intensity sequence, abrupt transition positions where the intensity change magnitude exceeds the continuity threshold are identified, resulting in a set of intensity transition points. Based on the set of intensity transition points, the task execution sequence is divided at the transition points to obtain multiple resource scheduling intervals. Each resource scheduling interval includes one or more consecutive computing load segments, and the difference in resource demand intensity values ​​between any adjacent computing load segments within the interval does not exceed the continuity threshold. Based on the task subset covered by each resource scheduling interval and the computational complexity characteristics of each task in the task execution sequence, the total cumulative computational complexity of each resource scheduling interval is determined. Based on the total schedulable computing resources and the total cumulative computational complexity of each resource scheduling interval, a corresponding share of computing resources is allocated to each task.

7. The intelligent terminal computing power allocation and processing control method based on vehicle-mounted video stream according to claim 6, characterized in that, The allocation of corresponding computing resource shares to each task based on the total schedulable computing resources and the total cumulative computational complexity of each resource scheduling interval includes: Based on the total cumulative computational complexity of each resource scheduling interval, the basic share of each resource scheduling interval in the overall computing power resource allocation is determined proportionally. Based on the basic share ratio of each resource scheduling interval and the total amount of schedulable computing power resources, the base amount of computing power resources that can be obtained in each resource scheduling interval is calculated, and the base amount of computing power resources in each resource scheduling interval is evenly distributed to each video processing task covered by each resource scheduling interval to obtain the initial share of computing power resources for each video processing task. Based on the initial computing power resource share of each video processing task and the execution interval constraints between adjacent tasks in the task execution sequence, it is determined whether there is a risk of delay in the start of subsequent tasks due to insufficient computing power of the preceding task, and the delay risk judgment result is obtained. Based on the delay risk assessment results, the initial computing power resource share of each video processing task is adjusted to obtain the target computing power resource share of each video processing task. Based on the target computing resource share for each video processing task and the total physical resource pool of the processor cores and acceleration units in the smart terminal, the computing resource share for each task is mapped to an integer to obtain the computing resource share for each video processing task.

8. A smart terminal computing power allocation and processing control system based on vehicle-mounted video stream, characterized in that, The method for allocating and processing computing power to intelligent terminals based on in-vehicle video streams, as described in any one of claims 1 to 7, is applied; the intelligent terminal computing power allocation and processing control system based on in-vehicle video streams includes: The video task analysis module is used to perform task analysis based on the vehicle video stream and user business intent, generate video processing tasks, and construct a task dependency graph based on the interdependencies of each video processing task in terms of input data area, output data purpose, and execution timing; each video processing task corresponds to a visual analysis operation. The task topology sorting module is used to perform topological sorting on the task dependency graph and generate a task execution sequence that satisfies the dependency constraints. The computing power allocation module is used to allocate a corresponding share of computing power resources to each task based on the computational complexity characteristics of each task in the task execution sequence and the total schedulable computing power resources of the smart terminal. The task scheduling module is used to generate binding instructions for processor cores and acceleration units based on the computing power resource share, obtain a task scheduling instruction set, and release the occupied computing power resources after the execution status monitoring based on the task scheduling instruction set detects that each task in the task execution sequence has completed its calculation process.

9. An electronic device, comprising: Memory, used to store computer software programs; A processor for reading and executing the computer software program, characterized in that, when the processor executes the computer software program, it implements the intelligent terminal computing power allocation and processing control method based on vehicle video stream as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, wherein a computer software program is stored therein, characterized in that, When the computer software program is executed by the processor, it implements the intelligent terminal computing power allocation and processing control method based on vehicle video stream as described in any one of claims 1 to 7.

Images