Continuous video stream super-gap scene-oriented sensing calculation collaborative visual inspection task multi-target scheduling system and method

Abstract

The invention discloses a multi-target scheduling system and a multi-target scheduling method for a continuous video stream super-gap scene, which are used for constructing a unified modeling framework integrating edge equipment, an acquisition device, a visual detection model and task parameters on the basis of a sensing-calculation cooperative computing architecture aiming at a typical application scene with limited inter-frame processing time. Core constraints are clearly defined, an improved fast non-dominated sorting multi-target cuckoo genetic algorithm is designed, and task efficient disassembly and edge resource collaborative scheduling are achieved with minimization of task latest completion time and minimization of the number of edge devices used as double optimization targets. According to the method, the task throughput rate and the equipment resource utilization rate of the edge end are improved, and the deployment number of the required edge equipment is greatly reduced while the high task completion rate and the system stability are guaranteed.

Description

A Multi-Objective Scheduling System and Method for Sensor-Computation Collaborative Visual Detection Tasks in Continuous Video Streams with Ultra-Gap Scenarios Technical Field

[0001] This invention belongs to the field of edge computing task scheduling, specifically relating to a multi-objective scheduling system and method for sensory computing collaborative visual detection tasks in continuous video stream ultra-gap scenarios. Background Technology

[0002] Computer vision, a key branch of artificial intelligence, aims to endow computers with the ability to analyze and understand visual information. This technology fully utilizes algorithms and computing resources to deeply analyze and process static images or dynamic video streams from cameras or other image acquisition devices, aiming to achieve or even surpass human visual capabilities. Currently, a common practice in the practical deployment of computer vision is to capture images or video streams using camera devices and then transmit the data to cloud platforms or data centers for in-depth analysis and processing. While this method can leverage abundant cloud computing resources to handle complex image detection and analysis tasks, the physical separation between image acquisition and processing inevitably introduces transmission delays. This delay affects the system's real-time performance, leading to decision-making delays in applications with extremely high real-time requirements, such as security monitoring and autonomous driving.

[0003] In recent years, many researchers have leveraged edge computing technology to enable real-time computer detection. This approach involves deploying devices in the actual detection environment, with personnel carrying acquisition equipment and edge devices. The acquisition equipment captures video, which is then directly transmitted to the edge devices for processing, initially solving the problem of separation between the acquisition and detection ends. However, due to limitations such as insufficient edge device resources, the variety of detection models, and the large volume of data to be detected, the processing speed of current edge detection devices lags behind the capture speed of the acquisition equipment, making it impossible to achieve real-time detection with sensor-computer collaboration and simultaneous image capture and inspection. There is an urgent need to model the real-time detection scheduling problem in computer vision, design appropriate task scheduling strategies, rationally allocate detection tasks, fully utilize the computing power of edge devices, reduce overall processing time, and meet the requirements of real-time detection. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-objective scheduling system and method for sensory-computer collaborative visual detection tasks in continuous video stream ultra-gap scenarios. By constructing a real-time computer vision detection task scheduling model, the problem of edge devices and acquisition devices being unable to work together is solved. An improved fast non-dominated sorting multi-objective cuckoo genetic algorithm (NSCKGA-II) is used to solve the real-time task scheduling optimization problem.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution: Firstly, the present invention provides a multi-objective scheduling system for sensor-computer collaborative visual detection tasks in continuous video stream ultra-gap scenarios, comprising:

[0006] The real-time computer vision detection module includes an edge device model, an acquisition device model, a computer vision detection model, and a task model, which are used to describe the scheduling of computer vision detection tasks under real-time detection requirements. Among them, the edge device model describes the relevant information and parameters of the edge device, the acquisition device model describes the relevant information and parameters of the sensing device, the computer vision detection model describes the model used for target detection, and the task model describes the relevant parameters of the edge detection task and constrains the execution time of the task.

[0007] The constraint construction module is used to establish a set of constraints for the task scheduling model that meet the scheduling requirements of real-time detection tasks in computer vision.

[0008] Furthermore, the edge device model is represented as follows:

[0009] ;

[0010] in, This represents a set of edge devices, with the number of edge devices being... ; Indicates the number is Edge devices;

[0011] A single edge device Represented as:

[0012] ;

[0013] in, Indicates edge device Available computing resources; Indicates edge device Network transmission bandwidth; Indicates edge device The execution frequency, i.e., the number of cycles executed per second;

[0014] The data acquisition device model is represented as follows:

[0015] ;

[0016] in, This represents the set of data acquisition devices, with the number of devices being [number missing]. ; Indicates the number is Image acquisition equipment;

[0017] Each data acquisition device is represented as follows:

[0018]

[0019] in, for The frame rate of the data stream, i.e., the number of images captured per second; express The moment when data stream acquisition begins; express The data volume of a single captured image frame. (By...) We can obtain, for The time interval between the acquired data frames, i.e., the time to acquire a single frame image;

[0020] The computer vision model is represented as follows:

[0021] ;

[0022] in, Represents a set of computer vision detection models; Computer vision detection models that may be used in the overall business process ;

[0023] in,

[0024] Indicate execution Number of calculation cycles required; express The computational resources required to detect a single frame of an image;

[0025] The task model is represented as follows:

[0026] ;

[0027] in, This represents the set of real-time tasks to be detected, with a number of tasks. ; Indicates the first The task to be detected in frame images.

[0028] in, ;

[0029] For the task The computer vision detection model used. Indicates task The data acquisition equipment that needs to be processed Indicates task The amount of data, Indicates task The number of calculation cycles, Indicates task Required computing resources. Among them:

[0030]

[0031]

[0032]

[0033] Furthermore, the super-gap task model represents a task whose processing time is longer than the data frame interval time, i.e. If at least one visual detection model holds true, then a hyper-gap task scenario is constituted.

[0034] Further, definition The smallest subtask to be scheduled represents the task on the edge device. Using computer vision detection models For camera device The first collection Frame images are processed; Indicates task At edge resource nodes The time required for the above to complete; Indicates task Transmitted to edge resource nodes Time required; Indicates that it is composed of edge resource nodes Complete the task The total time required; of which,

[0035]

[0036]

[0037]

[0038] Since edge device resource nodes can work in parallel, with each edge device independently completing its own task, the time required for the system to process all subtasks is... for:

[0039] .

[0040] Furthermore, After scheduling, we get , Indicates in Is the time on the edge device? Using computer vision detection models For camera device The first collection The frame image is processed.

[0041] Furthermore, a set of constraints is defined for the real-time detection requirements of high-speed rail catenary defect task scheduling, including task constraints, computational resource constraints, and real-time detection time constraints.

[0042] The task constraints are as follows:

[0043] For the model involved in the overhead contact line defect detection task, the detection task of a single model for a single frame image cannot be assigned to multiple edge devices during the frame intervals.

[0044] ;

[0045] The above definition means that for the split detection subtask, the task of a single detection model for a single frame image only needs to be assigned to one edge device;

[0046] The computing resource constraints are as follows:

[0047] At any given moment, the resources required for the task executed on each edge device cannot exceed the maximum resource limit of the edge device.

[0048] ;

[0049] The above definition means that, at any given moment, the resources required for the task executed on each edge device must be less than or equal to the maximum resource amount of the edge device.

[0050] The real-time detection time constraint is as follows:

[0051] In real-time detection, the earliest start time of the task for each frame must be later than the arrival time of the image, and the latest completion time must be earlier than the arrival time of the next frame, in order to meet the real-time requirements. Since gap-time tasks cannot meet strong real-time constraints, the maximum completion time in the time constraint does not need to be less than the arrival time of the next frame.

[0052] ;

[0053] in, Indicates the detection task The start time, That is, the earliest start time of all detection tasks for a single frame image; Indicates the detection task End time, The latest completion time of all detection tasks for a single frame image;

[0054] Among them, image acquisition equipment The first collection The arrival time of the frame image is:

[0055] ;

[0056] Regarding the first The time constraint for the frame image task is:

[0057] ;

[0058] Among them, the task The start time and finish time have the following relationship:

[0059] .

[0060] Secondly, this invention provides a multi-objective scheduling method for sensor-computer collaborative visual detection tasks in continuous video stream ultra-gap scenarios, implemented based on the system described in the first aspect. The scheduling method includes the following steps:

[0061] Step S1: The edge device computing end receives the data frame task transmitted by the acquisition device sensing end and performs schedulability verification;

[0062] Step S2: Based on the test result indicating that the task is schedulable, the main edge device divides the detection task into multiple sub-tasks to obtain a sub-task set. ;

[0063] Step S3: In the subtask set Based on this, the main edge device uses the improved NSCKGA-II algorithm to obtain a scheduling scheme to meet the constraints of the task scheduling model;

[0064] Step S4: Each edge device executes a subtask according to the scheduling scheme to obtain the final result.

[0065] Furthermore, in step S3, the task scheduling method based on the improved cuckoo genetic algorithm specifically includes:

[0066] Step S3.1: After the sensing end acquires a new image, the edge computing end divides the detection task into multiple sub-tasks and determines the scheduling solution representation form based on the number of tasks in the sub-task set.

[0067]

[0068] Assume there is Sub-tasks and If there are multiple edge devices, then the total length of the sequence is equal to the total number of subtasks. The range of values ​​for each digit is: , No. Number of digits The representative will number Tasks assigned to edge devices ;

[0069] Step S3.2, initialize the population; assume the number of nests in the population is... The number of cuckoos in each nest is Each individual cuckoo is composed of The population consists of several gene sequences; during population initialization, a completely random method is used to first generate... 1 gene sequence, then randomly assigned on average to 100 gene sequences. Individual cuckoos were randomly distributed among the nests; for each individual cuckoo was randomly assigned to a nest. Each of the different populations is generated using the method described above, thus yielding different initial scheduling strategies.

[0070] Step S3.3 updates the individual cuckoos in the nest, including selection, crossover, and mutation operations. The selection operation uses a tournament strategy, ensuring that better individuals have a higher probability of being selected as parents based on fitness function values. The crossover operation uses a single-point crossover method. The mutation operation, based on the mutation probability, mutates genes at a position other than the first and last genes in the individual sequence. The mutation probability is dynamically adjusted according to the total number of tasks, and the maximum mutation length is set to a fraction of the chromosome length. ;

[0071] During the individual selection process, task completion time and the number of edge devices used are used as multi-objective optimization indicators, and a comprehensive judgment is made in conjunction with real-time constraints: First, the feasibility of the scheduling solution sequence is determined based on the task deadline; if one meets the real-time constraint while the other does not, the feasible individual is selected first; if both meet the real-time constraint, the individual with better overall performance in terms of shorter task completion time and fewer edge devices is selected; if neither meets the real-time constraint, the individual with the higher fitness value after weighted fusion of task completion time and number of edge devices is selected, and the infeasible solution is accepted with a certain probability according to the Metropolis acceptance criterion to enhance the global search capability of the algorithm; if all candidate scheduling schemes fail to meet the task real-time requirements, the system automatically outputs the suboptimal scheduling scheme with the shortest latest completion time and the fewest edge devices used, thereby minimizing task latency while ensuring resource utilization efficiency.

[0072] Step S3.4: Random migration selects nests that need to be updated, using fast non-dominated sorting; fast non-dominated sorting includes the steps of level assignment and hierarchical progression.

[0073] The process involves: Rank assignment, which includes creating an empty non-dominated solution set and setting its rank to 1. For each individual in the population, the number of individuals dominated by each individual (denoted as n) and the set of individuals they dominate are calculated. If an individual's dominance count n is 0, it indicates that it is currently in a non-dominated state; its rank is set to 1, and it is added to this non-dominated solution set. Hierarchical progression involves setting the initial processing rank to 1. The non-dominated solution set of the current rank is then processed continuously until all individuals have been processed. For each individual A in the current rank, its dominated set is traversed. For each individual B in the set, B's dominance count n is decreased. If individual B's dominance count n drops to 0, it indicates that B is now in a non-dominated state; its rank is set to the current rank + 1, and B is added to the non-dominated solution set of the next rank.

[0074] Step S3.5: In the nest replacement operation, firstly, the fitness value of each cuckoo individual is calculated, and the optimal solution corresponding to the current nest is updated accordingly. Then, a new candidate nest is randomly generated and its fitness value is calculated: if the fitness value of the new nest is better than (i.e., less than) that of the original nest, the new nest replaces the original nest; otherwise, the new nest is accepted with a certain probability according to the Metropolis criterion to enhance the algorithm's ability to escape local optima. After completing the update of a single nest, all nests in the population are uniformly evaluated and sorted according to the fitness function, and several nests with low fitness rankings are randomly eliminated, while an equal number of new nests are generated to maintain the population size. At the end of each iteration, the sequence of individuals with the best fitness in the current population is recorded and retained. When the algorithm reaches the preset maximum number of iterations, the obtained optimal scheduling scheme is output.

[0075] Furthermore, in step S3.3, the fitness calculation function is designed for real-time detection:

[0076]

[0077] in, Indicates the target fitness value. The latest completion time is calculated to allocate tasks according to the scheduling scheme of individual cuckoo sequences; the fitness value of individual cuckoo sequences is obtained from the above fitness function, and the fitness value of a cuckoo nest is the maximum fitness value of the gene sequences it contains.

[0078]

[0079] in, This represents a sequence of individual cuckoo birds. This is a fitness function for the nest where a cuckoo resides, taking the maximum fitness value of all gene sequences contained within it.

[0080] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in the second aspect.

[0081] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention proposes a multi-objective scheduling optimization strategy for ultra-spacing computer vision tasks, combining sensing and computing, specifically for timeliness detection scenarios involving ultra-spacing tasks. It models the scheduling problem of ultra-spacing tasks as a multi-objective combinatorial optimization problem, with minimizing task completion time and reducing the number of edge devices used as dual optimization objectives. It comprehensively considers the execution efficiency of individual tasks and the collaborative operation capabilities among multiple edge devices, aiming to achieve a Pareto optimal trade-off between system performance and resource overhead. Based on this, a multi-objective scheduling optimization model combining sensing and computing is constructed, clearly defining key performance indicators such as task completion time and the number of edge devices used, and introducing practical constraints such as task processing volume and the number of available edge devices to ensure the model's effectiveness. Practicality and Efficiency in Engineering Applications: To overcome the shortcomings of traditional genetic algorithms in solving high-dimensional multi-objective scheduling problems, such as local convergence, insufficient population diversity, and low search efficiency, this invention proposes an improved fast non-dominated sorting multi-objective cuckoo genetic algorithm. This improved cuckoo genetic algorithm replaces the crossover and mutation operations in traditional genetic algorithms, effectively enhancing population diversity and global exploration capabilities. Furthermore, the proposed sensory-computer collaborative multi-objective scheduling model and the improved cuckoo genetic algorithm significantly reduce the number of edge devices deployed while maintaining a high task completion rate and system stability, demonstrating excellent comprehensive optimization performance and engineering applicability, all while meeting the stringent requirements of business operations for task throughput and timeliness. Attached Figure Description

[0082] Figure 1 is a schematic diagram of the multi-objective scheduling structure for sensory computation collaborative visual detection tasks in continuous video stream ultra-gap scenarios.

[0083] Figure 2 is a flowchart of the improved fast non-dominated sorting multi-objective cuckoo genetic algorithm.

[0084] Figure 3 shows the Pareto front obtained by NSCKGA-II. Detailed Implementation

[0085] This invention proposes a multi-objective scheduling system and method for sensor-computer collaborative visual inspection tasks in continuous video stream ultra-gap scenarios, solving the real-time bottleneck problem caused by "fast acquisition, slow computation". Targeting the typical application scenario of limited inter-frame processing time (i.e., "ultra-gap tasks"), this invention, based on a sensor-computer collaborative computing architecture, constructs a unified modeling framework integrating edge devices, acquisition devices, visual inspection models, and task parameters. It clearly defines core constraints and designs an improved fast non-dominated sorting multi-objective Cuckoo Genetic Algorithm (NSCKGA-II), with the dual optimization objectives of minimizing the latest task completion time and minimizing the number of edge devices used, achieving efficient task decomposition and collaborative scheduling of edge resources. This algorithm integrates tournament selection, dynamic mutation strategies, and the Metropolis criterion, effectively improving population diversity and global search capabilities. Even when infeasible solutions dominate, it can output the suboptimal scheduling scheme with the lowest latency and optimal resources. The system and method provided by this invention improve the task throughput and device resource utilization at the edge, significantly reducing the number of edge devices required while ensuring high task completion rates and system stability, demonstrating superior comprehensive optimization performance.

[0086] To enable those skilled in the art to understand the technical solution of the present invention more clearly, the present invention will be described in further detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0087] As shown in Figures 1 and 2, this invention provides a multi-objective scheduling system and method for sensor-computer collaborative visual detection tasks in continuous video stream ultra-gap scenarios. It comprehensively considers real-time visual detection tasks running on edge devices and supports the establishment of real-time task models and the definition of real-time constraints. The method includes:

[0088] Step S1: The edge device computing end receives the data frame task transmitted by the acquisition device sensing end, and performs schedulability verification to determine whether it is an ultra-gap scenario.

[0089] Step S2: Based on the test result indicating that the task is schedulable, the main edge device divides the detection task into multiple sub-tasks to obtain a sub-task set. ;

[0090] Step S3: In the subtask set Based on this, the main edge device adopts the improved non-dominated sorting genetic algorithm-II (NSCKGA-II) to obtain a scheduling scheme to meet the constraints of the task scheduling model.

[0091] Step S4: Each edge device executes a subtask according to the scheduling scheme to obtain the final result.

[0092] In this embodiment, as shown in FIG1, step S2 specifically includes the following:

[0093] Step S21: Establish edge device model, acquisition device model, computer vision detection model, and task model for real-time computer vision detection; these models together constitute an efficient and accurate detection system, ensuring the real-time performance of computer vision detection.

[0094] Step S22: Define a set of constraints for the task scheduling model to meet the requirements of real-time computer vision detection task scheduling. These constraints ensure the rationality and efficiency of task scheduling, enabling the detection work to proceed in an orderly and efficient manner.

[0095] Furthermore, module S21 includes the following models: S211: edge device model; S212: acquisition device model; S213: visual detection algorithm model; S214: task model;

[0096] The S211 edge device model includes: edge devices,

[0097] Furthermore, among which a single edge device It can be represented as:

[0098] in: Indicates edge device Available computing resources; Indicates edge device Network transmission bandwidth; Indicates edge device The execution frequency, i.e., the number of cycles executed per second.

[0099] S212 data acquisition device model: ,

[0100] in: The frame rate for the data stream collected by the camera device. This is the time when data stream acquisition begins.

[0101] Furthermore, This is the interval between frames (in milliseconds), and also the longest processing time for a single frame in real-time detection.

[0102] S213 target detection algorithm model: ,

[0103] in: Represents the object detection model ; Indicate execution Number of calculation cycles required; express The computational resources required to detect one frame of an image. In this embodiment, The detection time information is shown in the table below: Table 1

[0104]

[0105] S214 Task Model:

[0106] in: This represents the set of real-time tasks to be detected, with a number of tasks. ; Indicates the number is The tasks to be tested.

[0107] Furthermore, module S22 defines a set of model constraints based on task-related constraints and real-time detection constraints. Module S22 includes:

[0108] S221 Task Constraints

[0109] For computer vision detection tasks, the detection task of a single model for a single frame image cannot be assigned to multiple edge devices.

[0110]

[0111] S222 Computational Resource Constraints

[0112] At any given moment, the resources required for a task executed on each edge device cannot exceed the maximum resource limit of the edge device.

[0113]

[0114] S223 Execution Time Constraint

[0115] ;

[0116] In summary, step S2 specifically includes:

[0117] The overall model for real-time detection and scheduling of defects in the high-speed rail overhead contact system using a combination of sensing and computation is as follows:

[0118] Objective function:

[0119]

[0120] Time constraints:

[0121] ;

[0122]

[0123] Resource constraints:

[0124] Regarding time Edge devices have:

[0125]

[0126] Task constraints:

[0127]

[0128] In this embodiment, as shown in FIG2, step S3 specifically includes the following:

[0129] Step S31: After generating the subtask set, generate a sequence of individual cuckoo birds based on the number of tasks in the subtask set:

[0130]

[0131] Among them: the number of tasks is , Indicates numbering Subtasks are assigned to edge devices .

[0132] Step S32: Assume the number of nests in the population is The number of cuckoos in each nest is Each individual cuckoo is composed of It consists of several gene sequences. During population initialization, a completely random method is used to first generate... 1 gene sequence, then randomly assigned on average to 100 gene sequences. Individual cuckoos were then randomly distributed among the nests. Each of the different populations was generated using the method described above.

[0133] Step S33: Update the individual cuckoos in the nest, including selection, crossover, and mutation operations; the selection operation adopts a tournament strategy, ensuring that better individuals have a higher probability of being selected as parents based on fitness function values; the crossover operation uses single-point crossover; the mutation operation performs mutation operations on genes at a certain position other than the first and last genes of the individual sequence according to the mutation probability; the mutation probability is dynamically adjusted according to the total number of tasks, and the maximum mutation length is set to the chromosome length. The selection operation chooses individuals based on fitness assessment and real-time constraints: the sequence of cuckoo individuals is determined according to real-time constraints. And cuckoo individual sequence The feasibility is assessed by selecting the sequence of individuals from the feasible solution when one is feasible and the other is infeasible; otherwise, the sequence of individuals from the feasible solution is selected. A larger sequence of individuals; when both sequences of individuals are infeasible, choose... Larger sequences, and according to The algorithm uses criteria to determine whether to accept an infeasible solution; when a task cannot meet real-time requirements, the algorithm will provide the solution with the shortest latest completion time.

[0134] Step S34: Randomly migrate to select the nest that needs to be updated.

[0135] Step S35: Nest Replacement Operation. Calculate the fitness value of individual cuckoos and update the optimal nest solution. Specifically, new nests are randomly generated, and their fitness values ​​are calculated. If the fitness value of the new nest is less than that of the original nest, the original nest is retained; otherwise, the Metropolis criterion is used to determine whether to accept the new nest. Then, all nests in the population are evaluated according to the fitness function, sorted according to their fitness values, and the lowest-fitting nests are randomly eliminated, with new nests generated. After updating the nests, the sequence of individuals with the highest fitness in the population is calculated and the result is retained. The optimal solution is output when the maximum number of iterations is reached.

[0136] In this embodiment, the optimal solution is shown in Figure 3.

[0137] This invention aims to construct a co-operational real-time computer vision detection model for real-time detection scenarios, and designs an improved NSCKGA-II scheduling strategy to meet the specific requirements of real-time detection. The proposed model and method can effectively improve task execution efficiency and edge device utilization in computer vision detection scenarios, better meet the requirements of real-time detection, and have good application prospects.

[0138] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-objective scheduling system for sensor-computer collaborative visual detection tasks in continuous video stream ultra-gap scenarios, characterized in that, include: The real-time computer vision detection module includes an edge device model, an acquisition device model, a computer vision detection model, and a task model, used to describe the scheduling of computer vision detection tasks under real-time detection requirements. The edge device model describes the relevant information and parameters of the edge device, the acquisition device model describes the relevant information and parameters of the sensing device, the computer vision detection model describes the model used for target detection, and the task model describes the relevant parameters of the edge detection task and constrains the task execution time. The constraint construction module is used to establish a set of constraints for the task scheduling model to meet the requirements of real-time computer vision detection task scheduling.

2. The multi-objective scheduling system for sensor-computer collaborative visual detection tasks in continuous video stream ultra-gap scenarios as described in claim 1, characterized in that, The edge device model is represented as follows: ;in, This represents a set of edge devices, with the number of edge devices being... ； Indicates the number is Edge devices; where a single edge device Represented as: ;in, Indicates edge device Available computing resources; Indicates edge device Network transmission bandwidth; Indicates edge device The execution frequency, i.e., the number of cycles executed per second; the model of the acquisition device is represented as follows: ;in, This represents the set of data acquisition devices, with the number of devices being [number missing]. ； Indicates the number is Image acquisition devices; where each acquisition device is represented as: ;in, for The frame rate of the data stream, i.e., the number of images captured per second; express The moment when data stream acquisition begins; express The amount of data captured in a single frame image; by We can obtain, for The time interval between acquired data frames, i.e., the time to acquire a single frame image; the computer vision model is represented as follows: ;in, Represents a set of computer vision detection models; Computer vision detection models that may be used in the overall business process ;in, ； Indicate execution Number of calculation cycles required; express The computational resources required to detect one frame of an image; the task model is represented as follows: ;in, This represents the set of real-time tasks to be detected, with a number of tasks. ； Indicates the first The task to be detected in a frame image; where, ； For the task The computer vision detection model used. Indicates task The data acquisition equipment that needs to be processed Indicates task The amount of data, Indicates task The number of calculation cycles, Indicates task Required computing resources; of which: ； ； 。 3. The multi-objective scheduling system for sensor-computer collaborative visual detection tasks in continuous video stream ultra-gap scenarios according to claim 2, characterized in that, The task model is a super-gap task, meaning a task where the processing time of the detection algorithm model is longer than the data frame interval time. If at least one visual detection model holds true, then a hyper-gap task scenario is constituted.

4. The multi-objective scheduling system for sensor-computer collaborative visual detection tasks in continuous video stream ultra-gap scenarios as described in claim 1, characterized in that, definition The smallest subtask to be scheduled represents the task on the edge device. Using computer vision detection models For camera device The first collection Frame images are processed; Indicates task At edge resource nodes The time required for the above to complete; Indicates task Transmitted to edge resource nodes Time required; Indicates that it is composed of edge resource nodes Complete the task The total time required; of which, ； ； Since edge device resource nodes can work in parallel, with each edge device independently completing its own task, the time required for the system to process all subtasks is... for: 。 5. The multi-objective scheduling system for sensor-computer collaborative visual detection tasks in continuous video stream ultra-gap scenarios according to claim 4, characterized in that, After scheduling, we get , indicating in Is the time on the edge device? Using computer vision detection models For camera device The first collection The frame image is processed.

6. The multi-objective scheduling system for sensor-computer collaborative visual detection tasks in continuous video stream ultra-gap scenarios according to claim 1, characterized in that, A set of constraints is defined for the real-time detection requirements of computer vision task scheduling, including task constraints, computing resource constraints, and real-time detection time constraints. The task constraints are as follows: For the models involved in the catenary defect detection task, the detection task of a single model in a single frame image cannot be assigned to multiple edge devices during the frame interval. The above definition indicates that for the split detection subtask, the task of a single detection model for a single frame image only needs to be allocated to one edge device; the computational resource constraints are as follows: for each time step, the resources required for the task executed on each edge device cannot exceed the maximum resource amount of the edge device. The above definition indicates that, at any given moment, the resources required for the task executed on each edge device must be less than or equal to the maximum resource amount of the edge device. The real-time detection time constraint is as follows: In real-time detection, the earliest start time of the task for each frame must be later than the arrival time of the image, and the latest completion time must be earlier than the arrival time of the next frame, in order to meet the real-time requirements. Since the super-interval task cannot meet the strong real-time constraint, the maximum completion time in the time constraint does not have to be less than the arrival time of the next frame. ;in, Indicates the detection task The start time, That is, the earliest start time of all detection tasks for a single frame image; Indicates the detection task End time, The latest completion time of all detection tasks for a single frame image; where image acquisition equipment... The first collection The arrival time of the frame image is ; Regarding the first The time constraint for the frame image task is: Among them, the task The start time and finish time have the following relationship: 。 7. A multi-objective scheduling method for sensor-computer collaborative visual detection tasks in continuous video streams with large gaps, characterized in that, This method is implemented based on the system described in any one of claims 1-6, and includes the following steps: Step S1: The edge device computing end receives the data frame task transmitted by the sensing end of the acquisition device and performs schedulability verification to determine whether it is an ultra-gap scenario; Step S2: Based on the verification result that the task is schedulable, the main edge device divides the detection task into multiple sub-tasks to obtain a sub-task set. Step S3: In the subtask set Based on this, the main edge device uses the improved NSCKGA-II algorithm to obtain a scheduling scheme to meet the constraints of the task scheduling model; Step S4: Each edge device executes a sub-task according to the scheduling scheme to obtain the final result.

8. The method according to claim 7, characterized in that, In step S3, the improved fast non-dominated sorting multi-objective cuckoo genetic algorithm specifically includes: Step S3.1, after the sensing end acquires a new image, the edge computing end divides the detection task into multiple sub-tasks, and determines the scheduling solution representation form according to the number of tasks in the sub-task set: Assuming there is Sub-tasks and If there are multiple edge devices, then the total length of the sequence is equal to the total number of subtasks. The range of values ​​for each digit is: , the Number of digits The representative will number Tasks assigned to edge devices Step S3.2, initialize the population; assume the number of nests in the population is... The number of cuckoos in each nest is Each individual cuckoo is composed of The population consists of several gene sequences; during population initialization, a completely random method is used to first generate... 1 gene sequence, then randomly assigned on average to 100 gene sequences. Individual cuckoos were randomly distributed among the nests; for each individual cuckoo was randomly assigned to a nest. Different populations are generated using the methods described above, resulting in different initial scheduling strategies. Step S3.3 updates the individual cuckoos within the nests through selection, crossover, and mutation operations. The selection operation employs a tournament strategy, ensuring that superior individuals have a higher probability of being selected as parents based on fitness function values. The crossover operation uses a single-point crossover method. The mutation operation, based on the mutation probability, mutates genes at a position other than the first and last genes in the individual sequence. The mutation probability is dynamically adjusted according to the total number of tasks, and the maximum mutation length is set to the chromosome length. In the individual selection process, task completion time and the number of edge devices used are used as multi-objective optimization indicators, and a comprehensive judgment is made in conjunction with real-time constraints: First, the feasibility of the scheduling solution sequence is determined based on the task deadline; if one meets the real-time constraint while the other does not, the feasible individual is selected first; if both meet the real-time constraint, the individual with better overall performance in terms of shorter task completion time and fewer edge devices used is selected; if neither meets the real-time constraint, the individual with the higher fitness value after weighted fusion of task completion time and the number of edge devices used is selected, and the individual is further selected based on the Metropolis acceptance criterion. The system accepts the infeasible solution with probability; if none of the candidate scheduling schemes can meet the real-time requirements of the task, the system automatically outputs the suboptimal scheduling scheme with the shortest latest completion time and the fewest edge devices used; in step S3.4, random migration selects the nests that need to be updated, using fast non-dominated sorting; fast non-dominated sorting includes the steps of level assignment and hierarchical progression; wherein: level assignment includes creating an empty non-dominated solution set and setting its level to 1; traversing each individual in the population, calculating the number of individuals dominated by each individual, denoted as n, and the set of individuals they dominate; if the number of individuals dominated by an individual n is 0, it indicates that it is currently... The previous state was non-dominated, so its level was set to 1 and it was added to this non-dominated solution set; the hierarchical progression included setting the initial processing level to 1; then continuing to process the non-dominated solution set of the current level until all individuals were processed; for each individual A in the current level, iterated through the set of individuals it dominated, and for each individual B in the set, reduced the number of dominated individuals n of B; if the number of dominated individuals n of individual B decreased to 0, it meant that B was now in a non-dominated state, its level was set to the current level + 1, and B was added to the non-dominated solution set of the next level; in step S3.5, in the nest replacement operation, firstly, the fitness value of each cuckoo individual was calculated, and based on this... Update the optimal solution corresponding to the current nest; then, randomly generate a new candidate nest and calculate its fitness value: if the fitness value of the new nest is less than that of the original nest, replace the original nest with the new nest; otherwise, accept the new nest with a certain probability according to the Metropolis criterion; after completing the update of a single nest, uniformly evaluate and sort all nests in the population according to the fitness function, and randomly eliminate several nests with low fitness ranking, while generating an equal number of new nests to maintain the population size; at the end of each iteration, record and retain the sequence of individuals with the best fitness in the current population; when the algorithm reaches the preset maximum number of iterations, output the obtained optimal scheduling scheme.

9. The method according to claim 8, characterized in that, In step S3.3, the fitness calculation function is designed for real-time detection: ;in, Indicates the target fitness value. The task completion time is calculated based on the scheduling scheme that allocates tasks according to the individual cuckoo sequences. The number of edge devices used; the fitness value of the scheduling solution sequence is obtained by the fitness function mentioned above, and the fitness value of the cuckoo nest is the maximum fitness value of the gene sequences it contains; ;in, This represents a sequence of individual cuckoo birds. This is a fitness function for the nest where a cuckoo resides, taking the maximum fitness value of all gene sequences contained within it.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in claim 7.

Images