Camera scheduling method and system in video monitoring based on AI vision

Abstract

The invention discloses a camera scheduling method and system in video monitoring based on AI vision. The method comprises the following steps: S1, synchronously acquiring multi-channel real-time video stream data; s2, generating a dynamic semantic mask through dynamic semantic segmentation; s3, constructing a panoramic semantic map and dynamically generating an event thermodynamic diagram; s4, carrying out parallel processing on the multi-semantic entity state flow by utilizing an improved Performer to obtain a risk prediction value; s5, combining the information entropy, the risk prediction value and the cost to establish a dynamic value function evaluation; s6, all the cameras serve as intelligent agents, distributed competitive bidding negotiation is carried out through a CBBA algorithm, cooperative scheduling is autonomously executed, and a scheduling instruction is obtained; and S7, integrating the panoramic map, the thermodynamic diagram and the scheduling instruction to generate a visual interface. The method has the advantages of scene understanding depth, risk prediction foresight, efficient and robust scheduling decision and the like, and the monitoring resource utilization rate and the multi-target cooperative tracking capability in a complex scene are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and multi-agent, and particularly relates to a camera scheduling method and system in video monitoring based on AI vision. BACKGROUND

[0002] The video monitoring technology based on AI vision is widely applied in public safety, intelligent transportation, industrial production and other fields in recent years due to its strong scene perception ability, and becomes the core of building an intelligent security system. However, in practical application, the monitoring of large-scale complex scenes faces many challenges such as difficulty in multi-camera cooperation, inaccurate target behavior prediction, and lagging scheduling decision, and the intelligent level of the traditional monitoring system is still restricted by many aspects.

[0003] At present, most monitoring systems still adopt a centralized processing architecture, and all video data need to be transmitted back to the central server for analysis, which not only brings huge network bandwidth pressure and computing delay, but also has the risk of single point failure, and the system robustness is poor; although some systems have target detection function, they are mostly isolated single camera analysis, lack of understanding of the overall situation and effective cooperation between multi-cameras, resulting in blind area in the monitoring field of view, and the tracking of cross-camera moving targets is easily interrupted.

[0004] In addition, the scheduling logic of the existing system is mostly preset rules or simple target priority sorting, and lacks deep understanding and forward-looking prediction of the dynamic changes of the scene. It cannot evaluate the information gain and potential risks brought by the scheduling behavior itself, leading to unreasonable allocation of camera resources, and it is difficult to continuously and effectively focus on tracking high-value and high-risk events, which seriously affects the monitoring efficiency and early warning ability of the system in real complex scenes.

[0005] Therefore, how to provide a camera scheduling method and system in video monitoring based on AI vision is a problem to be solved by those skilled in the art. SUMMARY

[0006] One purpose of the present application is to propose a camera scheduling method and system in AI vision-based video monitoring, which fully integrates dynamic semantic segmentation, panoramic map construction, entity risk prediction and distributed collaborative scheduling, and constructs an intelligent monitoring process with pixel-level semantic understanding, multi-entity collaborative perception and distributed autonomous decision-making. The present application innovatively introduces an improved Performer with spatial perception sparse attention mechanism, which realizes accurate modeling of local interaction while maintaining linear computational complexity, and solves the efficiency bottleneck of large-scale entity parallel analysis. At the same time, the CBBA algorithm is used to construct a decentralized task allocation framework, so that each camera as an agent reaches a scheduling consensus through asynchronous bidding and negotiation, overcoming the delay and single point failure problem of centralized architecture. The present application has the advantages of deepening scene understanding, forward-looking risk prediction, and efficient and robust scheduling decision-making, which can significantly improve the monitoring resource utilization and multi-target collaborative tracking capability in complex scenes, thereby effectively solving the problems of limited monitoring field of view, target tracking interruption and scheduling response lag in existing methods.

[0007] According to an AI vision-based video monitoring camera scheduling method according to an embodiment of the present application, the following steps are included: S1, synchronously collecting real-time video stream data of a video monitoring area; S2, performing pixel-level dynamic semantic segmentation on the real-time video stream data, and generating a dynamic semantic mask for each identified dynamic semantic entity through mask query; S3, based on the dynamic semantic mask, fusing the fields of view of all cameras to construct a panoramic semantic map, and dynamically generating an event heat map on the panoramic semantic map according to the behavior category probability and interaction object information in the dynamic semantic mask; S4, constructing the semantic entities in the dynamic semantic mask into a plurality of semantic entity state streams, using an improved Performer with spatial perception sparse attention mechanism to perform parallel processing on the plurality of semantic entity state streams, dynamically selecting a neighbor semantic entity set, performing accurate modeling of local interaction, predicting the future behavior and event risk of each semantic entity, and outputting a risk prediction value; S5, defining monitoring information entropy for the field of view of each camera, combining the risk prediction value and the camera scheduling cost to establish a dynamic value function, and performing dynamic value evaluation for each scheduling instruction, and directly determining the highest priority when the total risk exceeds a preset threshold; S6, each camera as an agent performs distributed bidding and negotiation on the monitoring tasks defined by the dynamic value function through the CBBA algorithm, reaches a consistency consensus on task allocation through asynchronous communication, and autonomously executes the camera scheduling instruction to perform multi-target collaborative tracking and monitoring; S7, integrate the panoramic semantic map, event heat map, risk prediction value and execution status of camera scheduling instruction to generate a visual monitoring interface and output to the client.

[0008] Optionally, the S1 specifically comprises: S11, provide a unified clock reference for all cameras through a network time protocol server, synchronize the time stamp of the camera when collecting video stream data, configure a unique identifier for each camera, and embed the identifier into the data header of each frame of collected real-time video stream data; S12, convert the received optical signal into a continuous analog electrical signal through the image sensor built-in each camera, and sample and quantize the continuous analog electrical signal into a digital image frame through the analog-to-digital converter in each camera; S13, real-time compress and encode the digital image frame according to the preset video encoding standard, and package the encoded data into a network transmission protocol data packet to form and output real-time video stream data.

[0009] Optionally, the S2 specifically comprises: S21, decode the real-time video stream data to obtain continuous original image frames, and perform normalization processing on each original digital image frame to linearly scale the values of the RGB three channels of each pixel to the range of 0-1 to obtain the processed image frame; S22, input the normalized image frame into the encoder of a semantic segmentation model, extract multi-scale feature maps containing different spatial resolutions through a feature pyramid network composed of multiple preset convolution layers and down-sampling layers; S23, input the multi-scale feature map into the decoder of the semantic segmentation model, cross attention calculation between a preset number and size of mask query set and multi-scale feature map through a Transformer decoder to update the internal feature representation of each mask query; S24, update the internal feature representation of each mask query through a preset linear layer and Sigmoid activation function to generate a binary probability map with the same size as the original image as a pixel-level mask, and through another parallel multi-layer perceptron branch with a preset number of fully connected layers, each layer performs linear transformation and uses ReLU activation function to introduce nonlinearity, and extracts and integrates higher-order abstract features layer by layer; S25, the output vector of the last fully connected layer of the multi-layer perceptron is artificially and artificially divided into several continuous segments to obtain a multi-dimensional vector containing semantic entity categories, behavior category probabilities, position coordinates and velocity vectors as multi-dimensional behavior attributes; S26, bind each mask query generated pixel-level mask with the corresponding multi-dimensional behavior attribute one by one to form an independent dynamic semantic entity, and aggregate all semantic entities to form a structured data set containing all entity information, that is, a dynamic semantic mask.

[0010] Optionally, the S3 specifically comprises: S31, for each image frame dynamic semantic mask, extract the multi-dimensional behavior attribute of each dynamic semantic entity and bind it with the centroid coordinates of the pixel-level mask to form a three-dimensional space point cloud with semantic labels and motion states; S32, perform iterative nearest point matching between the three-dimensional space point cloud input by the current image frame and the point cloud of the previous image frame to calculate the rotation and translation transformation matrix of the camera, and perform coordinate transformation on the three-dimensional space point cloud based on the rotation and translation transformation matrix to preliminarily splice into a local point cloud map; S33, perform feature matching between the three-dimensional space point cloud of the current image frame and the three-dimensional space point clouds of a plurality of stored historical key frames, extract ORB descriptors for the three-dimensional space point clouds of the two image frames respectively, and count the number of feature point pairs with a Hamming distance less than a preset distance threshold, when the number of feature point pairs exceeds a preset number threshold, it is determined that a loop is detected, and the corresponding relationship between the historical key frame and the current image frame is recorded; S34, construct a pose graph containing all camera pose nodes and three-dimensional space point cloud nodes, use the corresponding relationship recorded by the loop detection as a loop constraint, perform global optimization on the entire pose graph based on the nonlinear least squares method, eliminate the cumulative error, and generate a globally consistent three-dimensional point cloud map; S35, project the optimized global three-dimensional point cloud map onto a two-dimensional grid map, and perform weighted summation on the behavior category probabilities of all dynamic semantic entities falling into each grid cell to calculate the comprehensive risk value of the current grid cell; S36, according to the comprehensive risk value of each grid cell and the preset risk threshold range, assign different colors to the two-dimensional grid map, red represents a high-risk area, and blue represents a low-risk area, and dynamically generate an event heat map on the panoramic semantic map.

[0011] Optionally, the S4 specifically comprises: S41, arrange the multi-dimensional behavior attribute of the same semantic entity in the dynamic semantic mask as a state vector in chronological order to construct a time series vector containing the historical state of the semantic entity, that is, a semantic entity state stream, and perform the same operation on all semantic entities to obtain a plurality of parallel semantic entity state streams; S42, input all parallel semantic entity state streams to the input embedding layer of the improved Performer, add a position encoding to the state vector of each time step in each state stream, splice all state vectors into an input matrix, the rows of the input matrix represent different semantic entities, and the columns represent the state vector of each semantic entity at a specific time step, i.e., features; S43, in the attention calculation of each layer of the improved Performer, the position coordinates of each semantic entity at the current time step are extracted from the input matrix, and the Euclidean distance between the current semantic entity and all other semantic entities is calculated, a dynamic neighborhood radius threshold is set, and the semantic entities with a Euclidean distance less than the dynamic neighborhood radius threshold from the current semantic entity are selected to form a neighbor semantic entity index set; S44, only for the current semantic entity and the semantic entities in the neighbor semantic entity index set, the attention approximation calculation of the improved Performer is performed, and the query, key, and value vectors are projected into a low-dimensional space through a preset random feature mapping function; S45, the query vector and the key vector are calculated through matrix multiplication in the low-dimensional space to obtain attention weights, the calculated attention weights are normalized through a Softmax function, and the neighbor semantic entity value vectors are weighted and summed using the normalized weights to generate an updated state vector of the current semantic entity; S46, the final state vector of each semantic entity after multi-layer spatial perception sparse attention calculation is input into a parallel multi-layer perception prediction head, and the classification probability of the semantic entity future behavior and the continuous numerical value of the event risk, i.e., the risk prediction value, are output through two preset weight fully connected layers.

[0012] Optionally, the dynamic neighborhood radius threshold specifically includes: The total number of all semantic entities in the current image frame is counted, and based on the position coordinates of these semantic entities on the panoramic semantic map, the area of the smallest convex polygon that can completely enclose all semantic entities is calculated. The scene density value of the current image frame is obtained by dividing the total number by the area; According to the category of the target semantic entity of the current neighborhood to be calculated, a preset reference table is searched to find and obtain a basic neighborhood radius corresponding to the category, the basic neighborhood radius is multiplied by a preset reference density value, and then divided by the scene density value of the current image frame to obtain a scaled adjustment radius; The adjusted radius is compared with a preset lower limit value of radius and a preset upper limit value of radius, if the adjusted radius is less than the lower limit value, the final neighborhood radius threshold value takes the lower limit value, if the adjusted radius is greater than the upper limit value, the final neighborhood radius threshold value takes the upper limit value, if the adjusted radius is between the lower limit value and the upper limit value, the final neighborhood radius threshold value takes the value of the adjusted radius itself.

[0013] Optionally, the S5 specifically comprises: S51, for a new field of view covered by a preset scheduling instruction executed by a camera, the continuous values of the event risk prediction values of all semantic entities in the new field of view are added to obtain a risk prediction value sum of the camera executing the instruction; S52, for a new field of view covered by a preset scheduling instruction executed by a camera, the number of all semantic entity categories is counted and the occurrence probability of each category is calculated, and the information entropy of the distribution of all semantic entity categories in the new field of view is calculated according to the information entropy formula, that is, for each semantic entity category, the occurrence probability is multiplied by the logarithm of the probability with 2 as the base, and the value of the reciprocal is taken; S53, the information entropy of the distribution of all semantic entity categories in the original field of view before executing the scheduling instruction is calculated by using the same information entropy formula, and the information entropy of the new field of view is subtracted from the information entropy of the original field of view, and the difference is the information entropy gain of the camera executing the scheduling instruction; S54, the scheduling cost is calculated, according to the horizontal angle and vertical angle required by the camera to execute the preset scheduling instruction, and the time required to move from the current position to the target position, a specific scheduling cost value is calculated through a preset cost conversion formula; S55, a high-risk trigger threshold is preset, when the risk prediction value sum corresponding to any preset scheduling instruction exceeds the high-risk trigger, the scheduling instruction is directly determined as the highest priority, and the dynamic value function calculation is skipped; S56, the dynamic value function is calculated, for all instructions whose risk prediction value sum does not exceed the high-risk trigger threshold, the risk prediction value sum and the information entropy gain are multiplied by the corresponding preset weight coefficient and added, and the scheduling cost value multiplied by the preset weight coefficient is subtracted, to obtain the dynamic value function value of the camera executing the preset scheduling instruction.

[0014] Optionally, the S6 specifically comprises: S61, a task list is constructed, all calculated monitoring tasks corresponding to a preset scheduling instruction and a dynamic value function value are constructed into a to-be-assigned monitoring task list, tasks directly determined as the highest priority due to the risk prediction value sum exceeding the high-risk trigger threshold are removed from the to-be-assigned list, and are marked as assigned state; S62, initialize bidding information, each camera as an agent, initialize its own data structure, including an empty local task allocation list, a local task bidding price list, and a local information copy recording the task allocation of all other camera agents; S63, start asynchronous distributed bidding, each camera agent independently and asynchronously executes a loop process until all monitoring tasks are allocated or a global consensus is reached, each camera agent traverses the list of tasks to be allocated, and for each unallocated task, calculates the dynamic value function value that can be obtained by executing the current task according to its current position and attitude, and takes it as its own bidding price; S64, each camera agent selects a task with the highest bidding price from all the bidding prices it calculates as the bidding target for this round, and adds the task and price information to its local task bidding price list; S65, each camera agent broadcasts the information in the local task bidding price list to other camera agents in the network through asynchronous communication, while continuously listening to and receiving bidding information broadcast from other camera agents, and updating its local information copy; S66, each camera agent checks whether its bidding for this round conflicts with the bidding of other agents according to the updated local information copy, i.e. multiple agents bidding for the same task, if there is a conflict, the agent with the highest bid wins the temporary allocation right of the task, and updates the allocation status of the task to the local information copy, if the bid is not the highest, the task is removed from the local task bidding price list and a new task is selected for bidding; S67, when a camera agent wins the bidding for a task, the task is moved from the local task bidding price list to the local task allocation list, and a final allocation confirmation message is broadcast to notify all other agents to permanently remove the task from the to-be-allocated list; S68, each camera agent generates corresponding camera scheduling instructions according to the monitoring tasks in the final confirmed local task allocation list, and controls the pan-tilt or drive motor to execute the scheduling instructions for multi-target cooperative tracking and monitoring.

[0015] Optionally, S7 specifically includes: taking the panoramic semantic map as the underlying geographic spatial reference, superimposing and rendering the event heat layer, binding the risk prediction value for each semantic entity and displaying it in the form of a dynamic label, and using a vector arrow and a state color block to real-time mark the position, orientation and execution state of the scheduling instructions of each camera, integrating all layers into an interactive monitoring picture, and pushing it to the display device of the client in real time through the streaming media protocol.

[0016] According to an embodiment of the present application, a camera scheduling system based on AI vision in video monitoring includes: A video stream synchronous acquisition module is configured to synchronously acquire real-time video stream data of a video monitoring area through multiple cameras. A dynamic semantic segmentation module is configured to perform pixel-level dynamic semantic segmentation on the real-time video stream data and output a dynamic semantic mask containing a pixel-level mask and multi-dimensional behavior attributes. A panoramic map construction module is configured to fuse all camera fields of view based on the dynamic semantic mask, construct a unified panoramic semantic map, and dynamically generate an event heat map according to behavior probability and interaction information. An entity risk prediction module is configured to construct a semantic entity state stream, use an improved Performer with spatial perception sparse attention mechanism, and output classification probability and continuous numerical values of event risk in parallel, which are risk prediction values. A dynamic value evaluation module is configured to define monitoring information entropy for each camera field of view, combine the risk prediction values with scheduling costs to establish a dynamic value function, and evaluate the value of scheduling instructions. A distributed collaborative scheduling module is configured to use each camera as an intelligent agent, perform distributed bidding and negotiation on monitoring tasks through a CBBA algorithm, and autonomously execute scheduling instructions after reaching a consensus. A visual monitoring interface module is configured to integrate the panoramic semantic map, event heat map, risk prediction value, and scheduling instruction execution state, generate a visual monitoring interface, and output the interface to a client.

[0017] The present application has the following advantages: First, the present application constructs a unified panoramic semantic map and dynamic event heat map through pixel-level dynamic semantic segmentation and multi-camera field of view fusion, realizes a leap from isolated pictures to global situation awareness, and provides a high-precision, global scene understanding basis for subsequent intelligent decision-making.

[0018] Second, the present application innovatively introduces an improved Performer with spatial perception sparse attention mechanism to perform parallel processing on multiple semantic entity state streams. The model dynamically selects a neighbor semantic entity set, realizes accurate modeling of local interaction while maintaining linear computational complexity, significantly improves the risk prediction accuracy and timeliness of individual behavior and group collaboration in complex scenes, and effectively solves the problem that traditional models are difficult to handle large-scale entity interaction due to high computational complexity.

[0019] In addition, the application adopts the CBBA algorithm to construct a decentralized distributed collaborative scheduling framework. Each camera acts as an agent and autonomously reaches a task allocation consensus through asynchronous bidding and negotiation, completely eliminating the dependence on the central server. This architecture not only greatly improves the response speed and scalability of the system, but also endows the system with excellent robustness, effectively overcoming the single-point failure risk and communication bottleneck problems existing in the centralized architecture.

[0020] In summary, through the deep integration of cutting-edge computer vision, efficient risk prediction models and robust multi-agent collaboration technology, the application realizes the intelligent upgrade of the monitoring system from passive recording to active perception, prediction and collaborative decision-making, significantly improving the monitoring efficiency, early warning accuracy and system reliability in complex scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, together with the embodiments of the application, to explain the application, and do not constitute a limitation on the application. In the drawings:

[0022] Fig. 1 A flowchart of a camera scheduling method in video monitoring based on AI vision is proposed for the application; Fig. 2 A structure diagram of a camera scheduling system in video monitoring based on AI vision is proposed for the application; Fig. 3 A multi-entity state flow parallel processing and risk prediction flowchart of an improved Performer based on spatial perception sparse attention mechanism is proposed for the application. DETAILED DESCRIPTION

[0023] The application will now be described in further detail with reference to the accompanying drawings. These drawings are all simplified schematic diagrams, and only illustrate the basic structure of the application in a schematic manner, and therefore only show the components related to the application.

[0024] REFERENCE Figs. 1-3 A camera scheduling method in video monitoring based on AI vision, comprising the following steps: S1, synchronously collecting real-time video stream data of a video monitoring area through multiple cameras; S2, performing pixel-level dynamic semantic segmentation on the real-time video stream data, and generating a dynamic semantic mask containing a pixel-level mask and multi-dimensional behavior attributes for each identified dynamic semantic entity through mask query; S3, based on the dynamic semantic mask, fusing the fields of view of all cameras to construct a unified panoramic semantic map, and dynamically generating an event heat map on the panoramic semantic map according to the behavior category probability and interaction object information in the dynamic semantic mask; S4, construct the semantic entities in the dynamic semantic mask into a plurality of semantic entity state streams in time sequence, use an improved Performer introducing a spatial perception sparse attention mechanism to perform parallel processing on the plurality of semantic entity state streams, dynamically screen a set of neighbor semantic entities for attention calculation according to the spatial position of each semantic entity in the panoramic semantic map, perform efficient attention calculation on the screened local neighbor set to accurately model local interaction, predict the future behavior and event risk of each semantic entity while maintaining linear computational complexity, and output a risk prediction value; S5, define monitoring information entropy for the field of view of each camera, combine the risk prediction value with the camera scheduling cost, and establish a dynamic value function, perform dynamic value evaluation for each scheduling instruction, and directly determine the highest priority when the total risk exceeds a preset threshold; S6, each camera acts as an agent to perform distributed bidding and negotiation on the monitoring tasks defined by the dynamic value function through the CBBA algorithm, reach a consensus on task allocation through asynchronous communication, and autonomously execute the camera scheduling instruction to perform multi-target collaborative tracking and monitoring; S7, integrate the panoramic semantic map, the event heat map, the risk prediction value, and the execution state of the camera scheduling instruction to generate a visual monitoring interface and output to the client.

[0025] The embodiment can significantly improve the resource utilization and emergency response efficiency in large-scale monitoring scenarios. By pixel-level dynamic semantic segmentation and multi-camera field of view fusion, a unified panoramic semantic map and dynamic event heat map are constructed, realizing the leap from isolated pictures to global situation awareness. The improved Performer is used to perform parallel risk prediction on multiple entity state streams, accurately capturing local interaction while maintaining efficient computation, greatly improving the accuracy and foresight of early warning. The decentralized scheduling framework constructed by the CBBA algorithm enables each camera to act as an agent to autonomously bid and negotiate, completely eliminating the dependence on the central server and giving the system excellent response speed and robustness. Finally, the monitoring mode is upgraded from passive recording to active perception, prediction, and collaborative decision-making, significantly enhancing the multi-target collaborative tracking capability and overall security level in complex scenarios.

[0026] In the embodiment, the S1 specifically includes: S11, provide a unified clock reference for all cameras through a network time protocol server, synchronize the time stamps of the cameras when collecting video stream data, configure a unique identifier for each camera, and embed the identifier in the data header of each frame of real-time video stream data collected by the camera to distinguish the video stream data collected by different cameras; S12, convert the received optical signals into continuous analog electrical signals through the image sensor built in each camera, and sample and quantize the continuous analog electrical signals into digital image frames through an analog-to-digital converter in each camera; S13, compress and encode the digital image frames in real time according to a preset video encoding standard, and encapsulate the encoded data into network transmission protocol data packets to form and output real-time video stream data.

[0027] In the embodiment, the S2 specifically comprises: S21, decode the real-time video stream data to obtain continuous original image frames, and perform normalization processing on each original digital image frame to linearly scale the numerical values of the RGB three channels of each pixel to the range of 0-1 to obtain a processed image frame; S22, input the normalized image frame into an encoder of a semantic segmentation model, and extract multi-scale feature maps containing different spatial resolutions through a feature pyramid network composed of multiple preset convolution layers and down-sampling layers; S23, input the multi-scale feature maps into a decoder of the semantic segmentation model, cross-attention calculate a preset number and size of mask query sets and the multi-scale feature maps through a Transformer decoder to update the internal feature representation of each mask query; S24, generate a binary probability map with the same size as the original image as a pixel-level mask through a preset linear layer and a Sigmoid activation function for the updated internal feature representation of each mask query, and extract and integrate higher-order abstract features layer by layer through another parallel multi-layer perceptron branch with a preset number of fully connected layers, each layer performing linear transformation and using a ReLU activation function to introduce nonlinearity; S25, artificially pre-divide the output vector of the last fully connected layer of the multi-layer perceptron into several continuous segments to obtain a multi-dimensional vector containing semantic entity categories, behavior category probabilities, position coordinates, and velocity vectors as multi-dimensional behavior attributes; S26, bind the pixel-level mask generated by each mask query with the corresponding multi-dimensional behavior attributes one by one to form an independent dynamic semantic entity, and aggregate all semantic entities to form a structured data set containing all entity information, i.e., a dynamic semantic mask.

[0028] In the embodiment, the S3 specifically comprises: S31, for each image frame dynamic semantic mask, extract the multi-dimensional behavior attributes of each dynamic semantic entity and bind them with the centroid coordinates of the pixel-level mask to form a three-dimensional spatial point cloud with semantic labels and motion states; S32, the three-dimensional space point cloud input by the current image frame is iteratively closest point matched with the point cloud of the previous image frame, a rotation and translation transformation matrix of the camera is calculated, and the three-dimensional space point cloud is coordinate-transformed based on the rotation and translation transformation matrix to preliminarily splice into a local point cloud map; S33, the three-dimensional space point cloud of the current image frame is feature-matched with the three-dimensional space point clouds of a plurality of historical key frames stored, ORB descriptors are extracted from the three-dimensional space point clouds of the two image frames respectively, and the number of feature point pairs with a Hamming distance less than a preset distance threshold is counted; when the number of feature point pairs exceeds a preset number threshold, it is determined that a loop is detected, and a corresponding relationship between the historical key frame and the current image frame is recorded; S34, a pose graph containing all camera pose nodes and three-dimensional space point cloud nodes is constructed, the corresponding relationship recorded by the loop detection is used as a loop constraint, a global optimization based on a nonlinear least square method is performed on the entire pose graph, cumulative errors are eliminated, and a globally consistent three-dimensional point cloud map is generated; S35, the optimized global three-dimensional point cloud map is projected onto a two-dimensional grid map, and the behavior category probabilities of all dynamic semantic entities falling into each grid cell are weighted and summed to calculate a comprehensive risk value of the current grid cell; S36, according to the comprehensive risk value of each grid cell and a preset risk threshold range, different colors are given on the two-dimensional grid map, red represents a high-risk area, and blue represents a low-risk area, and an event heat map is dynamically generated on the panoramic semantic map.

[0029] In the embodiment, the S4 specifically comprises: S41, the multi-dimensional behavior attributes of the same semantic entity in the dynamic semantic mask are taken as state vectors, which are arranged in time stamp order to construct a time sequence vector containing the historical state of the semantic entity, that is, a semantic entity state stream, and the same operation is performed on all semantic entities to obtain a plurality of parallel semantic entity state streams; S42, all parallel semantic entity state streams are input into an input embedding layer of the improved Performer, a position encoding is added to each time step state vector in each state stream, all state vectors are spliced into an input matrix, and the rows of the input matrix represent different semantic entities, and the columns represent the state vectors of each semantic entity at a specific time step, that is, features; S43, in each layer of attention calculation of the improved Performer, the position coordinates of each semantic entity at the current time step are extracted from the input matrix, the Euclidean distances between the current semantic entity and all other semantic entities are calculated, a dynamic neighborhood radius threshold is set, and the semantic entities with a Euclidean distance less than the dynamic neighborhood radius threshold from the current semantic entity are screened out to form a neighbor semantic entity index set. S44, only for the current real semantic entity and the semantic entity in the neighbor semantic entity index set, the attention approximation calculation of the improved Performer is performed, the query, key and value vectors are projected into a low-dimensional space through a preset random feature mapping function; S45, the query vector and the key vector are calculated by matrix multiplication in the low-dimensional space to obtain the attention weight, the calculated attention weight is normalized by a Softmax function, and the normalized weight is used to weight sum the value vector of the neighbor semantic entity to generate an updated state vector of the current semantic entity; S46, the final state vector of each semantic entity after the multi-layer space perception sparse attention calculation is input into a parallel multi-layer perception prediction head, and the classification probability of the semantic entity future behavior and the continuous value of the event risk, i.e., the risk prediction value, are output through two preset weight fully connected layers respectively.

[0030] The improved Performer with the space perception sparse attention mechanism is introduced in the step of the application, efficient parallel processing and accurate risk prediction of large-scale semantic entity state flow are realized. First, the dynamic semantic entities are constructed into multiple independent state flows according to the time sequence, and the structured space-time feature matrix is formed by input embedding and position coding. In the attention calculation, the model does not blindly calculate the global interaction, but dynamically selects the neighbor set according to the spatial position between entities, and reduces the calculation complexity from quadratic to linear. Through the efficient attention approximation calculation in the selected local neighbor set, the local interaction is accurately modeled, and the dynamic influence between group behavior and individuals is effectively captured. Finally, the state vector updated by multiple layers is input into the prediction head, and the future behavior and risk value are output. The application significantly improves the calculation efficiency and model scalability when processing large-scale entities while ensuring prediction accuracy, providing core support for real-time risk warning in complex scenarios.

[0031] In the embodiment, the dynamic neighborhood radius threshold specifically includes: The total number of all semantic entities in the current image frame is counted, and the area of the smallest convex polygon that can completely surround all semantic entities is calculated based on the position coordinates of the semantic entities on the panoramic semantic map. The scene density value of the current image frame is obtained by dividing the total number by the area. According to the category of the target semantic entity of the current neighborhood to be calculated, a preset reference density value is multiplied by the basic neighborhood radius corresponding to the category obtained from a preset reference table, and then divided by the scene density value of the current image frame to obtain a scaled adjustment radius. The adjusted radius is compared with a preset lower limit value of radius and a preset upper limit value of radius, if the adjusted radius is less than the lower limit value, the final neighborhood radius threshold value takes the lower limit value, if the adjusted radius is greater than the upper limit value, the final neighborhood radius threshold value takes the upper limit value, if the adjusted radius is between the lower limit value and the upper limit value, the final neighborhood radius threshold value takes the value of the adjusted radius itself.

[0032] In the embodiment, the S5 specifically includes: S51, for a new field of view covered by a preset scheduling instruction executed by a camera, the continuous values of the event risk prediction values of all semantic entities in the new field of view are added to obtain a risk prediction value sum of the camera executing the instruction; S52, for a new field of view covered by a preset scheduling instruction executed by a camera, the number of all semantic entity categories is counted and the occurrence probability of each category is calculated, and the information entropy of the distribution of all semantic entity categories in the new field of view is calculated according to the information entropy formula, that is, for each semantic entity category, the occurrence probability is multiplied by the logarithm of the probability with 2 as the base, and the value of the reciprocal is taken; S53, the information entropy of the distribution of all semantic entity categories in the original field of view before executing the scheduling instruction is calculated by using the same information entropy formula, and the information entropy of the new field of view is subtracted from the information entropy of the original field of view, and the difference is the information entropy gain of the camera executing the scheduling instruction; S54, the scheduling cost is calculated, according to the horizontal angle and vertical angle required by the camera to execute the preset scheduling instruction, and the time required to move from the current position to the target position, a specific scheduling cost value is calculated through a preset cost conversion formula; S55, a high-risk trigger threshold is preset, when the risk prediction value sum corresponding to any preset scheduling instruction exceeds the high-risk trigger threshold, the scheduling instruction is directly determined as the highest priority, and the dynamic value function calculation is skipped; S56, the dynamic value function is calculated, for all instructions whose risk prediction value sum does not exceed the high-risk trigger threshold, the risk prediction value sum and the information entropy gain are multiplied by the corresponding preset weight coefficient, added, and the scheduling cost value multiplied by the preset weight coefficient is subtracted to obtain the dynamic value function value of the camera executing the preset scheduling instruction.

[0033] In the embodiment, the S6 specifically includes: S61, a task list is constructed, all calculated monitoring tasks corresponding to a preset scheduling instruction and a dynamic value function value are constructed into a to-be-assigned monitoring task list, tasks directly determined as the highest priority due to the risk prediction value sum exceeding the high-risk trigger threshold are removed from the to-be-assigned list, and marked as an assigned state; S62, initialize bidding information, each camera as an agent, initialize its own data structure, including an empty local task allocation list, a local task bidding price list, and a local information copy recording the task allocation of all other camera agents; S63, start asynchronous distributed bidding, each camera agent independently and asynchronously executes a loop process until all monitoring tasks are allocated or a global consensus is reached, each camera agent traverses the list of tasks to be allocated, for each task that has not been allocated, calculates the dynamic value function value that can be obtained by executing the current task according to its current position and attitude, and takes it as its own bidding price; S64, each camera agent selects a task with the highest bidding price from all the bidding prices it calculates as the bidding target of this round, and adds the task and price information to its local task bidding price list; S65, each camera agent broadcasts the information in the local task bidding price list to other camera agents in the network through asynchronous communication, at the same time, continuously listens to and receives bidding information broadcast from other camera agents, and updates its local information copy; S66, each camera agent checks whether its bidding in this round conflicts with the bidding of other agents according to the updated local information copy, that is, multiple agents bid for the same task, if there is a conflict, the agent with the highest bid wins the temporary allocation right of the task, and updates the allocation status of the task to the local information copy, if the bid is not the highest, the task is removed from the local task bidding price list, and a new task is selected for bidding; S67, when a camera agent wins the bidding of a task, the task is moved from the local task bidding price list to the local task allocation list, and a final allocation confirmation message is broadcast to notify all other agents to permanently remove the task from the to-be-allocated list; S68, each camera agent generates corresponding camera scheduling instructions according to the monitoring tasks in the final confirmed local task allocation list, and controls the pan-tilt or drive motor to execute the scheduling instructions for multi-target cooperative tracking and monitoring.

[0034] The step of the application constructs a decentralized camera distributed cooperative scheduling framework by introducing a CBBA algorithm. First, all monitoring tasks are constructed into a to-be-allocated list, and each camera agent is allowed to independently calculate the task value and generate a bidding price according to its own state. In the asynchronous bidding process, each agent solves the task conflict through broadcasting and negotiation, and finally the highest bidder wins the task and reaches global consensus. This mechanism completely eliminates the dependence on the central server, and decomposes the complex global optimization problem into multiple parallel local decision problems. The application can maintain the robustness and response ability of the system under complex conditions such as partial node failure or communication obstruction, greatly improve the efficiency and scalability of scheduling decisions, and ensure that multi-objective cooperative tracking tasks in large-scale monitoring networks can be quickly and autonomously executed.

[0035] In the embodiment, the S7 specifically includes: taking the panoramic semantic map as the underlying geospatial reference, superimposing and rendering the event heat map layer, binding the risk prediction value for each semantic entity and displaying it in the form of a dynamic label, and using a vector arrow and a state color block to mark the position, orientation and execution state of the scheduling instruction of each camera in real time, integrating all the layers into an interactive monitoring picture, and pushing it to the display device of the client in real time through the streaming media protocol.

[0036] An AI vision-based camera scheduling system in video monitoring, comprising: A video stream synchronous acquisition module for synchronously acquiring real-time video stream data of a video monitoring area through multiple cameras; A dynamic semantic segmentation module for performing pixel-level dynamic semantic segmentation on the real-time video stream data and outputting a dynamic semantic mask containing a pixel-level mask and multi-dimensional behavior attributes; A panoramic map construction module for fusing the fields of view of all cameras based on the dynamic semantic mask, constructing a unified panoramic semantic map, and dynamically generating an event heat map according to the behavior probability and interaction information; An entity risk prediction module for constructing a semantic entity state stream, using an improved Performer with spatial perception sparse attention mechanism, and parallelly outputting classification probability and continuous numerical value of event risk, i.e., risk prediction value; A dynamic value evaluation module for defining monitoring information entropy for each camera field of view, combining the risk prediction value and scheduling cost to establish a dynamic value function, and evaluating the value of the scheduling instruction; A distributed cooperative scheduling module for each camera to act as an agent, perform distributed bidding and negotiation on monitoring tasks through the CBBA algorithm, and autonomously execute the scheduling instruction after reaching consensus; A visual monitoring interface module for integrating the panoramic semantic map, event heat map, risk prediction value and execution state of the scheduling instruction, generating a visual monitoring interface and outputting it to the client.

[0037] Example 1 To verify the feasibility of the application in practice, the application is applied to a smart city comprehensive management platform of a certain coastal economic zone, specifically covering its core central business district and three surrounding large traffic hubs. More than 1200 high-definition cameras are deployed in this area, forming an extremely complex monitoring network. The traditional monitoring center faces great challenges, generating more than 5 TB of video data daily, and monitoring personnel need to focus on dozens of screens at the same time. For cross-regional, multi-target abnormal events such as mass gathering, suspicious person tracking, traffic accident escape, etc., the response is slow, with an average event confirmation time of more than 5 minutes, and more than 30% of cross-camera tracking tasks fail due to the delay of manual scheduling, which seriously hinders the efficiency and level of urban public safety management.

[0038] In this complex urban scene, the method of the application is fully deployed to replace the original centralized, manual monitoring mode. First, all 1200 video streams of the cameras are synchronously collected and pixel-level dynamic semantic segmentation is performed, identifying pedestrians, vehicles, non-motor vehicles, etc. in the picture one by one, and assigning each target a digital identity card containing its position, speed, behavior category. Using these data, a panoramic semantic map of the entire CBD is constructed in real time, and an event heat map is dynamically generated, allowing security personnel to visually see where the crowd is abnormally dense, where vehicles are severely parked, where high-risk events such as chasing and fighting occur, etc. The most core application is reflected in the prediction of entity risks and the intelligent collaborative scheduling of cameras. For example, when the system identifies an abnormally behaving pedestrian, it does not simply alarm, but uses the improved Performer to combine the motion states of dozens of pedestrians and vehicles around it to predict its movement path and potential risk level within the next 30 seconds. At the same time, each camera becomes an intelligent agent that autonomously negotiates according to the dynamic value of each scheduling instruction evaluated by the system, such as how many high-risk targets it can cover by turning to a certain position, how much new information it can see, and how high the cost of turning is. When the system predicts that the above-mentioned abnormal pedestrian will soon enter a blind area, the cameras closest to the blind area will compete for the right to cover the area by CBBA algorithm within a few tenths of a second, and the optimal one or group of cameras will automatically turn to cover the area seamlessly, realizing seamless relay tracking without human intervention. After a three-month trial run, the application has shown excellent performance. The following Table 1 shows the comparison data of the key security indicators between the application method and the traditional centralized plus manual scheduling method:

[0039] Table 1 Performance comparison data table of the application and the traditional method

[0040] According to the comparison data shown in Table 1, it can be seen that the camera scheduling method based on AI vision proposed in the application has an overwhelming performance advantage over the traditional method in complex urban security tasks, especially in task success rate, response timeliness and automation level.

[0041] In terms of task success rate, the application improves the success rate to a very high level of more than 94% in all four key scenarios, far exceeding the interval of 40% to 80% of the traditional method. For example, in the task of "chase of traffic accident escape vehicle", which has high dynamic and high timeliness requirements, the success rate of the traditional method is less than 41%, while the success rate of the application is as high as 94.2%, which is due to its accurate prediction and seamless cooperative tracking capability, effectively solving the core pain point of target loss in traditional monitoring.

[0042] In terms of response timeliness, the application compresses the confirmation response time of various tasks from the minute level of the traditional method to the second level through distributed autonomous decision-making, with a speed increase of more than 10 times. Especially in the task of "suspected person cross-region tracking", the average first locking time is sharply reduced from 45.8 seconds to 8.2 seconds, which wins a decisive time window for the advance deployment and effective intervention of security forces.

[0043] In terms of automation and intelligence level, the reduction of human intervention rate is the most intuitive manifestation. The traditional method highly depends on manual operation, and the intervention rate is generally more than 70%, while the application controls the human intervention rate to be generally less than 8% through intelligent cooperative scheduling, so that the monitoring center changes from passive response to active management, greatly liberating manpower and improving overall operation efficiency. Overall, through technology integration, the application not only solves the inherent shortcomings of traditional monitoring, but also pushes the intelligent level of urban security to a new height.

[0044] The above is only the preferred specific embodiment of the application, but the protection scope of the application is not limited thereto, any person skilled in the art can make equivalent replacement or change according to the technical solution and inventive concept of the application within the technical range disclosed by the application, which should be covered within the protection scope of the application.

Claims

1. A camera scheduling method in video surveillance based on AI vision, characterized in that, Includes the following steps: S1. Synchronously collect real-time video stream data from the video surveillance area; S2. Perform pixel-level dynamic semantic segmentation on real-time video stream data, and generate dynamic semantic masks for each identified dynamic semantic entity through mask query. S3. Based on dynamic semantic mask, the field of view of all cameras is fused to construct a panoramic semantic map. According to the behavior category probability and interaction object information in the dynamic semantic mask, an event heat map is dynamically generated on the panoramic semantic map. S4. Construct the semantic entities in the dynamic semantic mask into multiple semantic entity state streams. Use the improved Performer with the introduction of a spatially aware sparse attention mechanism to process the multiple semantic entity state streams in parallel, dynamically filter the set of neighboring semantic entities, perform accurate modeling of local interactions, predict the future behavior and event risk of each semantic entity, and output the risk prediction value. S5. Define the monitoring information entropy for the field of view of each camera, and combine the risk prediction value with the camera scheduling cost to establish a dynamic value function. For each scheduling instruction, perform dynamic value evaluation. When the total risk exceeds the preset threshold, it is directly determined to be the highest priority. S6. Each camera acts as an intelligent agent, using the CBBA algorithm to conduct distributed bidding and negotiation for monitoring tasks defined by a dynamic value function. Through asynchronous communication, it achieves a consensus on task allocation, autonomously executes camera scheduling instructions, and performs multi-target collaborative tracking and monitoring. S7. Integrate the panoramic semantic map, event heat map, risk prediction value, and execution status of camera scheduling instructions to generate a visual monitoring interface and output it to the client.

2. The camera scheduling method in video surveillance based on AI vision according to claim 1, characterized in that, S1 specifically includes: S11. A unified clock reference is provided for all cameras through a network time protocol server. The timestamps of the cameras are kept synchronized when acquiring video stream data. Each camera is configured with a unique identifier and the identifier is embedded in the data header of each frame of the acquired real-time video stream data. S12. The received optical signals are converted into continuous analog electrical signals by the image sensor built into each camera. The continuous analog electrical signals are sampled and quantized into digital image frames by the analog-to-digital converter in each camera. S13. The digital image frames are compressed and encoded in real time according to the preset video encoding standard, and the encoded data is encapsulated into network transmission protocol data packets to form and output real-time video stream data.

3. The camera scheduling method in video surveillance based on AI vision according to claim 1, characterized in that, S2 specifically includes: S21. Decode the real-time video stream data to obtain continuous raw image frames, and perform normalization processing on each raw digital image frame, linearly scaling the values ​​of the three RGB channels of each pixel to the range of 0-1 to obtain the processed image frame. S22. Input the normalized image frame into the encoder of a semantic segmentation model, and extract multi-scale feature maps with different spatial resolutions through a feature pyramid network composed of multiple preset convolutional layers and downsampling layers. S23. Input the multi-scale feature map into the decoder of the semantic segmentation model, and perform cross-attention calculation with a set of mask queries of a preset number and size and the multi-scale feature map through a Transformer decoder to update the internal feature representation of each mask query. S24. The internal feature representation of each updated mask query is generated as a pixel-level mask by a binary probability map of the same size as the original image through a preset linear layer and a Sigmoid activation function. Then, it is processed by another parallel multilayer perceptron branch with a preset number of fully connected layers. Each layer is linearly transformed and nonlinearity is introduced using the ReLU activation function. Higher-order abstract features are extracted and integrated layer by layer. S25. The output vector of the last fully connected layer of the multilayer perceptron is artificially pre-divided into several continuous segments to obtain a multidimensional vector containing semantic entity category, behavior category probability, position coordinates and velocity vector, which is used as a multidimensional behavior attribute. S26. Bind each pixel-level mask generated by the mask query to the corresponding multi-dimensional behavioral attribute one-to-one to form an independent dynamic semantic entity. Then, aggregate all semantic entities to form a structured data set containing all entity information, i.e., a dynamic semantic mask.

4. The camera scheduling method in video surveillance based on AI vision according to claim 1, characterized in that, S3 specifically includes: S31. For each image frame dynamic semantic mask, extract the multi-dimensional behavioral attributes of each dynamic semantic entity and bind them to the centroid coordinates of the pixel-level mask to form a three-dimensional spatial point cloud with semantic labels and motion states. S32. Iteratively match the nearest point between the 3D spatial point cloud of the current image frame and the point cloud of the previous image frame, calculate the rotation and translation transformation matrix of the camera, and perform coordinate transformation on the 3D spatial point cloud based on the rotation and translation transformation matrix to initially stitch together a local point cloud map. S33. Perform feature matching between the 3D spatial point cloud of the current image frame and the 3D spatial point clouds of multiple stored historical key frames. Extract ORB descriptors from the 3D spatial point clouds of the two image frames respectively, and count the number of feature point pairs with a Hamming distance less than a preset distance threshold. When the number of feature point pairs exceeds the preset number threshold, determine that a loop closure has been detected, and record the correspondence between the historical key frames and the current image frame. S34. Construct a pose graph containing all camera pose nodes and 3D spatial point cloud nodes. Use the correspondence of loop closure detection records as a loop closure constraint to perform global optimization of the entire pose graph based on nonlinear least squares method, eliminate accumulated errors, and generate a globally consistent 3D point cloud map. S35. Project the optimized global 3D point cloud map onto a 2D grid map, and sum the weighted probabilities of the behavior categories of all dynamic semantic entities falling into each grid cell to calculate the comprehensive risk value of the current grid cell. S36. Based on the comprehensive risk value of each grid cell and the preset risk threshold range, assign different colors to the two-dimensional grid map, with red representing high-risk areas and blue representing low-risk areas, and dynamically generate an event heat map on the panoramic semantic map.

5. The camera scheduling method in video surveillance based on AI vision according to claim 1, characterized in that, S4 specifically includes: S41. Take the multi-dimensional behavioral attributes of the same semantic entity in the dynamic semantic mask as a state vector, arrange them in the order of timestamps, construct a time series vector containing the historical state of the semantic entity, that is, a semantic entity state stream, perform the same operation on all semantic entities, and obtain multiple parallel semantic entity state streams. S42. Input all parallel semantic entity state streams into the input embedding layer of the improved Performer, add a positional encoding to the state vector at each time step in each state stream, and concatenate all state vectors into an input matrix. The rows of the input matrix represent different semantic entities, and the columns represent the state vector of each semantic entity at a specific time step, i.e., features. S43. In each layer of attention calculation of the improved Performer, the position coordinates of each semantic entity at the current time step are extracted from the input matrix, and the Euclidean distance between the current semantic entity and all other semantic entities is calculated. A dynamic neighborhood radius threshold is set, and semantic entities whose Euclidean distance to the current semantic entity is less than the dynamic neighborhood radius threshold are selected to form a neighbor semantic entity index set. S44. Perform an improved Performer attention approximation calculation only for semantic entities within the current semantic entity and neighboring semantic entity index sets, and project the query, key, and value vectors into a low-dimensional space through a preset random feature mapping function. S45. In the low-dimensional space, the query vector and key vector are multiplied by matrix to calculate the attention weights. The calculated attention weights are normalized by the Softmax function. The normalized weights are then used to perform a weighted summation on the value vectors of the neighboring semantic entities to generate the updated state vector of the current semantic entity. S46. The final state vector of each semantic entity after multi-layer spatial perception sparse attention calculation is input into a parallel multi-layer perceptron prediction head. The classification probability and event risk of the semantic entity's future behavior are output by two fully connected layers with preset weights, which are the risk prediction values.

6. The camera scheduling method in video surveillance based on AI vision according to claim 5, characterized in that, The dynamic neighborhood radius threshold specifically includes: The total number of all semantic entities in the current image frame is counted. Based on the position coordinates of these semantic entities on the panoramic semantic map, the area of ​​the smallest convex polygon that can enclose all semantic entities is calculated. The total number is divided by the area to obtain the scene density value of the current image frame. Based on the category of the target semantic entity in the neighborhood to be calculated, the basic neighborhood radius corresponding to the category is retrieved from a preset lookup table. The basic neighborhood radius is multiplied by a preset reference density value and divided by the scene density value of the current image frame to obtain a scaled adjustment radius. The adjustment radius is compared with a preset lower radius value and a preset upper radius value. If the adjustment radius is less than the lower limit value, the final neighborhood radius threshold is the lower limit value. If the adjustment radius is greater than the upper limit value, the final neighborhood radius threshold is the upper limit value. If the adjustment radius is between the lower limit value and the upper limit value, the final neighborhood radius threshold is the value of the adjustment radius itself.

7. A camera scheduling method for video surveillance based on AI vision according to claim 1, characterized in that, S5 specifically includes: S51. For a camera, execute a preset scheduling instruction to cover a new field of view, add up the consecutive values ​​of the event risk prediction values ​​of all semantic entities in the new field of view to obtain the sum of the risk prediction values ​​of the camera execution instruction. S52. For the new field of view to be covered after the camera executes the preset scheduling command, count the number of all semantic entity categories and calculate the occurrence probability of each category. Calculate the information entropy of the distribution of all semantic entity categories in the new field of view according to the information entropy formula. That is, for each semantic entity category, multiply the occurrence probability by the logarithm of the probability to the base 2 and take the opposite value. S53. Calculate the information entropy of the distribution of all semantic entity categories in the original field of view before executing the scheduling command using the same information entropy formula. Subtract the information entropy of the original field of view from the information entropy of the new field of view. The difference is the information entropy gain of the camera when executing the scheduling command. S54. Calculate the scheduling cost. Based on the horizontal and vertical angles that the camera needs to rotate to execute the preset scheduling command, and the time required to rotate from the current position to the target position, calculate a specific scheduling cost value through a preset cost conversion formula. S55. Preset high-risk trigger threshold: When the sum of the risk prediction values ​​corresponding to any preset scheduling instruction exceeds the high-risk trigger, the scheduling instruction will be directly judged as the highest priority and the dynamic value function calculation will be skipped. S56. Calculate the dynamic value function. For instructions where the sum of all risk prediction values ​​does not exceed the high-risk trigger threshold, multiply the obtained sum of risk prediction values ​​and information entropy gain by the corresponding preset weight coefficient, add them together, and subtract the scheduling cost value multiplied by the preset weight coefficient to obtain the dynamic value function value of the camera executing the preset scheduling instruction.

8. The camera scheduling method in video surveillance based on AI vision according to claim 1, characterized in that, S6 specifically includes: S61. Construct a task list. Construct a list of all the calculated monitoring tasks that correspond to a preset scheduling instruction and a dynamic value function value into a list of monitoring tasks to be assigned. Remove the tasks that are directly judged as the highest priority because the sum of the risk prediction values ​​exceeds the high-risk trigger threshold from the list of tasks to be assigned and mark them as assigned. S62. Initialize bidding information. Each camera, as an intelligent agent, initializes its own data structure, including an empty local task allocation list, a local task bidding bid list, and a local information copy that records the task allocation of all other camera intelligent agents. S63. Initiate asynchronous distributed bidding. Each camera agent independently and asynchronously executes the loop process until all monitoring tasks are assigned or a global consensus is reached. Each camera agent traverses the list of monitoring tasks to be assigned. For each task that has not yet been assigned, it calculates the dynamic value function value that can be obtained by executing the current task based on its current position and posture, and uses it as its bidding bid. S64. Each camera agent selects the task with the highest bid from all the bids it has calculated as the bidding target for this round, and adds the task and bid information to its local task bidding list. S65. Each camera agent broadcasts the information in its local task bidding list to other camera agents in the network via asynchronous communication. At the same time, it continuously listens for and receives bidding information broadcast by other camera agents and updates its own local information copy. S66. Each camera agent checks whether its current bidding conflict with other agents' bidding based on the updated local information copy. If multiple agents bid for the same task, and a conflict exists, the agent with the highest bid wins the temporary allocation right of the task and updates the task allocation status to the local information copy. If its bid is not the highest, it cancels the task from the local task bidding list and re-executes the selection of a new task for bidding. S67. When a camera agent wins a bid for a task, it moves the task from the local task bidding list to the local task assignment list and broadcasts a final assignment confirmation message to notify all other agents to permanently remove the task from the assignment list. S68. Each camera agent generates corresponding camera scheduling instructions based on the monitoring tasks in the final confirmed local task allocation list, and controls its own gimbal or drive motor to execute the scheduling instructions for multi-target collaborative tracking and monitoring.

9. A camera scheduling method in video surveillance based on AI vision according to claim 1, characterized in that, S7 specifically includes: using a panoramic semantic map as the underlying geospatial reference and overlaying a rendered event heatmap layer; binding risk prediction values ​​to each semantic entity and displaying them in the form of dynamic labels; using vector arrows and status color blocks to mark the position, orientation, and execution status of each camera's scheduling instructions in real time; integrating all layers into an interactive monitoring screen; and pushing it to the client's display device in real time via a streaming media protocol.

10. A camera scheduling system for video surveillance based on AI vision, comprising executing the camera scheduling method for video surveillance based on AI vision as described in any one of claims 1 to 9, characterized in that, include: The video stream synchronous acquisition module is used to synchronously acquire real-time video stream data of the video surveillance area through multiple cameras; The dynamic semantic segmentation module is used to perform pixel-level dynamic semantic segmentation on real-time video stream data and outputs a dynamic semantic mask containing pixel-level mask and multi-dimensional behavioral attributes. The panoramic map building module is used to fuse the views of all cameras based on dynamic semantic masks to build a unified panoramic semantic map, and dynamically generate event heatmaps based on behavior probability and interaction information. The entity risk prediction module is used to construct the semantic entity state flow. It utilizes an improved Performer with a spatially aware sparse attention mechanism to output continuous values ​​of classification probability and event risk in parallel, which are the risk prediction values. The dynamic value assessment module is used to define the monitoring information entropy for each camera's field of view, and to establish a dynamic value function by combining risk prediction values ​​and scheduling costs to assess the value of scheduling instructions. The distributed collaborative scheduling module is used by each camera as an intelligent agent to conduct distributed bidding and negotiation for monitoring tasks through the CBBA algorithm, and then autonomously execute scheduling instructions after reaching a consensus. The visualization monitoring interface module is used to integrate panoramic semantic maps, event heatmaps, risk prediction values, and scheduling instruction execution status to generate a visualization monitoring interface and output it to the client.

Images