Urban operation event handling method based on cooperation of multi-modal perception and intelligent agent

Abstract

The invention discloses a multi-modal perception and agent collaborative urban operation event handling method, which belongs to the technical field of artificial intelligence, and adopts the technical scheme that multi-modal front-end equipment is deployed to collect data, a fusion confidence coefficient is generated based on multi-source data, and an event work order is generated when the fusion confidence coefficient reaches a threshold value; the method comprises the following steps: classifying work orders by utilizing an urban management knowledge graph, determining responsibility subjects and disposal guidance, and determining priorities by integrating risks and regions through a weight model; abstracting various resources as disposal resources, and realizing work order and resource matching assignment under the constraint of response time limit; when a complex event condition is met, disassembling into sub-work orders according to a knowledge graph and carrying out parallel processing; the processing resources upload processing feedback, automatic verification is carried out based on computer vision and multi-source evidence, closed loop is qualified, if not, processing or evidence supplementation is carried out, and a closed loop result is archived and used for continuously optimizing the model and scheduling. The beneficial effect of the invention is that the urban operation event handling method based on multi-modal perception and intelligent agent cooperation is provided.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and specifically relates to a method for handling urban operation events involving multimodal perception and intelligent agent collaboration. Background Technology

[0002] Currently, the handling of smart city operation and management incidents (such as illegally parked motor vehicles, disorderly parking of non-motorized vehicles, illegal street vending, overflowing garbage, and damaged facilities) generally relies on manual patrols or citizen reports, which are then processed through traditional work order systems. This approach has the following significant drawbacks: 1. Delayed detection: Manual patrols have limited coverage, and incident detection relies on chance, making it impossible to achieve 24 / 7 monitoring.

[0003] 2. Inefficiency: From discovery, reporting, assignment to handling, the process is lengthy, communication costs are high, and response speed is slow.

[0004] 3. Reliance on experience: Work order assignment relies heavily on the experience of human dispatchers, making it difficult to achieve optimal resource allocation, especially when multiple events occur concurrently, which can easily lead to uneven distribution.

[0005] 4. Difficulty in closing the loop: The processing results are difficult to verify automatically and require manual confirmation, which can easily lead to a backlog of work orders or false closures.

[0006] Although some existing technologies have developed solutions for event detection using image recognition, most of them only stay at the "recognition" level and fail to form a deep, intelligent closed-loop linkage with the backend "handling resources" and "handling processes," thus failing to build a complete, autonomous decision-making event handling lifecycle management system. Summary of the Invention

[0007] The purpose of this invention is to address the problems existing in current urban operation management, such as delayed event discovery, low efficiency in responsibility division and scheduling, difficulty in coordinating complex events, and poor reliability of closed-loop verification. It provides a method for handling urban operation events with multimodal perception and intelligent agent collaboration, realizing an automated closed loop for the entire process of urban operation events, from discovery, order creation, assignment, handling to verification and model iterative learning.

[0008] This invention is achieved through the following measures: a method for handling urban operation events involving multimodal perception and intelligent agent collaboration, characterized by comprising the following steps: S1. Continuously collect data on the urban operation and management area through multimodal front-end sensing devices deployed in the control area; S2. Generate corresponding confidence scores based on the perceived data and fuse them to obtain fused confidence scores. When the triggering conditions are met, an event work order is automatically created. S3: Generate a structured event ticket containing a unique identifier, event type, location, occurrence time, evidence link, and initial confidence level; the unique identifier is used for tracking the entire lifecycle of the ticket. Event type, used to indicate the corresponding management category; Latitude and longitude coordinates are used to indicate the location where an event occurs or is perceived. The time of occurrence is used for priority calculation and consistency verification. Evidence links are used to associate original or key evidence data at the time of triggering; The initial confidence level is used for subsequent priority evaluation and scheduling calculations.

[0009] S4: Based on a pre-built urban management knowledge graph, the work orders are intelligently classified to determine the responsible parties and guidance on handling methods; S5: A dynamic weighted priority evaluation model is adopted to determine the priority of work orders by comprehensively considering risk, regional sensitivity, time period importance, and historical frequency. S6: Abstract available personnel, vehicles, and equipment into disposal resources, and under response time or service level constraints, use a multi-objective optimization strategy to determine the dynamic matching and allocation scheme between work orders and disposal resources; S7: When a work order meets the conditions for a complex event, the complex event is automatically broken down into multiple sub-work orders with collaborative relationships and dispatched in parallel based on the knowledge graph. S8: Resources are dispatched to the site to perform the necessary actions and upload feedback data such as images / videos / text / location / sensor status, including: Post-treatment on-site images and videos are used to visually record the treatment effect; A written description is used to supplement the explanation of the cause of the on-site abnormality or the key points of handling it; Location and time information are used to verify the consistency between the handling and the location and timing of the event; Sensor status information is used to reflect objective state changes associated with events.

[0010] The processing feedback data is linked to the original work order or sub-work order for subsequent automatic verification and closed-loop determination.

[0011] S9: Automatically verify the handling results based on computer vision and multi-evidence consistency. If it passes, the loop is automatically closed. If it fails, supplementary handling or supplementary evidence is triggered. S10: Archive closed-loop work orders and verification results for continuous optimization and iteration of event identification, priority assessment, and assignment decisions.

[0012] Furthermore, the multimodal front-end sensing devices include: video acquisition devices for collecting real-time or periodic video streams of roads, parking areas, garbage disposal points, and public facility areas; geomagnetic detection devices for collecting data on parking space occupancy status and vehicle dwell time; and Internet of Things (IoT) devices for collecting data on garbage bin overflow status, facility health status, and electronic fence operation status.

[0013] Furthermore, based on the perceived data, corresponding confidence levels are generated and fused to obtain a fused confidence level. When the triggering conditions are met, an event work order is automatically created, including: Visual, geomagnetic, and IoT sensing data were selected for fusion, and the corresponding single-modal confidence scores were as follows: visual confidence score... Geomagnetic confidence level IoT confidence level ; The fusion confidence C is obtained by using a normalized weighted summation fusion method, where:

[0014] Among them, weight , , ≥0, and satisfy: .

[0015] The triggering condition is: If C ≥ θ, the work order will be automatically created; If the event type belongs to a preset high-risk type set (such as fire lane obstruction, major road blockage hazards, etc.), work order creation is allowed even when C < θ, to ensure timely handling of safety-sensitive scenarios. Furthermore, based on a pre-built urban management knowledge graph, the work orders are intelligently classified to determine the responsible parties and guidance on handling methods, including: The entities included in the urban management knowledge graph include event type entities, regional attribute entities, responsible entity entities, handling process entities, and resource type entities; The entities mentioned above are associated with each other using triplet relationships; Based on the event type and location corresponding to the region attribute in the work order, a triplet matching the event type and whose applicable region contains the region attribute is retrieved from the knowledge graph to determine the corresponding responsible entity. Based on the handling process entity associated with the event type entity, a handling method guide for guiding on-site operations is determined.

[0016] Furthermore, a dynamic weighted priority assessment model is adopted, which comprehensively considers risk, regional sensitivity, time period importance, and historical frequency to determine work order priorities, including: Risk levels are preset for different event types, regional sensitivity levels are preset for different regional attributes, and time period importance levels are preset for different time periods. Historical frequency levels are calculated based on the number of times similar events occur in the region within a certain time range. Sub-scores are calculated according to the risk level, regional sensitivity level, time period importance level, and historical frequency level, and the sub-scores are combined according to preset weights to obtain the comprehensive priority score of the work order. Then, based on the correspondence between the comprehensive priority score and the preset threshold range, the work order is divided into at least one of high priority, medium priority, and low priority.

[0017] Furthermore, available personnel, vehicles, and equipment are abstracted as disposal resources. Under response time limits or service level constraints, a multi-objective optimization strategy is adopted to determine the dynamic matching and allocation scheme between work orders and disposal resources, including: Obtain real-time status information of available personnel, vehicles, and equipment in the resource pool, and construct a resource feature set, which includes at least current location, service range, skill suitability, current task load, and historical handling efficiency. Combine the geographical location, event type, and priority of work orders to establish constraints that meet response time or service level requirements. Using a multi-objective optimization algorithm, with objectives such as minimizing travel / time costs, maximizing skill matching, load balancing, and prioritizing urgent work orders as optimization directions, solve for the work order assignment results. The output dynamic matching and assignment scheme includes at least: Each work order includes a corresponding resource identifier, recommended arrival path, or estimated arrival time; and the corresponding response time limit satisfaction judgment result.

[0018] When the status of the disposal resources or the work order changes, a re-solution can be triggered to achieve dynamic scheduling.

[0019] Furthermore, the complex event conditions include: Multiple similar or related events occur together within a preset time window in the same area; The event type belongs to a preset category that requires collaborative handling of multiple resources or across responsibilities; The scope of the event's impact or its risk level has reached a preset threshold; When the conditions for a complex event are met, multiple sub-work orders are automatically generated based on the "event-resource-responsibility" association structure predefined for the event type in the knowledge graph. Collaboration or dependency relationship identifiers are set for the sub-work orders, and then the sub-work orders are dispatched in parallel to different disposal resources according to the dispatch logic of S6.

[0020] Further, S9 includes: The system compares the handling feedback data with the original media data at the time the event was triggered, and calculates visual difference features or change features that reflect the degree of elimination or restoration of the event target. At the same time, based on the time, location and sensor status change information in the handling feedback data, it judges the spatiotemporal consistency and status consistency between the handling behavior and the location of the event, the duration of the event and the status of related facilities / traffic. When the visual comparison result and the consistency judgment result meet the preset verification conditions, the handling is deemed valid and the work order is automatically closed. When the verification conditions are not met or key evidence is lacking, the reason for the verification failure is output and the process of supplementary handling or supplementary evidence upload is triggered.

[0021] Furthermore, the closed-loop work order information, handling feedback data, verification results, and handling timeliness indicators are structured and archived to form a historical database of urban operation events. Based on the actual handling results of different event types, regions, and time periods in the historical database, statistical analysis and parameter updates are performed on event identification strategies, priority assessment parameters, and assignment rules to improve the accuracy of subsequent event identification, the rationality of priority ranking, and the efficiency of resource scheduling. False alarms, missed alarms, and overdue work orders are labeled as samples or used as a basis for rule correction, thereby achieving continuous adaptive optimization in response to changes in the city's operational status.

[0022] The beneficial effects of the technical solution provided by this invention are as follows: Utilizing multiple sensing methods such as video, geomagnetism, and the Internet of Things, and employing weighted fusion confidence to trigger work orders, false alarms and missed alarms are reduced. Knowledge graph-driven responsibility allocation and handling guidance are highly interpretable and easy to maintain. By modeling the relationships between "event type—regional attribute—responsible entity—handling process" through a triplet structure of entity-relationship-entity, a visualized and maintainable responsibility allocation and handling strategy is achieved. A configurable comprehensive priority score is generated by comprehensively considering event risk, regional sensitivity, time period importance, and historical frequency, ensuring that limited resources are prioritized for high-risk, high-impact events. Handling resources are abstracted into "intelligent agents" with location, skills, load, and historical performance. Under response time constraints, multi-objective optimization is used to achieve fast and near-optimal work order assignment. For complex events such as high-frequency clustering events or large-scale event support, they can be decomposed into multiple sub-work orders based on the knowledge graph and assigned to different responsible entities for parallel handling. By comprehensively considering visual restoration, spatiotemporal consistency, and sensor status consistency, the system objectively and quantitatively evaluates the processing results, automatically determining whether a closed-loop system is needed or if supplementary evidence / rework is required. The closed-loop work order and verification results are fed back to optimize the identification strategy, priority parameters, and scheduling strategy, enabling the system performance to continuously improve over time. Attached Figure Description

[0023] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings listed below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a method for handling urban operation events using multimodal perception and intelligent agent collaboration, as described in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0026] See Figure 1 A method for handling urban operation events using multimodal perception and intelligent agent collaboration, characterized by the following steps: S1. Continuously collect data on the urban operation and management area through multimodal front-end sensing devices deployed in the control area; The multimodal front-end sensing devices include: video acquisition devices for collecting real-time or periodic video streams of roads, parking areas, garbage disposal points, and public facility areas; geomagnetic detection devices for collecting data on parking space occupancy status and vehicle dwell time; and IoT devices for collecting data on garbage bin overflow status, facility health status, and electronic fence operation status.

[0027] S2. Generate corresponding confidence scores based on the perceived data and fuse them to obtain a fused confidence score. Automatically create an event work order when the triggering conditions are met; including: Visual, geomagnetic, and IoT sensing data were selected for fusion, and the corresponding single-modal confidence scores were as follows: visual confidence score... Geomagnetic confidence level IoT confidence level ; The fusion confidence C is obtained by using a normalized weighted summation fusion method, where:

[0028] Among them, weight , , ≥0, and satisfy: .

[0029] The triggering condition is: If C ≥ θ, the work order will be automatically created; If the event type belongs to a preset high-risk type set (such as fire lane obstruction, major road blockage hazards, etc.), then work order creation is allowed even when C < θ, to ensure the timely handling of safety-sensitive scenarios. S3: Generate a structured event ticket containing a unique identifier, event type, location, occurrence time, evidence link, and initial confidence level; the unique identifier is used for tracking the entire lifecycle of the ticket. Event type, used to indicate the corresponding management category; Latitude and longitude coordinates are used to indicate the location where an event occurs or is perceived. The time of occurrence is used for priority calculation and consistency verification. Evidence links are used to associate original or key evidence data at the time of triggering; The initial confidence level is used for subsequent priority evaluation and scheduling calculations.

[0030] S4: Based on a pre-built urban management knowledge graph, the work orders are intelligently classified to determine the responsible parties and guidance on handling methods; The work orders are intelligently classified based on a pre-built urban management knowledge graph to determine the responsible party and guidance on handling methods, including: The entities included in the urban management knowledge graph include event type entities, regional attribute entities, responsible entity entities, handling process entities, and resource type entities: The entities mentioned above are associated with each other using triplet relationships; Based on the event type and location corresponding to the region attribute in the work order, a triplet matching the event type and whose applicable region contains the region attribute is retrieved from the knowledge graph to determine the corresponding responsible entity. Based on the handling process entity associated with the event type entity, a handling method guide for guiding on-site operations is determined.

[0031] S5: A dynamic weighted priority evaluation model is adopted to determine the priority of work orders by comprehensively considering risk, regional sensitivity, time period importance, and historical frequency. Risk levels are preset for different event types, regional sensitivity levels are preset for different regional attributes, and time-period importance levels are preset for different time periods. A historical frequency level is calculated based on the number of similar events occurring in the region within a certain time range. Sub-scores are calculated according to the risk level, regional sensitivity level, time-period importance level, and historical frequency level, and these sub-scores are combined according to preset weights to obtain a comprehensive priority score for the work order. Then, based on the correspondence between the comprehensive priority score and a preset threshold range, the work order is classified into at least one of high priority, medium priority, and low priority. The priority assessment model is as follows: .

[0032] In the formula, Characterizes the inherent risk of the event. Characterize the sensitivity of the area (schools, hospitals, key controlled road sections, etc.). Represents the weight of time periods (morning and evening rush hours, holidays, etc.). Characterizing historical frequency, α, β, γ, and δ are weighting parameters. Priority levels are divided based on the interval between the P-value and a preset threshold. For example, the interval of the preset threshold is ( , Then P≥ As a high priority, ≤P< For medium priority, P < It has low priority.

[0033] S6: Abstract available personnel, vehicles, and equipment into disposal resources, and under response time or service level constraints, use a multi-objective optimization strategy to determine the dynamic matching and allocation scheme between work orders and disposal resources; Obtain real-time status information of available personnel, vehicles, and equipment in the disposal resource pool, construct a disposal resource feature set, wherein the features include at least current location, service range, skill suitability, current task load, and historical disposal efficiency. Denote the set of work orders to be assigned as I, and the set of available disposal resources as A. Define decision variables: , indicating whether work order i is assigned to resource a.

[0034] Based on the geographical location, event type, and priority of work orders, constraints are established to meet response time or service level requirements. With the optimization goals of minimizing travel / time costs, maximizing skill matching, load balancing, and prioritizing urgent work orders, a multi-objective optimization algorithm is used to solve for the work order assignment results. The objectives comprehensively consider: estimated arrival time (shorter is better); current resource task load (lower is better); skill matching degree (higher is better); and historical timeout or rework records (fewer is better). Therefore, the objective function is:

[0035] The estimated time for resource a to reach the location of work order i; Characterizes the current task load of resource a; Characterizes the skill suitability of resource a for the event type corresponding to work order i; It can be used to penalize resources with high historical timeout or rework rates.

[0036] The constraints include at least the following: each work order is assigned to only one processing resource, and the number of work orders in progress for each processing resource does not exceed its preset processing capacity. At the same time, the expected arrival time of the processing resource assigned to a high-priority work order does not exceed the response time limit corresponding to that priority level.

[0037] Based on the minimization result of the objective function, the dynamic matching and assignment scheme output by the matching and assignment scheme between work orders and disposal resources includes at least the following: Each work order includes a corresponding resource identifier, recommended arrival path or estimated arrival time, and corresponding response time limit satisfaction judgment result.

[0038] S7: When a work order meets the conditions for a complex event, the complex event is automatically broken down into multiple sub-work orders with collaborative relationships and dispatched in parallel based on the knowledge graph. The complex event conditions include: Multiple similar or related events occur together within a preset time window in the same area; The event type belongs to a preset category that requires collaborative handling of multiple resources or across responsibilities; The scope of the event's impact or its risk level has reached a preset threshold; When the conditions for a complex event are met, multiple sub-work orders are automatically generated based on the "event-resource-responsibility" association structure predefined for the event type in the knowledge graph. Collaboration or dependency relationship identifiers are set for the sub-work orders, and then the sub-work orders are dispatched in parallel to different disposal resources according to the dispatch logic of S6.

[0039] S8: Resources are dispatched to the site to perform the necessary actions and upload feedback data such as images / videos / text / location / sensor status, including: Post-treatment on-site images and videos are used to visually record the treatment effect; A written description is used to supplement the explanation of the cause of the on-site abnormality or the key points of handling it; Location and time information are used to verify the consistency between the handling and the location and timing of the event; Sensor status information is used to reflect objective state changes associated with events.

[0040] The processing feedback data is linked to the original work order or sub-work order for subsequent automatic verification and closed-loop determination.

[0041] S9: Automatically verify the handling results based on computer vision and multi-evidence consistency. If it passes, the loop is automatically closed. If it fails, supplementary handling or supplementary evidence is triggered. The feedback data from the handling process is compared with the original media data at the time the event was triggered to calculate visual difference features or change features reflecting the degree of elimination or restoration of the event target. For events mainly involving specific target entities, such as illegal parking of motor vehicles, disorderly parking of non-motor vehicles, and illegal roadside stalls, the same target detection model as in the event recognition stage is used to detect the images before and after the handling process. By comparing changes in indicators such as the number of targets related to the event category and the total confidence score of target detection before and after the handling process, the degree of restoration of the target detection dimension is assessed to determine whether the event-related targets have disappeared or significantly decreased. For events mainly involving regional states, such as overflowing garbage and piled-up debris, a semantic segmentation model is used to segment the garbage or debris areas in the images before and after the handling process. By comparing changes in indicators such as the area size and coverage ratio of the segmented areas, the degree of restoration of the environmental state from overflowing or piled-up to a basically clean state is assessed.

[0042] Using the area where the event occurred as the region of interest, the similarity in structure and texture between the scene image before and after processing is calculated, and the similarity is used as an auxiliary reference index for scene restoration. Meanwhile, based on the time, location, and sensor status change information in the handling feedback data, the system judges the spatiotemporal consistency and status consistency between the handling behavior and the location of the event, the duration of the event, and the status of related facilities / traffic. When the visual comparison result and the consistency judgment result meet the preset verification conditions, the handling is deemed valid and the work order is automatically closed. When the verification conditions are not met or key evidence is lacking, the system outputs the reason for the verification failure and triggers the supplementary handling or supplementary evidence upload process.

[0043] S10: Archive closed-loop work orders and verification results for continuous optimization and iteration of event identification, priority assessment, and dispatch decisions. This includes: structurally archiving closed-loop work order information, handling feedback data, verification results, and handling timeliness indicators to form a historical database of urban operation events. Based on the actual handling results of different event types, regions, and time periods in the historical database, perform statistical analysis and parameter updates on event identification strategies, priority assessment parameters, and dispatch rules to improve the accuracy of subsequent event identification, the rationality of priority ranking, and the efficiency of resource scheduling; and create labeled samples or rule correction basis for false alarms, missed alarms, and overdue work orders, thereby achieving continuous adaptive optimization in response to changes in urban operation status.

[0044] Example 2: Based on Example 1: Event type entities (e.g., illegal parking of motor vehicles, disorderly parking of non-motorized vehicles, overflowing garbage, damaged facilities, security for large-scale events, etc.); Regional attribute entities (e.g., commercial districts, residential areas, areas surrounding schools and hospitals, scenic spots, main roads, secondary roads, etc.); The responsible entity (e.g., Urban Management Team A, Traffic Police Brigade B, Sanitation Company A, Property Management Company C, etc.); The physical aspects of the handling process (e.g., standard waste removal process B, illegal parking notification process C, facility emergency repair process D, comprehensive support process for large-scale events E, etc.). Resource type entities (e.g., cleaning staff, patrol staff, garbage trucks, emergency repair vehicles, etc.).

[0045] The entities mentioned above are associated through triples, and examples of triples include, but are not limited to: (Event type: Garbage overflow, Applicable area: Commercial district, Relationship: Relationship between applicable areas) (Event type: garbage overflow, responsible party: sanitation company A, relationship: primary responsibility relationship); (Event type: Garbage overflow, Disposal process: Standard collection and disposal process B, Relationship: Disposal process relationship). (Regional attribute: commercial district; superior management unit: First Brigade of Urban Management Bureau; relationship: subordinate management relationship) (Event type: illegal parking of motor vehicles; responsible party: Traffic Police Brigade B; relationship: primary responsibility). (Event type: illegal parking of motor vehicles; responsible party: Urban Management Team A; relationship: collaborative responsibility relationship). (Event type: Large-scale event security, handling process: Comprehensive security process for large-scale events E, relationship: handling process relationship).

[0046] In actual runtime, when the generated structured work order contains event type (Type) and location... At that time, the system first determines the location. Query the corresponding regional attribute entity ,For example: If the POI corresponding to the work order location is marked as "commercial district", then the regional attribute entity is obtained. =Commercial district. Based on this, the system uses knowledge graph queries or a rule engine to execute predefined rules to determine the responsible party and the appropriate handling methods, and then performs responsibility matching: , The function for determining the responsible party based on knowledge graph relation reasoning or rule matching can be implemented using the following logic: 1. Regional-level rule matching: Query for triples (Type, applicable region relationship, The record of the event type is used to obtain the set of applicable configurations for the current region attribute. 2. Rules for determining the responsible party If a triple (Type, primary responsibility relationship, responsible entity) exists that is compatible with the applicable area relationship matched in the previous step, then the "responsible entity" will be given priority as the responsible unit for this work order; If only a collaborative responsibility relationship exists, the regional management unit can be identified as the responsible party by tracing back the regional attribute relationship. In situations where there are both primary and secondary responsibilities, the entity corresponding to the primary responsibility relationship can be designated as the primary responsible unit, and the entity corresponding to the secondary responsibility relationship can be designated as the secondary responsible unit.

[0047] 3. Rules for determining the handling guidelines Query the triple (Type, handling process relationship, handling process Y) to obtain the handling process entity associated with this event type; If there is a further mapping between the handling process and the regional attributes (e.g., different regions use different versions of the process), then the triplet can be matched (handling process Y, applicable region relationship, ...). To determine the final version of the disposal process to be used.

[0048] Taking "garbage overflow + commercial district" as a specific example, when the work order event type is "garbage overflow" and the area attribute is "commercial district", the knowledge graph contains the following triples: (Garbage overflow, applicable area relationship: commercial district); (Garbage overflow, primary responsibility relationship: sanitation company A); (Garbage overflow, disposal process relationship: standard collection and transportation process B).

[0049] but The reasoning result for (Type = overflowing garbage, Zone = commercial district) is: Responsible entity: Sanitation Company A; Disposal guidelines: Standard waste removal procedure B.

[0050] Based on this, the system sets the work order responsibility subject field to "Sanitation Company A" and sends the corresponding operation steps or instructions of the standard cleaning process B to the subsequent disposal terminal as a process guide for on-site operations.

[0051] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for handling urban operation events using multimodal perception and intelligent agent collaboration, characterized in that, Includes the following steps: S1. Continuously collect data on the urban operation and management area through multimodal front-end sensing devices deployed in the control area; S2. Generate corresponding confidence scores based on the perceived data and fuse them to obtain fused confidence scores. When the triggering conditions are met, an event work order is automatically created. S3: Generate a structured event ticket containing a unique identifier, event type, location, occurrence time, evidence link, and initial confidence level; S4: Based on a pre-built urban management knowledge graph, the work orders are intelligently classified to determine the responsible parties and guidance on handling methods; S5: A dynamic weighted priority evaluation model is adopted to determine the priority of work orders by comprehensively considering risk, regional sensitivity, time period importance, and historical frequency. S6: Abstract available personnel, vehicles, and equipment into disposal resources, and under response time or service level constraints, use a multi-objective optimization strategy to determine the dynamic matching and allocation scheme between work orders and disposal resources; S7: When a work order meets the conditions for a complex event, the complex event is automatically broken down into multiple sub-work orders with collaborative relationships and dispatched in parallel based on the knowledge graph. S8: Resources are dispatched to the scene to carry out the disposal and upload disposal feedback data such as images / videos / text / location / sensor status; S9: Automatically verify the handling results based on computer vision and multi-evidence consistency. If it passes, the loop is automatically closed. If it fails, supplementary handling or supplementary evidence is triggered. S10: Archive closed-loop work orders and verification results for continuous optimization and iteration of event identification, priority assessment, and assignment decisions.

2. The urban operation incident handling method according to claim 1, characterized in that, Multimodal front-end sensing devices include: video acquisition devices for collecting real-time or periodic video streams of roads, parking areas, garbage disposal points, and public facility areas; geomagnetic detection devices for collecting data on parking space occupancy status and vehicle dwell time; and IoT devices for collecting data on garbage bin overflow status, facility health status, and electronic fence operation status.

3. The urban operation incident handling method according to claim 2, characterized in that, Based on the perceived data, corresponding confidence levels are generated and fused to obtain a fused confidence level. When the triggering conditions are met, an event work order is automatically created, including: Visual, geomagnetic, and IoT sensing data were selected for fusion, and the corresponding single-modal confidence scores were as follows: visual confidence score... Geomagnetic confidence level IoT confidence level ; The fusion confidence C is obtained by using a normalized weighted summation fusion method, where: Among them, weight , , ≥0, and satisfy: ; The triggering condition is: If C≥θ, the work order will be automatically created; If the event type belongs to the preset high-risk type set, the creation of the work order is allowed to be triggered even when C < θ, so as to ensure the timely handling of security-sensitive scenarios.

4. The urban operation incident handling method according to claim 2, characterized in that, The work orders are intelligently classified based on a pre-built urban management knowledge graph to determine the responsible party and guidance on handling methods, including: The entities included in the urban management knowledge graph include event type entities, regional attribute entities, responsible entity entities, handling process entities, and resource type entities; The entities mentioned above are associated with each other using triplet relationships; Based on the event type and location corresponding to the region attribute in the work order, a triplet matching the event type and whose applicable region contains the region attribute is retrieved from the knowledge graph to determine the corresponding responsible entity. Based on the handling process entity associated with the event type entity, a handling method guide for guiding on-site operations is determined.

5. The urban operation incident handling method according to claim 1, characterized in that, A dynamic weighted priority assessment model is used to determine work order priorities by comprehensively considering risk, regional sensitivity, time period importance, and historical frequency, including: Risk levels are preset for different event types, regional sensitivity levels are preset for different regional attributes, and time period importance levels are preset for different time periods. Historical frequency levels are calculated based on the number of times similar events occur in the region within a certain time range. Sub-scores are calculated according to the risk level, regional sensitivity level, time period importance level, and historical frequency level, and the sub-scores are combined according to preset weights to obtain the comprehensive priority score of the work order. Then, based on the correspondence between the comprehensive priority score and the preset threshold range, the work order is divided into at least one of high priority, medium priority, and low priority.

6. The urban operation incident handling method according to claim 5, characterized in that, Available personnel, vehicles, and equipment are abstracted as disposal resources. Under response time limits or service level constraints, a multi-objective optimization strategy is adopted to determine the dynamic matching and allocation scheme between work orders and disposal resources, including: Obtain real-time status information of available personnel, vehicles, and equipment in the resource pool, and construct a resource feature set, which includes at least current location, service range, skill suitability, current task load, and historical handling efficiency. Combine the geographical location, event type, and priority of work orders to establish constraints that meet response time or service level requirements. Using a multi-objective optimization algorithm, with objectives such as minimizing travel / time costs, maximizing skill matching, load balancing, and prioritizing urgent work orders as optimization directions, solve for the work order assignment results. The output dynamic matching and assignment scheme includes at least: Each work order includes a corresponding resource identifier, recommended arrival path or estimated arrival time, and corresponding response time limit satisfaction judgment result.

7. The urban operation incident handling method according to claim 5, characterized in that, The complex event conditions include: Multiple similar or related events occur together within a preset time window in the same area; The event type belongs to a preset category that requires collaborative handling of multiple resources or across responsibilities; The scope of the event's impact or its risk level has reached a preset threshold; When the conditions for a complex event are met, multiple sub-work orders are automatically generated based on the "event-resource-responsibility" association structure predefined for the event type in the knowledge graph. Collaboration or dependency relationship identifiers are set for the sub-work orders, and then the sub-work orders are dispatched in parallel to different disposal resources according to the dispatch logic of S6.

8. The method for handling urban operation incidents according to claim 5, characterized in that, The system automatically verifies the handling results based on computer vision and multi-evidence consistency. If the verification passes, the process is automatically closed; otherwise, supplementary handling or supplementary evidence is triggered, including: The system compares the handling feedback data with the original media data at the time the event was triggered, and calculates visual difference features or change features that reflect the degree of elimination or restoration of the event target. At the same time, based on the time, location and sensor status change information in the handling feedback data, it judges the spatiotemporal consistency and status consistency between the handling behavior and the location of the event, the duration of the event and the status of related facilities / traffic. When the visual comparison result and the consistency judgment result meet the preset verification conditions, the handling is deemed valid and the work order is automatically closed. When the verification conditions are not met or key evidence is lacking, the reason for the verification failure is output and the process of supplementary handling or supplementary evidence upload is triggered.

9. The method for handling urban operation incidents according to claim 8, characterized in that, Archive closed-loop work orders and verification results for continuous optimization and iteration of event identification, priority assessment, and assignment decisions, including: The closed-loop work order information, handling feedback data, verification results, and handling timeliness indicators are structured and archived to form a historical database of urban operation events. Based on the actual handling results of different event types, regions, and time periods in the historical database, statistical analysis and parameter updates are performed on event identification strategies, priority evaluation parameters, and assignment rules to improve the accuracy of subsequent event identification, the rationality of priority ranking, and the efficiency of resource scheduling. False alarms, missed alarms, and overdue work orders are marked as samples or used as a basis for rule correction, thereby achieving continuous adaptive optimization in response to changes in the urban operation status.

Images