Urban environment supervision method and system based on AI video algorithm

Abstract

The invention discloses an urban environment supervision method and system based on an AI video algorithm. The method comprises the following steps: (1) AI identification: acquiring urban environment related data through a multi-modal data acquisition device and performing real-time illegal behavior identification; (2) automatic order sending: pushing an identification result to a cloud management platform to generate a task; (3) processing feedback: intelligently sending an order according to an identification result; and (4) dynamically scoring. According to the method, the violation identification precision in a complex scene is improved through multi-modal data fusion, and efficient identification, automatic order sending, closed-loop processing and dynamic scoring of urban violation behaviors are realized by utilizing an edge computing and cloud collaborative computing framework; the problems that in traditional city management, efficiency is low, cooperation is difficult, and decision-making lacks data support are effectively solved, the recognition precision in a complex scene is improved, the event response time is shortened, data-driven fine management is achieved, and data support is provided for city management decision-making.

Description

Technical Field

[0001] The present invention relates to the technical field of urban environmental supervision, and in particular to an urban environmental supervision method and system based on AI video algorithm. Background Art

[0002] With the acceleration of urbanization, urban environmental management faces enormous challenges. Traditional urban management models rely primarily on manual inspections, which are inefficient, costly, and slow to respond, making them inadequate for modern urban management. For example, violations such as street occupation and littering are often difficult to detect promptly, are prone to omissions in manual records, and lack systematic support for follow-up rectification.

[0003] In recent years, although intelligent monitoring and AI recognition technologies have been gradually applied to urban governance, existing solutions are mostly limited to a single function, such as only realizing violation identification or data recording, and failing to form a closed-loop management system of "identification-dispatching-processing-assessment", resulting in fragmented management and low collaborative efficiency. Summary of the Invention

[0004] The purpose of the present invention is to provide an urban environment monitoring method and system based on AI video algorithm that can solve the above-mentioned shortcomings.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions:

[0006] A method for urban environmental supervision based on AI video algorithm, comprising the following steps:

[0007] (1) AI Identification: Collecting urban environment-related data through multimodal data acquisition equipment, which includes at least a high-definition camera, an infrared sensor, and a GPS positioning device; using a deep learning model deployed on an edge computing device to perform real-time violation identification on the collected data;

[0008] (2) Automatic dispatch: push the recognition results to the cloud management platform to generate tasks;

[0009] (3) Processing feedback: Based on the recognition results, the task is assigned to the corresponding processing personnel through the intelligent dispatching algorithm, and the task processing process is tracked and managed with feedback;

[0010] (4) Dynamic scoring: The score is dynamically calculated based on the violation type and time decay factor.

[0011] As a preferred solution of the present invention, the deep learning model is a lightweight YOLOv5 model, which is used to achieve low-latency violation identification with an identification delay of ≤3 seconds.

[0012] As a preferred solution of the present invention, the dynamic calculation score is achieved by the following formula:

[0013] Score = ∑(Wi × 11 + α·t)

[0014] Among them, Wi is the violation type weight, α is the time decay factor, and t is the dynamic adjustment scoring period.

[0015] As a preferred solution of the present invention, the cloud management platform includes a video management module, an event management module and a dynamic scoring module.

[0016] As a preferred solution of the present invention, the cloud management platform also includes a broadcast scheduling module, which is used to combine AI analysis data and manual review results to provide remote voice reminders and warnings to illegal merchants.

[0017] As a preferred solution of the present invention, the intelligent dispatching algorithm is based on GIS maps and the real-time location of operation and maintenance personnel to achieve intelligent allocation of tasks, with a response time of ≤5 minutes and a timeout warning function. When a task is not processed within 30 minutes, the task is automatically upgraded and the superior administrator is notified.

[0018] As a preferred solution of the present invention, a red and black list is generated based on the dynamically calculated scoring results and pushed to the merchant mini program and management background.

[0019] A system for implementing the above-mentioned urban environment supervision method based on AI video algorithm, comprising:

[0020] Device layer: used for multimodal data collection and transmission;

[0021] AI analysis layer: used for real-time video analysis and violation identification;

[0022] Platform layer: used for task management, dynamic scoring and data storage;

[0023] Application layer: Provides an interactive interface between the operation and maintenance personnel App and the management backend.

[0024] As a preferred solution of the present invention, the AI ​​analysis layer uses a deep learning model to intelligently identify littering, road occupation and related violations, and pushes the analysis results to the platform layer through HTTP or MQTT protocol.

[0025] As a preferred solution of the present invention, it also includes an external service layer, which integrates maps, notifications, streaming media and related third-party services to provide the system with corresponding functional support for location display, message push and video streaming transmission.

[0026] The beneficial effects of the present invention are: the present invention improves the accuracy of violation identification in complex scenarios through multimodal data fusion, and uses edge computing and cloud collaborative computing framework to achieve efficient identification, automatic dispatching, closed-loop processing and dynamic scoring of urban violations. It not only effectively solves the problems of low efficiency, coordination difficulties and lack of data support for decision-making in traditional urban management, but also improves the recognition accuracy in complex scenarios, shortens the event response time, and realizes data-driven refined management, providing data support for urban management decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the system architecture of the present invention.

[0028] Figure 2 Schematic diagram of the method of the present invention. DETAILED DESCRIPTION

[0029] Example: See Figures 1 to 2 , an embodiment of the present invention provides an urban environment monitoring method and system based on AI video algorithm.

[0030] The system adopts a layered architecture, divided into the device layer, AI analysis layer, platform layer, application layer, and external service layer. Data exchange between each layer is carried out through standard protocols (such as RTSP, HTTP, WebSocket, and MQTT), ensuring the modularity and scalability of the system.

[0031] The device layer is the foundation of the system and is used for multimodal data acquisition and transmission. It is mainly responsible for the acquisition and transmission of video data. It mainly includes cameras and NVRs (network video recorders). The cameras transmit real-time video streams to the NVRs through the RTSP or RTMP protocols, and the NVRs are responsible for the storage and distribution of video streams.

[0032] The camera is a high-definition camera that captures 1080P video streams and supports RTSP / RTMP transmission protocols. It is also equipped with an infrared sensor and GPS positioning device. The infrared sensor assists in object detection at night or in obstructed scenarios (such as roadside businesses). The GPS positioning device marks the location of events for intelligent dispatching and heat map generation.

[0033] It is preferred to deploy a lightweight YOLOv5 model in cameras or NVR devices to achieve low-latency (≤3 seconds) violation identification. At the same time, multimodal data fusion combined with infrared data enhances nighttime detection accuracy (false alarm rate reduced to less than 5%).

[0034] A system for implementing the above-mentioned urban environment supervision method based on AI video algorithm, comprising:

[0035] The AI ​​analysis layer is the core of the system, used for real-time video analysis and violation identification. By obtaining video streams from the NVR, deep learning models (such as YOLO and Faster R-CNN) are used to intelligently identify events such as littering and road occupation, and the analysis results are pushed to the platform layer via HTTP or MQTT protocols.

[0036] The platform layer is the core of the system, used for task management, dynamic scoring and data storage. Specifically, it is responsible for event management, task distribution, data storage and video stream management. It supports event classification, priority sorting and task allocation, and realizes the viewing and management of real-time video streams through streaming media services (such as FFmpeg and Nginx-RTMP).

[0037] The application layer provides an interactive interface between the operations and maintenance personnel app and the management backend. Specifically, it includes the operations and maintenance personnel app, the urban management backend, and the public participation app (optional), responsible for task processing, system management, and event reporting, respectively. The operations and maintenance personnel app receives real-time task push notifications via WebSocket, supporting voice reminders and navigation to the incident location. Uploading processing results (photos and text) triggers a secondary AI review.

[0038] Preferably, the system may further include an external service layer, which integrates maps, notifications, streaming media and related third-party services to provide the system with corresponding functional support for location display, message push and video streaming.

[0039] The system of the present invention realizes a high-cohesion, low-coupling architecture through layered design and standardized protocols, which facilitates subsequent function expansion and maintenance.

[0040] The urban environment supervision method of the system of the present invention is as follows:

[0041] (1) AI recognition: Collect urban environment-related data through multimodal data acquisition equipment, which includes at least high-definition cameras, infrared sensors, and GPS positioning devices; multimodal fusion can improve recognition accuracy, reduce the false alarm rate of night scenes by 40%, and increase the recognition accuracy of occluded scenes to 92%. And shorten the response time through closed-loop management: the time from event discovery to processing feedback is ≤10 minutes (traditional mode requires more than 2 hours). Use the deep learning model deployed on the edge computing device to perform real-time violation identification on the collected data; AI recognition is mainly based on artificial intelligence technology to achieve video analysis, the specific contents are as follows:

[0042] a. Real-time video analysis: Real-time information mainly displays the number of online devices today (total number of devices, number of online devices, number of offline devices), the latest events, AI recognition statistics, confirmed violation statistics, event reporting platform statistics, review rate analysis, event location ranking, number of effective recognitions / number of events, event time period ranking, event type ratio, etc.

[0043] b. Data retrieval: All violation events are retrieved based on the location information, the time of automatic violation identification (today, the past three days, the past seven days), the automatic event identification type, and the event review status, and all events of interest to the user are screened;

[0044] c. Review and Processing: To ensure the reliability of reporting violations to the business platform, reduce invalid incident reports, and reduce the processing pressure on the business platform, manual review and processing of the algorithm recognition results is required;

[0045] d. Algorithm Service: Based on the Urban Management Violation Intelligent Analysis Component, this service utilizes high-performance GPU servers and deep learning algorithms to intelligently analyze and process high-definition video resources. It can analyze and alert on violations related to urban appearance and order, and continuously conduct machine learning and optimization based on the footage. Identification events include: illegal umbrella use, operating outside a store, operating unlicensed peddlers, illegal outdoor advertising, occupying the road, random stacking of materials, drying on the street, packaged garbage, exposed garbage, overflowing trash cans, illegal non-motorized vehicle parking, illegal motor vehicle parking, and helmet recognition. The algorithm is continuously updated to expand the supported violation types.

[0046] In general, by fusing video streams, infrared sensors, and GPS positioning data through multimodal data, and dynamically adjusting the recognition threshold through adaptive algorithms (such as reducing light sensitivity at night), the recognition accuracy in complex scenarios (occlusion, low light) is improved.

[0047] (2) Automatic dispatch: Push the recognition results to the cloud management platform to generate tasks; the cloud management platform includes a video management module, an event management module and a dynamic scoring module.

[0048] The event management module manually reviews alarm events through video recognition to reduce invalid event reports, and pushes, queries, handles, and archives events. Specifically, it includes the following:

[0049] a. Event push: Review the reported incidents and push them if they are confirmed to be violations. They can be assigned to field law enforcement personnel for handling;

[0050] b. Event retrieval: Search and query violation events by day, week, time, monitoring point, event type, etc.

[0051] c. Incident handling: Broadcast reminders to offending stores, requiring immediate rectification. If the store fails to rectify within the specified time, the task will be assigned to the field law enforcement team to complete the handling within 2 hours;

[0052] d. Event Archiving: Events are automatically archived, and the archived information includes the time the event was filed, the time it was handled, the results of the handling, etc. This provides data basis for subsequent "Red and Black List" evaluation.

[0053] Preferably, the cloud management platform also includes a broadcast scheduling module, which combines AI analysis data with manual review results to provide remote voice reminders and warnings to merchants violating regulations. When businesses or citizens violate regulations, audio transmission technology is used to provide remote voice reminders and warnings to merchants along the street, issuing instant voice reminders to the violating merchants and requiring them to make immediate corrections.

[0054] (3) Processing feedback: Based on the identification results, the task is assigned to the corresponding processing personnel through the intelligent dispatching algorithm, and the task processing process is tracked and managed. Specifically, based on multiple dimensions such as time, location, and type, statistical analysis is conducted on illegal incidents such as illegal parking of non-motor vehicles, illegal dumping of garbage, occupying the road for business, littering, illegal placement of materials, and damaged sidewalk facilities. Event reports are generated and provided in a variety of intuitive ways such as graphs, tables, and curves. Equipment problem data such as equipment online status and fault status are analyzed and an analysis report is issued to facilitate the understanding of the prominent types of equipment problems and problem-prone areas. Intelligent dispatching: Based on GIS maps, tasks are assigned to operation and maintenance personnel nearby, with a response time of ≤5 minutes. It also has a timeout warning function. When a task is not processed within 30 minutes, the task is automatically upgraded and the superior administrator is notified. The intelligent dispatching algorithm based on GIS maps and the real-time location of operation and maintenance personnel can combine timeout warning (automatic upgrade of tasks if not processed within 30 minutes) with dynamic priority adjustment (such as garbage overflow is a high priority). The dispatching algorithm logic is a nearby allocation strategy and a timeout warning mechanism.

[0055] (4) Dynamic scoring: The score is dynamically calculated based on the violation type and the time decay factor. The dynamic scoring is achieved by the following formula:

[0056] Score = ∑(Wi × 11 + α·t)

[0057] Where Wi is the violation type weight, α is the time decay factor, and t is the dynamic adjustment scoring period. Custom weights (e.g., road occupation weight > illegal material stacking) and dynamic adjustment of the scoring period (day / month / year) are supported.

[0058] Based on dynamically calculated scoring results, a red and black list is publicly displayed. For example, a list can be automatically generated based on monthly scores and pushed to the merchant mini-program and management backend. Through objective and quantitative management through a dynamic scoring model, the frequency of merchant score updates can be increased from monthly to minute-by-minute, supporting real-time reward and punishment decisions.

[0059] For example, when business operations occupy the road, the handling method is as follows: the road occupation business events are collected through the camera in the multimodal data collection device and pushed to the cloud management platform; the task dispatch module assigns it to the nearest operation and maintenance personnel; the operation and maintenance personnel arrive at the scene within 10 minutes to handle and provide feedback; the dynamic scoring module updates the merchant's points deduction, and the red and black lists are refreshed in real time.

[0060] If garbage overflows at night and causes merchants to be deducted points, the handling method is as follows: use the cameras and infrared sensors in the multimodal data acquisition equipment to assist in identifying nighttime garbage accumulation events and push them to the cloud management platform; trigger high-priority tasks and notify the nearest cleaning team; after the cleaning team completes the processing, the dynamic scoring module automatically restores the merchant's points, and the red and black lists are refreshed in real time.

[0061] In summary, the system of the present invention adopts a closed-loop management architecture and a real-time data intercommunication mechanism to construct a full-chain closed-loop architecture of "AI identification → automatic dispatch → processing feedback → dynamic scoring", and realizes real-time data interaction of multiple modules (monitoring, tasks, scoring) through a standardized API gateway, thereby realizing efficient identification, automatic dispatch, closed-loop processing and dynamic scoring of urban violations. It not only effectively solves the problems of inefficiency, coordination difficulties and lack of data support for decision-making in traditional urban management, but also improves the recognition accuracy in complex scenarios, shortens the event response time, and realizes data-driven refined management, providing data support for urban management decisions.

[0062] Based on the disclosure and teachings of the above description, those skilled in the art to which the present invention belongs may also make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the scope of protection of the claims of the present invention. In addition, although some specific terms are used in this description, these terms are only for convenience of description and do not constitute any limitation to the present invention. As described in the above embodiments of the present invention, the use of the same or similar methods and systems is within the scope of protection of the present invention.

Claims

1. A method for urban environmental supervision based on AI video algorithm, characterized in that: It includes the following steps: (1) AI Identification: Collecting urban environment-related data through multimodal data acquisition equipment, which includes at least a high-definition camera, an infrared sensor, and a GPS positioning device; using a deep learning model deployed on an edge computing device to perform real-time violation identification on the collected data; (2) Automatic dispatch: push the recognition results to the cloud management platform to generate tasks; (3) Processing feedback: Based on the recognition results, the task is assigned to the corresponding processing personnel through the intelligent dispatching algorithm, and the task processing process is tracked and managed with feedback; (4) Dynamic scoring: The score is dynamically calculated based on the violation type and time decay factor.

2. The urban environment monitoring method based on AI video algorithm according to claim 1 is characterized in that: The deep learning model is a lightweight YOLOv5 model, which is used to achieve low-latency violation identification with an identification delay of ≤3 seconds.

3. The urban environment monitoring method based on AI video algorithm according to claim 2 is characterized in that: The dynamic calculation score is achieved by the following formula: Score = ∑(Wi × 11 + α·t) Among them, Wi is the violation type weight, α is the time decay factor, and t is the dynamic adjustment scoring period.

4. The urban environment monitoring method based on AI video algorithm according to claim 1 is characterized in that: The cloud management platform includes a video management module, an event management module and a dynamic scoring module.

5. The urban environment monitoring method based on AI video algorithm according to claim 4 is characterized in that: The cloud management platform also includes a broadcast scheduling module, which is used to combine AI analysis data and manual review results to provide remote voice reminders and warnings to merchants who violate regulations.

6. The urban environment monitoring method based on AI video algorithm according to claim 1 is characterized in that: The intelligent dispatching algorithm is based on GIS maps and the real-time location of operation and maintenance personnel to achieve intelligent task allocation, with a response time of ≤5 minutes and a timeout warning function. If a task is not processed within 30 minutes, it will automatically escalate the task and notify the superior administrator.

7. The urban environment monitoring method based on AI video algorithm according to any one of claims 1 to 6, characterized in that: A red and black list is generated based on the dynamically calculated scoring results and pushed to the merchant mini program and management backend.

8. A system for implementing the urban environment monitoring method based on AI video algorithm according to any one of claims 1 to 7, characterized in that: It includes: Device layer: used for multimodal data collection and transmission; AI analysis layer: used for real-time video analysis and violation identification; Platform layer: used for task management, dynamic scoring and data storage; Application layer: Provides an interactive interface between the operation and maintenance personnel App and the management backend.

9. The system according to claim 8, characterized in that The AI ​​analysis layer uses deep learning models to intelligently identify littering, road occupation and related violations, and pushes the analysis results to the platform layer via HTTP or MQTT protocol.

10. The system according to claim 8, wherein: It also includes an external service layer, which integrates maps, notifications, streaming media and related third-party services to provide the system with corresponding functional support for location display, message push and video streaming transmission.

Images