Visual network AI calculation method and system based on SIP signaling and video gateway

Abstract

The invention provides a visual network AI calculation method and system based on SIP signaling and a video gateway. The method comprises the following steps: acquiring a real-time video stream address of camera equipment; accessing and processing a video stream by using a video gateway; carrying out algorithm processing on the frame image by using an AI analyzer; according to an analysis result of the AI analyzer, under the condition that the event meets a preset condition, the event data are stored in a receiving center, and the event is pushed to a specified subordinate system by utilizing a pushing center; under the condition that the number of camera devices or the concurrency of video streams is increased, dynamically expanding service nodes of a video gateway and an AI analyzer; and under the condition of equipment failure or network abnormity, a failover mechanism is automatically triggered. Dynamic expansion of the video gateway and the AI resolver is realized through a containerization technology, and high availability and stability of the system can be ensured to be kept even under the condition of high concurrency in combination with a load balancing strategy.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance and visual computing technology, and in particular to a visual network AI computing method and system based on SIP signaling and video gateway. Background Art

[0002] In the current video surveillance field, video surveillance systems based on SIP signaling and GB28181 specifications are widely used in scenarios such as public safety, traffic management, and urban monitoring. Existing video surveillance platforms are usually composed of video management services, signaling services, and streaming services. SIP signaling is used to achieve camera device registration, real-time video playback, and historical video playback. This platform structure works well at a certain scale, but with the increase in the number of surveillance devices and the complexity of application scenarios, existing technologies face many bottlenecks and challenges.

[0003] First, in terms of device access capabilities, the signaling service in the existing system has strict concurrent access restrictions on camera devices. Due to the limited number of ports on a signaling server, a single node can usually only support concurrent interactions of about 2,000 devices. When the number of devices connected to the monitoring platform grows to a certain scale, the access capability of the signaling service becomes a bottleneck of the system, limiting the scalability of the system.

[0004] Secondly, the bottleneck of network IO is also one of the main problems faced by existing video platforms. With the increase of camera devices, the amount of video stream data has increased dramatically, and the network IO consumption of a single video stream is usually around 3M. When a large number of video on demand operations are performed simultaneously in the system, the bandwidth consumption of the computer room will quickly reach the upper limit. For example, under the condition of 100GE computer room bandwidth, the maximum concurrency is about 30,000 devices. Once this limit is exceeded, the system will not be able to process more video stream data, resulting in decreased network performance and interruption of video transmission.

[0005] In addition, video service latency is also a major issue in the existing system. Since the existing video platform uses a centralized processing architecture, the video stream data of all camera devices must be transmitted to the central computer room for processing. However, the deployment locations of camera devices are often distributed in different areas. Especially in cross-regional and cross-city situations, camera devices far away from the central computer room need to go through longer network links, resulting in increased latency in video stream transmission. This latency not only affects the real-time performance of the video, but may also cause freezes and loss of video images.

[0006] Finally, there is a great risk to system stability. As existing video platforms are usually deployed in a single computer room, when the computer room or related services fail, the entire video platform's business will be affected, and may even lead to the paralysis of the platform. The single cluster deployment method lacks sufficient redundancy and fault tolerance mechanisms, making it difficult to cope with sudden failures in large-scale applications, affecting system reliability and business continuity.

[0007] The existence of these problems seriously restricts the performance of existing video surveillance platforms in large-scale, cross-regional applications. Insufficient device access capabilities, network IO bottlenecks, video service delays and system stability issues may cause the video surveillance system to be unable to meet actual business needs, affecting the normal operation of key areas such as public safety and traffic management. Therefore, a technical solution that can effectively solve these problems is needed to improve the scalability, real-time performance and reliability of the system and adapt to increasingly complex and diverse application scenarios. Summary of the invention

[0008] In order to overcome the shortcomings of the prior art, the present invention proposes a visual network AI computing method and system based on SIP signaling and video gateway. By integrating advanced SIP signaling, video processing, AI parsing and distributed storage technologies, an efficient, intelligent and stable visual network AI computing platform is constructed, which can significantly improve the efficiency of video monitoring and analysis, and provide reliable technical support for large-scale application scenarios.

[0009] To achieve the above object, the present invention provides a visual network AI computing method based on SIP signaling and video gateway, the method comprising: Step 1: Obtain the real-time video stream addresses of multiple camera devices. The video stream addresses are obtained from the camera devices through the SIP signaling service and registered to the signaling server. The signaling server completes the mapping and storage of the video stream addresses according to the national standard coding of the camera devices; Step 2: Access and process the video stream using a video gateway, wherein the processing includes intercepting frame images of the video stream according to a predetermined frame interception frequency, wherein the frame interception frequency is configured according to specific application requirements, and the intercepted frame images are pushed to the AI ​​parser through the video gateway; Step 3: Using an AI parser to perform algorithm processing on the frame image, the algorithm includes a preset deep learning model, and the model dynamically selects the most suitable algorithm module according to the location and application scenario of the camera device. The algorithm module analyzes the frame image to identify a specific target or event, and pushes the analysis results to the receiving center and the push center; Step 4: According to the analysis results of the AI ​​parser, if the event meets the preset conditions, the event data is stored in the receiving center, and the event is pushed to the specified lower-level system or user interface by the push center. The push process is completed through the Gateway service of the edge node to ensure the timeliness and reliability of event push; Step 5: When the number of camera devices or the concurrent video stream volume increases, dynamically expand the service nodes of the video gateway and AI parser. The expansion process is implemented through containerized deployment and load balancing strategy to ensure high availability and processing performance of the system. Step 6: In the event of device failure or network anomaly, the failover mechanism is automatically triggered to reallocate camera devices to other normally operating video gateways and AI parser nodes, and fault tracking and recovery are performed through logging and monitoring systems to ensure system stability and continuity.

[0010] Furthermore, obtaining the real-time video stream addresses of multiple camera devices includes: a) When a camera device is connected to the system, a unique national standard code is generated for each camera device through the SIP signaling service. The national standard code includes the geographical location, device type and other identification information of the device; b) In the signaling server, the physical address of the camera device is mapped to the corresponding video stream address using the national standard code, and the video stream address includes a URL link under the RTSP or RTMP protocol; c) When the national standard code of the camera device changes, the mapping relationship in the signaling server is dynamically updated, and the relevant service nodes are broadcast and notified within the system to ensure that the address mapping of the video stream is accurate.

[0011] Furthermore, in step 2, the video stream is processed by using a video gateway, including: a) accessing the real-time video stream obtained from the signaling server in the video gateway, and intercepting the frame image of the video stream according to the preset frame interception frequency, which can be configured to 5 frames per second, 1 frame per second, 1 frame per 2 seconds, 1 frame per 5 seconds, or a longer interval to meet different application requirements; b) The video gateway pre-processes the captured frame images, including denoising, color adjustment, and resolution scaling, to optimize image quality and reduce the computational load of the subsequent AI parser; c) The processed frame images are transmitted to the AI ​​parser according to the set queue rules to ensure the order and timeliness of the images to ensure the continuity and accuracy of the AI ​​algorithm.

[0012] Furthermore, in step 3, the frame image is processed by an AI parser, including: a) Select appropriate algorithm modules from the preset deep learning model library according to the deployment location and application scenarios of the camera equipment. The algorithm modules include but are not limited to face recognition, license plate recognition, behavior analysis, object detection, etc.; b) After receiving the frame image from the video gateway, the AI ​​parser calls the corresponding algorithm module according to the preset algorithm parameters to identify and analyze the specific target or event in the frame image; c) During the recognition process, the AI ​​parser performs multi-layer filtering and weighted processing on the recognition results to eliminate false positives and low-confidence results, and generates standardized event data for the recognition events that meet the conditions. The event data includes information such as the time of event occurrence, recognition type, and confidence level; d) Push the generated event data to the receiving center and the push center for further processing and distribution.

[0013] Furthermore, the storage and push process of event data in step 4 includes: a) After the AI ​​parser generates event data, the receiving center stores the event data in a distributed database based on the type and urgency of the event. The database is partitioned according to the geographical location and type of the event to improve query efficiency and scalability of data storage; b) The push center obtains the event data to be pushed from the receiving center according to the pre-configured push rules, and selects the optimal push path through the Gateway service of the edge node according to the event type, location and network conditions of the push target. The path selection is based on the principle of optimal geographical location and lowest network latency; c) During the push process, if a network interruption occurs or the target system cannot receive the message, the push center will automatically retry the push. After multiple failed retries, the event data will be saved to local storage and an alarm will be generated to notify the administrator for manual intervention.

[0014] Furthermore, in step 5, the service nodes of the video gateway and the AI ​​parser are dynamically expanded, including: a) When the number of camera devices or the concurrent video stream volume increases and exceeds the processing capacity of the existing video gateway and AI parser, the system automatically triggers the expansion mechanism and dynamically increases service nodes based on containerization technology; b) The expansion mechanism monitors the CPU, memory and network IO usage of existing nodes. When the usage of any resource exceeds a preset threshold (e.g., 80%), the system automatically calls container orchestration tools such as Kubernetes or Docker Swarm to create a new video gateway or AI parser instance; c) The newly added service node automatically connects to the existing cluster through the load balancing strategy, and reallocates some camera devices to the new node according to the predetermined camera device allocation rules to balance the load of each node and prevent the occurrence of system bottlenecks; d) During the expansion process, the system continuously monitors the operating status of the newly added nodes and dynamically adjusts the number of nodes based on the actual business volume to ensure efficient use of system resources and high availability of services.

[0015] The visual network AI computing system based on SIP signaling and video gateway is applicable to the visual network AI computing method based on SIP signaling and video gateway, and is characterized in that it includes: Signaling server, video gateway, AI parser, receiving center, push center and management node; The signaling server is used to obtain and manage the real-time video stream addresses of camera devices through SIP signaling, and map and store these addresses with the national standard codes of camera devices; The input end of the video gateway is connected to the output end signal of the signaling server. The video gateway is used to access the real-time video stream and process the video stream, including capturing frame images according to a preset frame capture frequency and pushing the frame images to the AI ​​parser; The input end of the AI ​​parser is connected to the output end signal of the video gateway. The AI ​​parser is used to select and call a preset deep learning model according to the deployment location and application scenario of the camera device, perform target or event recognition analysis on the frame image, and generate event data; The input end of the receiving center is connected to the output end signal of the AI ​​parser, and the receiving center is used to store the event data generated by the AI ​​parser and perform partition management on the event data; The input end of the push center is connected to the output end signal of the receiving center. The push center is used to push the event data to the lower-level system or user interface according to the preset push rules, and select the optimal push path through the Gateway service; The management node is used to monitor and manage the operating status of the entire system, including the resource usage of the signaling server, video gateway, AI parser, receiving center, and push center, and automatically triggers expansion or failover mechanisms when system resources are insufficient or a failure occurs.

[0016] Furthermore, the signaling server includes: National standard code generation module and video stream address mapping module; The national standard code generation module is used to generate a unique national standard code when the camera device is connected to the system. The national standard code includes the geographical location, device type and other identification information of the camera device; The video stream address mapping module is used to map the national standard code of the camera device to its corresponding real-time video stream address, and store the mapping result in the signaling server.

[0017] The video gateway includes: Video stream access module, frame image capture module and image preprocessing module; The video stream access module is used to access and manage the real-time video stream obtained from the signaling server; The frame image interception module is used to intercept frame images of the video stream according to a preset frame interception frequency; The image preprocessing module is used to perform denoising, color adjustment and resolution scaling on the captured frame images to optimize image quality and reduce the computational load of the AI ​​parser.

[0018] Furthermore, the AI ​​parser includes: Algorithm selection module, target recognition module and event data generation module; The algorithm selection module is used to select a suitable algorithm module from the deep learning model library according to the deployment location and application scenario of the camera device; The target recognition module is used to receive the frame image and call the selected algorithm module to perform target or event recognition analysis on the frame image; The event data generation module is used to perform multi-layer filtering and weighted processing on the results of the identification analysis to generate standardized event data; The push center includes: Event storage module and path selection module; The event storage module is used to receive and store event data generated by the AI ​​parser; The path selection module is used to select the optimal push path through the Gateway service according to the preset push rules and network conditions, and push the event data to the lower-level system or user interface.

[0019] Furthermore, the management node includes: Resource monitoring module, fault detection module and expansion control module; The resource monitoring module is used to monitor the CPU, memory, network IO and other resource usage of the signaling server, video gateway, AI parser, receiving center and push center; The fault detection module is used to detect faults or abnormal conditions within the system and to track faults through logging and monitoring systems; The expansion control module is used to automatically trigger expansion based on containerization technology when resource usage exceeds a preset threshold, and dynamically increase service nodes to ensure high availability and processing performance of the system.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention provides a visual network AI computing method and system based on SIP signaling and video gateway. Through SIP signaling services and video gateways, the system can support the access and management of a large number of camera devices, and each device can generate a unique national standard code to ensure the flexibility and scalability of access; the video gateway can access and process video streams in real time, and use the preset frame capture frequency and image preprocessing functions to significantly reduce the computational burden of subsequent AI analysis, thereby improving the overall processing efficiency of the system.

[0021] 2. The present invention provides a visual network AI computing method and system based on SIP signaling and video gateway. The AI ​​parser uses a deep learning model to perform real-time analysis of video frame images, and can dynamically select the most suitable algorithm module according to the application scenario to achieve efficient and accurate target recognition and event detection; the receiving center and the push center manage event data through a distributed architecture, and select the optimal push path according to preset rules to ensure that event data can be transmitted to the lower-level system or user interface in a timely and secure manner.

[0022] 3. The present invention provides a visual network AI computing method and system based on SIP signaling and video gateway. The system realizes dynamic expansion of video gateway and AI parser through containerization technology, and combines load balancing strategy to ensure high availability and stability of the system even under high concurrency conditions. The management node monitors the system resource usage in real time, and automatically triggers failover and resource expansion when a failure occurs, so as to ensure the continuous operation and service quality of the system.

[0023] 4. The present invention provides a visual network AI computing method and system based on SIP signaling and video gateway. Through the provincial central node, the system can achieve unified management of multiple clusters, support cross-regional camera equipment allocation and task execution, and enhance the flexibility and expansion capability of the system. The system has a built-in complete fault handling mechanism, including failover, automatic recovery and multiple retry strategies, to ensure that even in abnormal situations, the system can quickly recover and maintain data consistency and security. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a schematic diagram of the steps of the present invention; Figure 2 It is a schematic diagram of gateway request forwarding of the present invention; Figure 3 Issue a schematic diagram for the task of the present invention; Figure 4 This is a schematic diagram of the internal logic of the AI ​​parser of the present invention; Figure 5 This is a schematic diagram of event push according to the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention will be more clearly and completely explained below through description of preferred embodiments of the present invention in combination with the accompanying drawings.

[0027] like Figure 1 As shown, the present invention is specifically: Step 1: Get the real-time video stream addresses of multiple camera devices. The video stream addresses are obtained from the camera devices through the SIP signaling service and registered to the signaling server. The signaling server completes the mapping and storage of the video stream addresses according to the national standard coding of the camera devices. Step 2: Use the video gateway to access and process the video stream. The processing includes intercepting frame images of the video stream according to a predetermined frame interception frequency. The frame interception frequency is configured according to specific application requirements, and the intercepted frame images are pushed to the AI ​​parser through the video gateway. Step 3: Use the AI ​​parser to perform algorithm processing on the frame image in step 2. The algorithm includes a preset deep learning model. The deep learning model dynamically selects an algorithm module according to the location and application scenario of the camera device. The algorithm module analyzes the frame image to identify specific targets or events, and pushes the analysis results to the receiving center and the push center. Step 4: According to the analysis results of the AI ​​parser, if the event meets the preset conditions, the event data is stored in the receiving center, and the event is pushed to the specified lower-level system or user interface through the push center. The push process is completed through the Gateway service of the edge node to ensure the timeliness and reliability of event push; Step 5: When the number of camera devices or the number of concurrent video streams increases, dynamically expand the service nodes of the video gateway and AI parser. The expansion process is achieved through containerized deployment and load balancing strategies to ensure high availability and processing performance of the system. Step 6: In the event of device failure or network anomaly, the failover mechanism is automatically triggered to reallocate camera devices to other normally operating video gateways and AI parser nodes, and fault tracking and recovery are performed through logging and monitoring systems to ensure system stability and continuity.

[0028] As a specific implementation method, In microservices, video management, video collaboration, push center, and receiving center are management services. Management services are manually expanded by operation and maintenance based on the usage of hardware resources. The gateway controller, video gateway, AI parser, inference gateway, and algorithm capabilities are deployed in containers. The gateway controller, video gateway, and AI parser are started according to the tenant dimension of the SaaS platform. Gateway controller: A tenant starts a fixed gateway controller, because the gateway controller is only used for load calculation and is not designed for large business processing. It receives the service heartbeats of the video gateway and AI parser under the same tenant, records the survival status, and is responsible for allocating new camera access to which video gateway; Video Gateway: The number of nodes is calculated according to the number of cameras that pull streams and capture frames. The resource specifications of a node are 2.5 cores, 8G memory, and connected to 24 stream-pulling and frame-cutting cameras. The number of video gateway nodes under a tenant = the total number of cameras turned on / 24. The number of nodes is dynamically adjusted according to the number of camera connections. When a camera is turned off, this camera will be removed from the corresponding video gateway. The current video gateway will not be recycled until there is no stream-pulling and frame-cutting camera. If there is a new stream-pulling and frame-cutting camera, it will be allocated to the video gateway with the least number of free access routes.; AI parser: Calculated in a 1:1 ratio with the number of video gateway nodes, the resource specifications are 1.5 cores and 3G memory as the video gateway is scaled up and down synchronously; Inference Gateway: Dynamically scale up and down based on CPU and memory usage. If the CPU and memory usage reaches 80%, it will be dynamically scaled up, and if it is less than 20%, it will be scaled down. The resource specifications are 4 cores and 8G memory. Algorithm service: Dynamically scale up and down based on GPU and memory usage. If the GPU and memory usage reaches 80%, it will be dynamically scaled up, and if it is less than 20%, it will be scaled down. Therefore, the failure of the management node does not affect the current production business operation, but only affects the access of new cameras. The access of new cameras and the opening of stream capture need to be operated through the interface. At the same time, it affects the storage and push of event data. If the receiving service and the push service fail, the events will be accumulated in Kafka. No loss will be caused. After the service is restored, the accumulated data will be consumed first. Adding push and receiving service nodes will accelerate the consumption of accumulated data and recover timeliness. At the same time, the execution service will perform tenant isolation, and some failures will only affect the current tenant. A management node failure will only affect one edge cluster and will not affect the entire platform.

[0029] As a specific implementation method, Figure 2 As shown: OpenAPI call: The openAPI call address is provided by the platform. In the multi-cluster management interface, the service gateway address of each edge is maintained. When the operation and maintenance personnel provide the openAPI call address, they will give the edge service gateway call address closest to the application based on the network conditions of the city. If the provincial gateway address or the gateway address of other cities is given, after receiving the request, the system will determine whether it belongs to its own city. If it does not, it will be forwarded directly to the provincial gateway, and then forwarded by the provincial gateway to the corresponding city gateway, because the networks between cities are not interoperable, but the city and provincial networks are interoperable. There are two advantages to doing this: ① When the platform provides the call address to the outside, only one address needs to be connected; ② If the business development of the application requires calling multiple clusters, it can be forwarded within the gateway without modifying the address. It also shields the complex forwarding logic and does not invade specific business functions.

[0030] 2. Video event push. The generation of events follows edge production, edge storage, and edge push. When the application is accessed, the push address will be configured on the platform. Generally, the application in this city will only use the camera data in this city, so no shortest path is calculated. The path should naturally be the shortest path. If a city application has cross-city camera business, a push address will be configured in the clusters of the two cities, and the two city clusters will push data to the application respectively.

[0031] As a specific implementation method, Figure 3 As shown: The task dispatching function is dispatched from the provincial node to the edge node. After the provincial node completes the operation of the management function landing in the MySQL database, the specific execution still depends on the update of the redis cache value. Therefore, we developed a custom annotation @CloudEdgeCoordinate based on the basic operation method of redis, and used AOP to intercept the cache operation and write it to Kafka after interception. The specific logic is as follows: ① After the provincial node is executed, it intercepts the redis operation and writes it into the task delivery topic of Kafka. This is equivalent to the task broadcast of the provincial node to the edge node; ② All edge ends listen to the task dispatch topic of Kafka. Different edge ends use different consumer groups to ensure that each consumer group can consume the complete task message, start task consumption and execute it. According to the parameter information, tasks belonging to the edge end are retained for execution. Tasks that do not belong to the edge end are directly discarded; ③The success / failure result of the task execution is written to the task receipt topic by the edge. If the task execution fails, the Kafka monitoring of the current edge consumer group is directly stopped, and the consumption of task messages is stopped. The receipt message is consumed by the provincial node and stored in Clickhouse; ④ Display the task delivery status through the front-end interface, view the list of failed tasks, and manually click the button to trigger a retry. The provincial node rewrites the task information into the task retry topic, which is consumed by different consumer groups on the edge. Messages that do not belong to you are directly discarded, and messages that belong to you are executed; ⑤After successful execution, start the monitoring of your current consumer group again and start consuming new task messages. Use KafkaListenerEndpointRegistry to manually control the consumption stop and start of the entire consumer group.

[0032] As a specific implementation method, Kafka service is deployed in a cluster of at least 3 nodes to ensure reliability. Message loss can be retrieved through ClickHouse. When a task is sent to Kafka, a log message will be recorded in ClickHouse, recording the complete message body, sending time, task ID and other information. It will be compared with the receipt information query result. If the sent task has a successful receipt, the task is completed. If the receipt fails, you can click Retry. If there is no receipt, it is considered to be in progress. Task sending involves 3 topi; ① The task sends a topic with only one partition to ensure the order of message consumption. There are multiple provincial management node producers, one collaborative service in each city, and different consumer groups in each city. Messages are saved in Kafka for 7 days by default. ② The task receipt topic will have multiple partitions. Considering that the order of receipt information does not need to be guaranteed and it can be consumed normally, the number of partitions matches the number of provincial management end nodes. Each city and prefecture collaborates to produce messages, and the provincial node management service consumes the messages and stores them in Clickhouse. The messages are stored in Kafka for 7 days by default. ③ The failed task retry topic has only one partition to ensure the order of message consumption. There are multiple provincial management node producers, one collaborative service in each city, and different consumer groups in each city. Messages are saved in Kafka for 7 days by default. As a specific implementation method, Figure 4 As shown in the figure, the main function of the AI ​​parser is to assemble the parameters required to call the algorithm, complete the algorithm call, and then disassemble the algorithm result and determine whether it meets the conditions to become an event.

[0033] As a specific implementation method, the algorithm call configures the input parameters of the algorithm interface through the front-end interface. The input parameter json contains a placeholder of {image}. When the call is actually made, the obtained image is converted into a base64 string to obtain a string, and the base64 string is replaced with {image} to obtain the complete call input parameter, and an http call is initiated. The return result of the algorithm is a piece of json. By parsing and flattening the json, the flattened results are compared and judged according to the filtering conditions configured in the front-end interface to obtain the final result of whether the conditions are met. If not, it is discarded, and if it is satisfied, it is saved and pushed. For example, in the flame recognition scenario, most of the time, the pictures taken are without flames. After disassembling the algorithm results, it is determined whether the fire is recognized. If there is no fire, the result and picture will be directly discarded, and only the event with fire will be left for subsequent push logic.

[0034] As a specific implementation method, the technical details of API forwarding are as follows: openAPI forwarding is implemented through gateway, and the input parameters are parsed through RoutePredicateHandlerMapping. The cluster ID is obtained from the input parameters, and the cluster ID is used to determine whether it belongs to the current cluster processing. If so, it is directly forwarded to the management service. If not, it is directly transferred to the central node gateway service for processing. The central node gateway handles the same logical forwarding. Requests forwarded to the management end obtain the node list through nacos to perform client load.

[0035] As a specific implementation method, fast transmission and reliable reception of event data are achieved as follows Figure 5 As shown: ① The networks of all clusters are interconnected, and the data generated by the edge is pushed to the corresponding edge application by the edge. If an application needs data from multiple cities, the push services of multiple cities will directly push it to the target application. It can be forwarded without going through provincial nodes; ② All edge networks are not connected to each other and are only connected to the provincial node network. The event data generated by the edge is pushed to the edge application, which is the shortest path push. If an application requires data from a city, the data from the city is pushed to the provincial node, and then pushed to the target city by the provincial node.

[0036] As a specific implementation, the detailed design of the data storage architecture is as follows: ① MySQL is used as the relational database. Each prefecture-level city or province has an independent MySQL database, with two nodes serving as primary and backup storage. ② ClickHouse is selected as the distributed database. Each prefecture-level city or province has an independent cluster, with 3 shards, 1 copy and 6 nodes in a cluster. Data cleaning method: Add the expire_time field to the table, set the expiration time when inserting data, and clickhouse automatically cleans up expired data; MergeTree storage engine, based on the event di field modulo 18 as the partition strategy. The default data compression method of MergeTree is LZ4, with a compression ratio of 35%; ③ Distributed file storage system ceph; each municipal or provincial node has an independent ceph cluster, and each cluster consists of 6 nodes. We have developed a self-developed storage acceleration solution, using SSD hard disks to store hot data within 1 hour. Because most video events have high business timeliness, they need to be processed quickly after the event occurs. The access frequency of event image data within 1 hour is high. Placing it in the SSD hot storage area can improve access efficiency and reduce the retrieval of the overall data. All data is stored on HDD disks, and a self-developed ceph cleanup acceleration plug-in is used. Image data is placed in a bucket based on the day dimension. The bucket name will carry date information. If the bucket name matches the cleanup requirements, the bucket will be deleted directly to avoid retrieval matching of a single file and improve data cleanup efficiency.

[0037] As a specific implementation method, the technical details of monitoring and managing cluster resources are as follows: k8s cluster monitoring is collected and monitored through Prometheus, and visualized through Rancher interface and Grafana. Video platform monitoring is completed through skywalking. Each municipal node or provincial node has a skywalking service node, which is visualized by the monitoring interface provided by skywalking. At the same time, in the multi-cluster management list, the access address of the skywalking visualization interface of each cluster is entered, and it can be directly jumped to the display in the provincial management page.

[0038] As a specific implementation method, the specific steps of camera access and management, as well as the algorithm or strategy for selecting the nearest edge cluster are as follows: The camera is connected to the system through the management node. The platform provides the following four functions: National standard code application: Support national standard code application, apply for the national standard code and channel code of the device according to the area where the device is located, and the code needs to be generated according to the national standard code specification combined with the area; National standard coding management: Supports the addition of national standard coding management, including three situations: platform access, national standard camera access, and NVR device access; National standard code download: support the download of the national standard code applied for and provide it to the equipment access personnel in the prefecture and city, and access the equipment according to the specifications; National standard code recycling: After the code is generated, it is valid for 3 months (configurable). Once it expires, the code cannot be accessed and needs to be reapplied. The code needs to be recycled and can be used for the next application. After the operation and maintenance personnel obtain the assigned national standard code, they complete the configuration on the specific camera device.

[0039] The following is an implementation of a complete visual network AI computing system based on SIP signaling and video gateway: 1. System Architecture The present invention relates to a three-level visual network AI computing system based on SIP signaling and video gateway, including a signaling server, a video gateway, an AI parser, a receiving center, a push center and a management node. The system is used for large-scale camera device access and video stream processing, supports multi-level cluster management, and realizes efficient AI visual computing and event push.

[0040] Signaling server: The signaling server is responsible for the access management of camera devices, and generates a unique national standard code through SIP signaling to ensure the uniqueness of the camera device in the system. The signaling server maps the physical address of the camera device to the corresponding video stream address and maintains these mapping relationships. Through the signaling server, the real-time video stream address of each camera device can be accurately obtained and used for subsequent processing of the video stream in the system.

[0041] Video Gateway: The video gateway is one of the core modules in the system, mainly used for accessing and processing video streams. After obtaining the video stream address from the signaling server, the video gateway accesses the video stream and intercepts the frame images of the video stream according to the preset frame interception frequency. The intercepted frame images are preprocessed (such as denoising, color adjustment, and resolution scaling) to optimize the image quality and reduce the computational load of the subsequent AI parser.

[0042] AI Parser: The AI ​​parser receives the frame images transmitted by the video gateway and analyzes the images using the preset deep learning model library. According to the deployment location and application scenario of the camera equipment, the AI ​​parser automatically selects the most appropriate algorithm module (such as face recognition, license plate recognition, behavior analysis, etc.) and performs recognition processing on the image. During the recognition process, the AI ​​parser performs multi-layer filtering and weighted processing on the recognition results to ensure the accuracy of the results. The event data finally generated includes information such as the time of event occurrence, recognition type, and confidence, and is pushed to the receiving center and the push center.

[0043] Receiving Center: The receiving center is used to store the event data generated by the AI ​​parser and partition the data according to the event's geographic location, event type, and other information. The data storage in the receiving center uses a distributed database, which can efficiently manage large-scale event data and ensure good performance and scalability when querying.

[0044] Push Center: The push center pushes the event data stored in the receiving center to the designated lower-level system or user interface according to the preset push rules. The push center selects the optimal push path through the Gateway service to ensure the timeliness and reliability of event push. During the push process, if a network interruption occurs or the target system cannot receive data, the push center will automatically retry the push, and after multiple failed retries, record the exception and notify the administrator.

[0045] Management Node: The management node is used to monitor and manage the operating status of the entire system, and monitor the resource usage of the signaling server, video gateway, AI parser, receiving center, and push center in real time (including CPU, memory, network IO, etc.). When insufficient system resources or a failure is detected, the management node will automatically trigger the expansion mechanism or failover mechanism, dynamically increase or decrease service nodes through containerization technology, and ensure the high availability and stability of the system.

[0046] 2. System Workflow Camera device access: The camera device accesses the system through SIP signaling. The signaling server generates a unique national standard code and maps it to the video stream address. The signaling server stores this mapping information and provides it to the video gateway and AI parser for use.

[0047] Video stream processing: The video gateway accesses the video stream of the camera device, captures frame images according to the set frame capture frequency, and pre-processes the images. The pre-processed images are transmitted to the AI ​​parser through the processing queue of the video gateway.

[0048] AI analysis and event generation: After receiving the frame image, the AI ​​parser calls the corresponding deep learning algorithm module to identify the target in the image. The algorithm selection in the recognition process is based on the specific application scenario of the camera device. The AI ​​parser filters and weights the recognition results to generate standardized event data.

[0049] Event data storage and push: After the event data is generated, the receiving center stores and partitions the data according to the characteristics of the event. The push center selects the optimal path based on the preset push rules and pushes the event data to the lower-level system or user interface.

[0050] System monitoring and troubleshooting: The management node continuously monitors the system resource usage and automatically expands or switches service nodes when resource usage is too high or a failure occurs to ensure efficient operation of the system. All operations and events in the system are recorded to facilitate subsequent troubleshooting and performance optimization.

[0051] 3. System Expansion and Optimization Multi-level cluster management: The system supports the management and deployment of multi-level clusters, and manages clusters in multiple edge cities in a unified manner through the container management service (such as Kubernetes) of the provincial central node. Camera devices will be assigned to the nearest edge cluster for processing based on their actual deployment location to reduce the delay of video transmission.

[0052] Load balancing and fault tolerance mechanism: The video gateway and AI parser in the system are deployed in containers, which supports dynamic expansion or reduction of the number of nodes according to load conditions. The system also introduces a load balancing strategy to ensure load balancing of camera devices and improve the processing efficiency and stability of the system.

[0053] Failover and automatic recovery: In the event of a device failure or network anomaly, the system will automatically trigger a failover mechanism to reallocate the camera device to another normally functioning node. At the same time, through the logging and monitoring system, administrators can track and recover from failures in real time to ensure the continuous operation of the system.

[0054] 4. Practical application scenarios of the system City security monitoring: The system can be used for large-scale urban security monitoring. By integrating a large number of camera devices, it can monitor the security situation of various areas in the city in real time. The AI ​​parser can identify potential security threats (such as suspicious behavior, illegal parking, etc.) and generate alarm information in a timely manner.

[0055] Traffic Management and Monitoring: In the field of traffic management, the system can monitor and analyze traffic flow in real time. By identifying license plate numbers and monitoring traffic flow, the system can help traffic management departments respond quickly and optimize traffic flow distribution.

[0056] Smart Building and Park Management: The system can also be used for smart building and campus management. By integrating various sensors and cameras, it can monitor the security situation inside the building in real time and make intelligent responses as needed, such as triggering alarms, adjusting air conditioning and lighting, etc.

[0057] 5. System Expansion Capabilities Cross-region deployment: The system supports cross-regional deployment and centrally manages edge clusters in multiple cities through provincial central nodes. Each cluster can operate independently according to actual needs, while supporting cross-regional camera data sharing and event push.

[0058] Flexible configuration of AI algorithms: The algorithm library in the AI ​​parser can be flexibly configured according to different application scenarios. Users can customize or update algorithm modules according to actual needs to ensure that the system's analysis capabilities are always in the best state.

[0059] Multiple protocols support: The system supports multiple video streaming protocols (such as RTSP, RTMP), and new protocol support can be added as needed to adapt to different camera devices and application scenarios.

[0060] Through the above implementation, the three-level visual network AI computing system based on SIP signaling and video gateway of the present invention can effectively solve the device access bottleneck, network IO bottleneck, video processing delay and system stability problems in the prior art, realize efficient access and AI computing of large-scale video surveillance equipment, and provide intelligent solutions for various application scenarios.

[0061] The above specific implementations are only descriptions of the preferred implementations of the present invention, and do not limit the protection scope of the present invention. Without departing from the design concept and spirit of the present invention, various modifications, substitutions and improvements made by ordinary technicians in this field to the technical solution of the present invention based on the text description and drawings provided by the present invention should all fall within the protection scope of the present invention. The protection scope of the present invention is determined by the claims.

Claims

1. A visual network AI computing method based on SIP signaling and video gateway, characterized in that: The method comprises: Step 1: Get the real-time video stream addresses of multiple camera devices. The video stream addresses are obtained from the camera devices through the SIP signaling service and registered to the signaling server. The signaling server completes the mapping and storage of the video stream addresses according to the national standard coding of the camera devices. Step 2: Use the video gateway to access and process the video stream. The processing includes intercepting frame images of the video stream according to a predetermined frame interception frequency. The frame interception frequency is configured according to specific application requirements, and the intercepted frame images are pushed to the AI ​​parser through the video gateway. Step 3: Use the AI ​​parser to perform algorithm processing on the frame image in step 2. The algorithm includes a preset deep learning model. The deep learning model dynamically selects an algorithm module according to the location and application scenario of the camera device. The algorithm module analyzes the frame image to identify specific targets or events, and pushes the analysis results to the receiving center and the push center. Step 4: According to the analysis results of the AI ​​parser, if the event meets the preset conditions, the event data is stored in the receiving center, and the event is pushed to the specified lower-level system or user interface through the push center. The push process is completed through the Gateway service of the edge node to ensure the timeliness and reliability of event push; Step 5: When the number of camera devices or the number of concurrent video streams increases, dynamically expand the service nodes of the video gateway and AI parser. The expansion process is achieved through containerized deployment and load balancing strategies to ensure high availability and processing performance of the system. Step 6: In the event of device failure or network anomaly, the failover mechanism is automatically triggered to reallocate camera devices to other normally operating video gateways and AI parser nodes, and fault tracking and recovery are performed through logging and monitoring systems to ensure system stability and continuity.

2. The visual network AI computing method based on SIP signaling and video gateway according to claim 1 is characterized in that: In step 1, obtain the real-time video stream addresses of multiple camera devices, including: a) When a camera device is connected to the system, a unique national standard code is generated for each camera device through the SIP signaling service. The national standard code includes the geographical location, device type and other identification information of the device; b) In the signaling server, the physical address of the camera device is mapped to the corresponding video stream address using the national standard code, and the video stream address includes a URL link under the RTSP or RTMP protocol; c) When the national standard code of the camera device changes, the mapping relationship in the signaling server is dynamically updated, and the relevant service nodes are broadcast and notified within the system to ensure that the address mapping of the video stream is accurate.

3. The visual network AI computing method based on SIP signaling and video gateway according to claim 1 is characterized in that: In step 2, the video stream is processed by the video gateway, including: a) The real-time video stream obtained from the signaling server is accessed in the video gateway, and frame images of the video stream are captured according to the preset frame capture frequency. The frame capture frequency can be configured to 5 frames per second, 1 frame per second, 1 frame per 2 seconds, 1 frame per 5 seconds, or a longer interval to meet different application requirements; b) The video gateway pre-processes the captured frame images, including denoising, color adjustment, and resolution scaling, to optimize image quality and reduce the computational load of the subsequent AI parser; c) The processed frame images are transmitted to the AI ​​parser according to the set queue rules to ensure the order and timeliness of the images to ensure the continuity and accuracy of the AI ​​algorithm.

4. The visual network AI computing method based on SIP signaling and video gateway according to claim 1 is characterized in that: In step 3, the AI ​​parser is used to perform algorithmic processing on the frame image, including: a) Select algorithm modules from the preset deep learning model library according to the deployment location and application scenarios of the camera equipment. The algorithm modules include but are not limited to face recognition, license plate recognition, behavior analysis and object detection; b) After receiving the frame image from the video gateway, the AI ​​parser calls the corresponding algorithm module according to the preset algorithm parameters to identify and analyze the specific target or event in the frame image; c) During the recognition process, the AI ​​parser performs multi-layer filtering and weighted processing on the recognition results to eliminate false positives and low-confidence results, and generates standardized event data for the recognition events that meet the conditions. The event data includes the time of event occurrence, recognition type and confidence information; d) Push the generated event data to the receiving center and the push center for further processing and distribution.

5. The method according to claim 1, characterized in that The storage and push process of event data in step 4 includes: a) After the AI ​​parser generates event data, the receiving center stores the event data in a distributed database based on the type and urgency of the event. The distributed database is partitioned according to the geographic location and type of the event to improve query efficiency and scalability of data storage; b) The push center obtains the event data to be pushed from the receiving center according to the pre-configured push rules, and selects the optimal push path through the Gateway service of the edge node based on the event type, location, and network conditions of the push target. The push path selection is based on the principle of optimal geographical location and lowest network latency; c) During the push process, if a network interruption occurs or the target system cannot receive the message, the push center will automatically retry the push. After multiple failed retries, the event data will be saved to local storage and an alarm will be generated to notify the administrator for manual intervention.

6. The visual network AI computing method based on SIP signaling and video gateway according to claim 1 is characterized in that: In step 5, the service nodes of the video gateway and AI parser are dynamically expanded, including: a) When the number of camera devices or the concurrent video stream volume increases and exceeds the processing capacity of the existing video gateway and AI parser, the system automatically triggers the expansion mechanism and dynamically increases service nodes based on containerization technology; b) The expansion mechanism monitors the CPU, memory, and network IO usage of existing nodes. When the usage of any resource exceeds the preset threshold, the system automatically calls the container orchestration tool to create a new video gateway or AI parser instance; c) The newly added service node automatically connects to the existing cluster through the load balancing strategy, and reallocates some camera devices to the new node according to the predetermined camera device allocation rules to balance the load of each node and prevent the occurrence of system bottlenecks; d) During the expansion process, the system continuously monitors the operating status of the newly added nodes and dynamically adjusts the number of nodes based on the actual business volume to ensure efficient use of system resources and high availability of services.

7. A visual network AI computing system based on SIP signaling and video gateway, applicable to a visual network AI computing method based on SIP signaling and video gateway as described in any one of claims 1 to 6, characterized in that: include: Signaling server, video gateway, AI parser, receiving center, push center and management node; The signaling server is used to obtain and manage the real-time video stream address of the camera device through SIP signaling, and map and store the real-time video stream address with the national standard code of the camera device; The input end of the video gateway is connected to the output end signal of the signaling server. The video gateway is used to access the real-time video stream and process the video stream, including capturing frame images according to the preset frame capture frequency and pushing the frame images to the AI ​​parser; The input end of the AI ​​parser is connected to the output end signal of the video gateway. The AI ​​parser is used to select and call the preset deep learning model according to the deployment location and application scenario of the camera device, identify and analyze the target or event of the frame image, and generate event data; The input end of the receiving center is connected to the output end signal of the AI ​​parser, and the receiving center is used to store the event data generated by the AI ​​parser and perform partition management on the event data; The input end of the push center is connected to the output end signal of the receiving center. The push center is used to push event data to the lower-level system or user interface according to the preset push rules, and select the optimal push path through the Gateway service; The management node is used to monitor and manage the operating status of the entire system, including the resource usage of the signaling server, video gateway, AI parser, receiving center, and push center, and automatically triggers expansion or failover mechanisms when system resources are insufficient or a failure occurs.

8. The visual network AI computing system based on SIP signaling and video gateway according to claim 7 is characterized in that: The signaling server includes: National standard code generation module and video stream address mapping module; The national standard code generation module is used to generate a unique national standard code when the camera device is connected to the system. The national standard code includes the geographical location, device type and other identification information of the camera device; The video stream address mapping module is used to map the national standard code of the camera device to its corresponding real-time video stream address, and store the mapping result in the signaling server; The video gateway includes: Video stream access module, frame image capture module and image preprocessing module; The video stream access module is used to access and manage the real-time video stream obtained from the signaling server; The frame image interception module is used to intercept frame images of the video stream according to a preset frame interception frequency; The image preprocessing module is used to perform denoising, color adjustment and resolution scaling on the captured frame images to optimize image quality and reduce the computational load of the AI ​​parser.

9. The visual network AI computing system based on SIP signaling and video gateway according to claim 7, characterized in that: AI parser includes: Algorithm selection module, target recognition module and event data generation module; The algorithm selection module is used to select a suitable algorithm module from the deep learning model library according to the deployment location and application scenario of the camera device; The target recognition module is used to receive the frame image and call the selected algorithm module to perform target or event recognition analysis on the frame image; The event data generation module is used to perform multi-layer filtering and weighted processing on the results of the identification analysis to generate standardized event data; The push center includes: Event storage module and path selection module; The event storage module is used to receive and store event data generated by the AI ​​parser; The path selection module is used to select the optimal push path through the Gateway service according to the preset push rules and network conditions, and push the event data to the lower-level system or user interface.

10. The visual network AI computing system based on SIP signaling and video gateway according to claim 7, characterized in that: The management nodes include: Resource monitoring module, fault detection module and expansion control module; The resource monitoring module is used to monitor the CPU, memory and network IO resource usage of the signaling server, video gateway, AI parser, receiving center and push center; The fault detection module is used to detect faults or abnormal conditions within the system and to track faults through logging and monitoring systems; The expansion control module is used to automatically trigger expansion based on containerization technology when resource usage exceeds a preset threshold, and dynamically increase service nodes to ensure high availability and processing performance of the system.

Images